mirror of
https://github.com/pnpm/pnpm.git
synced 2026-05-18 22:02:53 -04:00
fcf95c7faa
Closes #11687. ## What Cache the result of the post-resolution lockfile verification gate (#11583) so repeat installs against an unchanged lockfile skip the per-package registry round trips entirely. Persisted as JSON Lines at `<cacheDir>/lockfile-verified.jsonl`. The cache layer is policy-neutral. Today there's one verifier (`minimumReleaseAge`); future resolver-side verifiers (jsr trust, attestation, …) plug in by declaring their own `policy` slot and `canTrustPastCheck` comparator — no install-side changes. ## Why #11583 re-hits the registry on every install for every locked (name, version) pair. On warm/repeat installs where the lockfile hasn't moved, that's a stack of per-package round trips with nothing to show for them. This change makes the steady-state case effectively free without weakening the protection — the gate still runs in full whenever the lockfile changes, any verifier's policy tightens, or no record exists. ## How ### Cache lookup, in order The cache is **indexed by content hash** so git worktrees with identical lockfile bytes share a cache entry. A secondary path-keyed index drives the same-machine stat shortcut. 1. **`stat()` shortcut** — when a previous record for this exact `lockfilePath` matches today's `size + mtime + inode`, trust the cached hash without reading anything. Zero I/O beyond the stat. Microseconds. 2. **Content lookup** — hash the in-memory lockfile (not the file bytes — we already have the parsed object) and look up by content hash. Catches worktrees (same content, different path) and CI checkouts (same content, reset stat). On hit, append a refreshed path/stat entry so the next install at this path takes the stat shortcut. 3. **Any active verifier rejects the cached `policy`** — run the full gate. 4. **No record** — run the full gate. The in-memory object is hashed with `hashObject` from `@pnpm/crypto.object-hasher` (streaming, key-order-stable). ### Record shape ```json { "lockfile": { "hash": "<sha256 base64>", "path": "/abs/path/to/pnpm-lock.yaml", "size": 154, "mtimeNs": "1736245123000000000", "inode": "12345" }, "verifiedAt": "2026-05-17T...", "policy": { "minimumReleaseAge": 1440 } } ``` `policy` is the union of every active verifier's `policy` contribution. Verifiers checking the same logical policy (e.g. `minimumReleaseAge` honored by multiple registries) name it the same and share the slot — no resolver namespacing. ### File semantics - **Sync fs throughout** — the cache is consulted once before verification fan-out and recorded once after. No concurrent install work to overlap with; keeping the call sites straight-line. - **JSONL appends are atomic** on POSIX/NTFS, so parallel pnpm processes (monorepo installs, CI matrices sharing a cache) write without coordination. Latest record per `(path, hash)` tuple wins on read. - **Bounded file** — capped at ~1000 entries; compaction is triggered by a single `stat()` of the cache file (1.5 MiB byte budget) so we never parse the file on the steady-state path. When triggered, the tail is rewritten via tempfile + rename. - **No record on rejection** — a failing verification deliberately doesn't write a record; the next install must rerun the gate. - **Single hash per install** — the in-memory hash is computed lazily and reused: `tryLockfileVerificationCache` returns the precomputed stat+hash to `recordVerification` on a miss, and the stat-shortcut hit forwards the cached record's hash unchanged. ## Plumbing The verifier contract changed alongside the cache to make this composable without install-side knowledge of each policy: - **`@pnpm/resolving.resolver-base`** — `ResolutionVerifier` is now `{ verify, policy, canTrustPastCheck }` (was a bare function in #11583). Each resolver-side verifier owns its policy snapshot and the comparator that decides whether a cached policy is still trustworthy. - **`@pnpm/resolving.npm-resolver`** — `createNpmResolutionVerifier` returns the new shape: `policy: { minimumReleaseAge }`, `canTrustPastCheck` reads `minimumReleaseAge` from the merged cached bag. - **`@pnpm/resolving.default-resolver`** — `createResolutionVerifier` (singular, returning a combined function) → `createResolutionVerifiers` (plural, returning a `ResolutionVerifier[]`). No combinator; each verifier handles its own protocol short-circuit inside `verify`, so dispatch happens naturally at the install side. - **`@pnpm/installing.client`** — `Client.verifyResolution?` → `Client.resolutionVerifiers: ResolutionVerifier[]`. Same rename propagates through `@pnpm/store.connection-manager`, `@pnpm/testing.temp-store`, and `StrictInstallOptions`. - **`@pnpm/installing.deps-installer`** — new `verifyLockfileResolutionsCache.ts` (`tryLockfileVerificationCache` + `recordVerification`). `verifyLockfileResolutions` takes the verifier list plus `cacheDir` + `lockfilePath` as flat options; the cache fires when both are present, otherwise the gate runs without memoization. The dedup key for in-flight candidates includes a serialization of `resolution` so two entries sharing a (name, version) but pinned via different protocols don't collapse. Breaking but safe — `@pnpm/resolving.npm-resolver` hasn't been released since #11583 introduced the verifier abstraction, so no downstream consumer is on the old shape. ## Tests - **17 unit tests** in `verifyLockfileResolutionsCache.ts`: cache miss/hit, stat shortcut, size mismatch falling through to hash lookup, hash-fallback on reset stat, content change with matching size, stricter/weaker policy, missing-field policy rejection, multi-verifier policy merge (shared field stored once), worktree case (same content, different path), JSONL append semantics, malformed-line tolerance. - **12 integration tests** in `verifyLockfileResolutions.ts`: dedup of peer/patch-suffix variants, distinct-resolution dedup at the same (name, version), stable violation ordering, the 20-entry cap, multi-verifier fan-out (first failure wins), cache short-circuit on a passing run, no cache write on a rejecting run, empty-verifier-list passthrough. - **1 e2e test** in `pnpm/test/install/minimumReleaseAge.ts`: bundled CLI plumbing — install once to seed the lockfile, enable `minimumReleaseAge` + `cacheDir`, install again, assert the cache file lands at `<cacheDir>/lockfile-verified.jsonl` with the documented record shape. - Existing `minimumReleaseAge` (13) and `frozenLockfile` (12) suites still pass.
75 lines
2.3 KiB
TypeScript
75 lines
2.3 KiB
TypeScript
import type { CustomResolver } from '@pnpm/hooks.types'
|
|
import type { InstallOptions } from '@pnpm/installing.deps-installer'
|
|
import { REGISTRY_MOCK_PORT } from '@pnpm/registry-mock'
|
|
import type { ResolutionVerifier } from '@pnpm/resolving.resolver-base'
|
|
import type { StoreController } from '@pnpm/store.controller-types'
|
|
import { createTempStore } from '@pnpm/testing.temp-store'
|
|
import type { Registries } from '@pnpm/types'
|
|
|
|
const registry = `http://localhost:${REGISTRY_MOCK_PORT}/`
|
|
|
|
export function testDefaults<T> (
|
|
opts?: T & {
|
|
fastUnpack?: boolean
|
|
storeDir?: string
|
|
prefix?: string
|
|
registries?: Registries
|
|
customResolvers?: CustomResolver[]
|
|
minimumReleaseAge?: number
|
|
minimumReleaseAgeStrict?: boolean
|
|
minimumReleaseAgeExclude?: string[]
|
|
},
|
|
resolveOpts?: any, // eslint-disable-line
|
|
fetchOpts?: any, // eslint-disable-line
|
|
storeOpts?: any // eslint-disable-line
|
|
): InstallOptions &
|
|
{
|
|
cacheDir: string
|
|
registries: Registries
|
|
storeController: StoreController
|
|
storeDir: string
|
|
resolutionVerifiers: ResolutionVerifier[]
|
|
} &
|
|
T {
|
|
// Forward minimumReleaseAge policy into the Client so it builds the
|
|
// matching ResolutionVerifier; tests that set these options exercise the
|
|
// same code path the CLI command would.
|
|
const policyClientOptions = {
|
|
...(opts?.minimumReleaseAge != null ? { minimumReleaseAge: opts.minimumReleaseAge } : {}),
|
|
...(opts?.minimumReleaseAgeStrict != null ? { minimumReleaseAgeStrict: opts.minimumReleaseAgeStrict } : {}),
|
|
...(opts?.minimumReleaseAgeExclude != null ? { minimumReleaseAgeExclude: opts.minimumReleaseAgeExclude } : {}),
|
|
}
|
|
const { storeController, storeDir, cacheDir, resolutionVerifiers } = createTempStore({
|
|
...opts,
|
|
clientOptions: {
|
|
...(opts?.registries != null ? { registries: opts.registries } : {}),
|
|
customResolvers: opts?.customResolvers,
|
|
...policyClientOptions,
|
|
...resolveOpts,
|
|
...fetchOpts,
|
|
},
|
|
storeOptions: storeOpts,
|
|
})
|
|
const result = {
|
|
cacheDir,
|
|
registries: {
|
|
default: registry,
|
|
},
|
|
storeController,
|
|
storeDir,
|
|
resolutionVerifiers,
|
|
...opts,
|
|
} as (
|
|
InstallOptions &
|
|
{
|
|
cacheDir: string
|
|
registries: Registries
|
|
storeController: StoreController
|
|
storeDir: string
|
|
resolutionVerifiers: ResolutionVerifier[]
|
|
} &
|
|
T
|
|
)
|
|
return result
|
|
}
|