From fcf95c7faa891c5eb2b216e0a836f0257f611d7a Mon Sep 17 00:00:00 2001 From: Zoltan Kochan Date: Sun, 17 May 2026 13:07:24 +0200 Subject: [PATCH 001/169] perf: cache the post-resolution lockfile verification gate (#11691) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #11687. ## What Cache the result of the post-resolution lockfile verification gate (#11583) so repeat installs against an unchanged lockfile skip the per-package registry round trips entirely. Persisted as JSON Lines at `/lockfile-verified.jsonl`. The cache layer is policy-neutral. Today there's one verifier (`minimumReleaseAge`); future resolver-side verifiers (jsr trust, attestation, …) plug in by declaring their own `policy` slot and `canTrustPastCheck` comparator — no install-side changes. ## Why #11583 re-hits the registry on every install for every locked (name, version) pair. On warm/repeat installs where the lockfile hasn't moved, that's a stack of per-package round trips with nothing to show for them. This change makes the steady-state case effectively free without weakening the protection — the gate still runs in full whenever the lockfile changes, any verifier's policy tightens, or no record exists. ## How ### Cache lookup, in order The cache is **indexed by content hash** so git worktrees with identical lockfile bytes share a cache entry. A secondary path-keyed index drives the same-machine stat shortcut. 1. **`stat()` shortcut** — when a previous record for this exact `lockfilePath` matches today's `size + mtime + inode`, trust the cached hash without reading anything. Zero I/O beyond the stat. Microseconds. 2. **Content lookup** — hash the in-memory lockfile (not the file bytes — we already have the parsed object) and look up by content hash. Catches worktrees (same content, different path) and CI checkouts (same content, reset stat). On hit, append a refreshed path/stat entry so the next install at this path takes the stat shortcut. 3. **Any active verifier rejects the cached `policy`** — run the full gate. 4. **No record** — run the full gate. The in-memory object is hashed with `hashObject` from `@pnpm/crypto.object-hasher` (streaming, key-order-stable). ### Record shape ```json { "lockfile": { "hash": "", "path": "/abs/path/to/pnpm-lock.yaml", "size": 154, "mtimeNs": "1736245123000000000", "inode": "12345" }, "verifiedAt": "2026-05-17T...", "policy": { "minimumReleaseAge": 1440 } } ``` `policy` is the union of every active verifier's `policy` contribution. Verifiers checking the same logical policy (e.g. `minimumReleaseAge` honored by multiple registries) name it the same and share the slot — no resolver namespacing. ### File semantics - **Sync fs throughout** — the cache is consulted once before verification fan-out and recorded once after. No concurrent install work to overlap with; keeping the call sites straight-line. - **JSONL appends are atomic** on POSIX/NTFS, so parallel pnpm processes (monorepo installs, CI matrices sharing a cache) write without coordination. Latest record per `(path, hash)` tuple wins on read. - **Bounded file** — capped at ~1000 entries; compaction is triggered by a single `stat()` of the cache file (1.5 MiB byte budget) so we never parse the file on the steady-state path. When triggered, the tail is rewritten via tempfile + rename. - **No record on rejection** — a failing verification deliberately doesn't write a record; the next install must rerun the gate. - **Single hash per install** — the in-memory hash is computed lazily and reused: `tryLockfileVerificationCache` returns the precomputed stat+hash to `recordVerification` on a miss, and the stat-shortcut hit forwards the cached record's hash unchanged. ## Plumbing The verifier contract changed alongside the cache to make this composable without install-side knowledge of each policy: - **`@pnpm/resolving.resolver-base`** — `ResolutionVerifier` is now `{ verify, policy, canTrustPastCheck }` (was a bare function in #11583). Each resolver-side verifier owns its policy snapshot and the comparator that decides whether a cached policy is still trustworthy. - **`@pnpm/resolving.npm-resolver`** — `createNpmResolutionVerifier` returns the new shape: `policy: { minimumReleaseAge }`, `canTrustPastCheck` reads `minimumReleaseAge` from the merged cached bag. - **`@pnpm/resolving.default-resolver`** — `createResolutionVerifier` (singular, returning a combined function) → `createResolutionVerifiers` (plural, returning a `ResolutionVerifier[]`). No combinator; each verifier handles its own protocol short-circuit inside `verify`, so dispatch happens naturally at the install side. - **`@pnpm/installing.client`** — `Client.verifyResolution?` → `Client.resolutionVerifiers: ResolutionVerifier[]`. Same rename propagates through `@pnpm/store.connection-manager`, `@pnpm/testing.temp-store`, and `StrictInstallOptions`. - **`@pnpm/installing.deps-installer`** — new `verifyLockfileResolutionsCache.ts` (`tryLockfileVerificationCache` + `recordVerification`). `verifyLockfileResolutions` takes the verifier list plus `cacheDir` + `lockfilePath` as flat options; the cache fires when both are present, otherwise the gate runs without memoization. The dedup key for in-flight candidates includes a serialization of `resolution` so two entries sharing a (name, version) but pinned via different protocols don't collapse. Breaking but safe — `@pnpm/resolving.npm-resolver` hasn't been released since #11583 introduced the verifier abstraction, so no downstream consumer is on the old shape. ## Tests - **17 unit tests** in `verifyLockfileResolutionsCache.ts`: cache miss/hit, stat shortcut, size mismatch falling through to hash lookup, hash-fallback on reset stat, content change with matching size, stricter/weaker policy, missing-field policy rejection, multi-verifier policy merge (shared field stored once), worktree case (same content, different path), JSONL append semantics, malformed-line tolerance. - **12 integration tests** in `verifyLockfileResolutions.ts`: dedup of peer/patch-suffix variants, distinct-resolution dedup at the same (name, version), stable violation ordering, the 20-entry cap, multi-verifier fan-out (first failure wins), cache short-circuit on a passing run, no cache write on a rejecting run, empty-verifier-list passthrough. - **1 e2e test** in `pnpm/test/install/minimumReleaseAge.ts`: bundled CLI plumbing — install once to seed the lockfile, enable `minimumReleaseAge` + `cacheDir`, install again, assert the cache file lands at `/lockfile-verified.jsonl` with the documented record shape. - Existing `minimumReleaseAge` (13) and `frozenLockfile` (12) suites still pass. --- .../cache-aware-minimum-release-age-gate.md | 4 +- installing/client/src/index.ts | 16 +- installing/commands/src/fetch.ts | 2 +- installing/commands/src/import/index.ts | 2 +- installing/commands/src/installDeps.ts | 2 +- installing/commands/src/recursive.ts | 8 +- installing/commands/src/remove.ts | 2 +- .../src/install/extendInstallOptions.ts | 24 +- .../deps-installer/src/install/index.ts | 5 +- .../src/install/verifyLockfileResolutions.ts | 140 +++++- .../install/verifyLockfileResolutionsCache.ts | 428 ++++++++++++++++++ .../test/install/verifyLockfileResolutions.ts | 177 +++++++- .../install/verifyLockfileResolutionsCache.ts | 359 +++++++++++++++ .../deps-installer/test/utils/testDefaults.ts | 8 +- pnpm/test/install/minimumReleaseAge.ts | 46 ++ resolving/default-resolver/src/index.ts | 34 +- .../src/createNpmResolutionVerifier.ts | 18 +- resolving/resolver-base/src/index.ts | 42 +- .../src/createNewStoreController.ts | 6 +- store/connection-manager/src/index.ts | 2 +- testing/temp-store/src/index.ts | 6 +- 21 files changed, 1220 insertions(+), 111 deletions(-) create mode 100644 installing/deps-installer/src/install/verifyLockfileResolutionsCache.ts create mode 100644 installing/deps-installer/test/install/verifyLockfileResolutionsCache.ts diff --git a/.changeset/cache-aware-minimum-release-age-gate.md b/.changeset/cache-aware-minimum-release-age-gate.md index e9c39f1d41..957872248e 100644 --- a/.changeset/cache-aware-minimum-release-age-gate.md +++ b/.changeset/cache-aware-minimum-release-age-gate.md @@ -9,4 +9,6 @@ "pnpm": patch --- -Restructured the `minimumReleaseAge` lockfile revalidation gate around a generic `ResolutionVerifier` interface. Each resolver may now export a sibling verifier factory (today: `createNpmResolutionVerifier`) that re-checks an already-resolved lockfile entry against its policies; `createResolver`'s companion `createResolutionVerifier` combines them and the `Client` exposes the combined `verifyResolution` for the install layer to consume. The npm verifier reuses the same on-disk metadata mirror the resolver writes to, so steady-state installs pay only a headers-only conditional GET per locked package [#11675](https://github.com/pnpm/pnpm/issues/11675). +Restructured the `minimumReleaseAge` lockfile revalidation gate around a generic `ResolutionVerifier` interface. Each resolver may now export a sibling verifier factory (today: `createNpmResolutionVerifier`) that re-checks an already-resolved lockfile entry against its policies; the resolver chain returns the verifier list as `resolutionVerifiers` and the install side fans out across it. A `ResolutionVerifier` carries `verify` plus `policy` and `canTrustPastCheck` — the cache contract that lets repeat installs against an unchanged lockfile skip the per-package registry round trip entirely. + +Verification results are memoized in JSON Lines at `/lockfile-verified.jsonl`: a stat-only fast path matches on lockfile size, mtime, and inode, falling back to a content hash when those drift (typical after a CI checkout). Every active verifier's policy contribution is merged into a single `policy` bag on the record; the gate runs in full whenever the lockfile changes, any verifier rejects the cached policy, or no record exists [#11687](https://github.com/pnpm/pnpm/issues/11687). diff --git a/installing/client/src/index.ts b/installing/client/src/index.ts index cfd62c93f1..2673f51d02 100644 --- a/installing/client/src/index.ts +++ b/installing/client/src/index.ts @@ -9,7 +9,7 @@ import type { CustomFetcher, CustomResolver } from '@pnpm/hooks.types' import { createGetAuthHeaderByURI } from '@pnpm/network.auth-header' import { createFetchFromRegistry, type DispatcherOptions } from '@pnpm/network.fetch' import { - createResolutionVerifier, + createResolutionVerifiers, createResolver as _createResolver, type ResolutionVerifierFactoryOptions, type ResolveFunction, @@ -45,12 +45,13 @@ export interface Client { resolve: ResolveFunction clearResolutionCache: () => void /** - * Combined verifier across the resolver chain. `undefined` when no - * resolver-level policy is active (today: minimumReleaseAge strict mode). - * Used by the install layer to re-validate an already-resolved lockfile - * entry without re-doing resolution. + * List of resolver-side verifiers — one entry per active policy + * (today: at most one, `npm.minimumReleaseAge`). Empty when no policy + * is active. The install layer fans out across the list to re-validate + * each lockfile entry; each verifier handles its own protocol + * short-circuit inside `verify`. */ - verifyResolution?: ResolutionVerifier + resolutionVerifiers: ResolutionVerifier[] } export function createClient (opts: ClientOptions): Client { @@ -58,12 +59,11 @@ export function createClient (opts: ClientOptions): Client { const getAuthHeader = createGetAuthHeaderByURI(opts.configByUri, opts.registries?.default) const { resolve, clearCache: clearResolutionCache } = _createResolver(fetchFromRegistry, getAuthHeader, { ...opts, customResolvers: opts.customResolvers }) - const verifyResolution = createResolutionVerifier(fetchFromRegistry, opts) return { fetchers: createFetchers(fetchFromRegistry, getAuthHeader, opts), resolve, clearResolutionCache, - verifyResolution, + resolutionVerifiers: createResolutionVerifiers(fetchFromRegistry, opts), } } diff --git a/installing/commands/src/fetch.ts b/installing/commands/src/fetch.ts index b6209e3141..fd86cc1d9d 100644 --- a/installing/commands/src/fetch.ts +++ b/installing/commands/src/fetch.ts @@ -70,7 +70,7 @@ export async function handler (opts: FetchCommandOptions): Promise { pruneStore: true, storeController: store.ctrl, storeDir: store.dir, - verifyResolution: store.verifyResolution, + resolutionVerifiers: store.resolutionVerifiers, // Hoisting is skipped anyway, // so we store these empty patterns in node_modules/.modules.yaml // to let the subsequent install know that hoisting should be performed. diff --git a/installing/commands/src/import/index.ts b/installing/commands/src/import/index.ts index d249d2b738..27cb169b9d 100644 --- a/installing/commands/src/import/index.ts +++ b/installing/commands/src/import/index.ts @@ -186,7 +186,7 @@ export async function handler ( preferredVersions, storeController: store.ctrl, storeDir: store.dir, - verifyResolution: store.verifyResolution, + resolutionVerifiers: store.resolutionVerifiers, } await install(manifest, installOpts) } diff --git a/installing/commands/src/installDeps.ts b/installing/commands/src/installDeps.ts index fa109f5321..4155da7ed9 100644 --- a/installing/commands/src/installDeps.ts +++ b/installing/commands/src/installDeps.ts @@ -279,7 +279,7 @@ export async function installDeps ( skipRuntimes: opts.runtime === false, storeController: store.ctrl, storeDir: store.dir, - verifyResolution: store.verifyResolution, + resolutionVerifiers: store.resolutionVerifiers, workspacePackages, preferredVersions: opts.packageVulnerabilityAudit ? preferNonvulnerablePackageVersions(opts.packageVulnerabilityAudit) : undefined, } diff --git a/installing/commands/src/recursive.ts b/installing/commands/src/recursive.ts index afeb074ecf..44f3934021 100755 --- a/installing/commands/src/recursive.ts +++ b/installing/commands/src/recursive.ts @@ -115,7 +115,7 @@ export type RecursiveOptions = CreateStoreControllerOptions & Pick> = mutatedPkgs.map(async ({ originalManifest, manifest, rootDir }) => { @@ -418,7 +418,7 @@ export async function recursive ( }), configByUri: installOpts.configByUri, storeController: store.ctrl, - verifyResolution: store.verifyResolution, + resolutionVerifiers: store.resolutionVerifiers, } ) if (opts.save !== false) { diff --git a/installing/commands/src/remove.ts b/installing/commands/src/remove.ts index ac7367f9fe..6913fbb497 100644 --- a/installing/commands/src/remove.ts +++ b/installing/commands/src/remove.ts @@ -188,7 +188,7 @@ export async function handler ( linkWorkspacePackagesDepth: opts.linkWorkspacePackages === 'deep' ? Infinity : opts.linkWorkspacePackages ? 0 : -1, storeController: store.ctrl, storeDir: store.dir, - verifyResolution: store.verifyResolution, + resolutionVerifiers: store.resolutionVerifiers, include, }) const allProjects = opts.allProjects ?? ( diff --git a/installing/deps-installer/src/install/extendInstallOptions.ts b/installing/deps-installer/src/install/extendInstallOptions.ts index 584c4a829f..7cf60a611a 100644 --- a/installing/deps-installer/src/install/extendInstallOptions.ts +++ b/installing/deps-installer/src/install/extendInstallOptions.ts @@ -176,14 +176,23 @@ export interface StrictInstallOptions { minimumReleaseAge?: number minimumReleaseAgeExclude?: string[] /** - * Optional verifier that re-checks each lockfile-pinned resolution - * against policies configured upstream (today: minimumReleaseAge strict - * mode). Constructed by `createClient` and surfaced via the - * `createStoreController` return; mutateModules invokes it once, right - * after the lockfile is loaded from disk. When omitted, no revalidation - * runs. + * Resolver-side verifiers that re-check each lockfile-pinned resolution + * against policies configured upstream (today: at most one, + * `npm.minimumReleaseAge` in strict mode). Constructed by `createClient` + * and surfaced via the `createStoreController` return; mutateModules + * fans out across the list once, right after the lockfile is loaded + * from disk. Empty when no policy is active. */ - verifyResolution?: ResolutionVerifier + resolutionVerifiers: ResolutionVerifier[] + /** + * pnpm's on-disk cache directory. When set together with non-empty + * `resolutionVerifiers`, the lockfile verification result is memoized + * in `/lockfile-verified.jsonl` so repeat installs against an + * unchanged lockfile skip the per-package registry round trip. The + * record is policy-neutral; each active resolver-side verifier writes + * its own slot under `verifiers[]`. + */ + cacheDir?: string trustPolicy?: TrustPolicy trustPolicyExclude?: string[] trustPolicyIgnoreAfter?: number @@ -301,6 +310,7 @@ const defaults = (opts: InstallOptions): StrictInstallOptions => { peersSuffixMaxLength: 1000, blockExoticSubdeps: false, omitSummaryLog: false, + resolutionVerifiers: [] as ResolutionVerifier[], } as StrictInstallOptions } diff --git a/installing/deps-installer/src/install/index.ts b/installing/deps-installer/src/install/index.ts index 5b4ecd7564..59d363bd76 100644 --- a/installing/deps-installer/src/install/index.ts +++ b/installing/deps-installer/src/install/index.ts @@ -344,7 +344,10 @@ export async function mutateModules ( // exactly once, right after the lockfile is loaded from disk, before any // path branches. try { - await verifyLockfileResolutions(ctx.wantedLockfile, opts.verifyResolution) + await verifyLockfileResolutions(ctx.wantedLockfile, opts.resolutionVerifiers, { + cacheDir: opts.cacheDir, + lockfilePath: path.resolve(ctx.lockfileDir, WANTED_LOCKFILE), + }) } catch (err) { // verifyLockfileResolutions is the one throw site in this function // that's part of normal user-facing operation (a rejected lockfile); diff --git a/installing/deps-installer/src/install/verifyLockfileResolutions.ts b/installing/deps-installer/src/install/verifyLockfileResolutions.ts index 1f1ad89773..a3fe703163 100644 --- a/installing/deps-installer/src/install/verifyLockfileResolutions.ts +++ b/installing/deps-installer/src/install/verifyLockfileResolutions.ts @@ -1,10 +1,16 @@ +import { hashObject } from '@pnpm/crypto.object-hasher' import { PnpmError } from '@pnpm/error' import type { LockfileObject } from '@pnpm/lockfile.fs' import { nameVerFromPkgSnapshot } from '@pnpm/lockfile.utils' -import type { ResolutionVerifier } from '@pnpm/resolving.resolver-base' +import type { Resolution, ResolutionVerifier } from '@pnpm/resolving.resolver-base' import type { DepPath } from '@pnpm/types' import pLimit from 'p-limit' +import { + recordVerification, + tryLockfileVerificationCache, +} from './verifyLockfileResolutionsCache.js' + interface Violation { pkgId: string code: string @@ -21,39 +27,110 @@ const MAX_VIOLATIONS_TO_PRINT = 20 // verification pass doesn't push past what the rest of the install respects. const DEFAULT_CONCURRENCY = 16 +export interface VerifyLockfileResolutionsOptions { + concurrency?: number + /** + * pnpm's on-disk cache directory. When set together with + * `lockfilePath`, verification results are memoized in + * `/lockfile-verified.jsonl` and the gate short-circuits on + * a repeat run against an unchanged lockfile + same-or-stricter + * policy. Omit to disable the cache entirely (every call rehashes + * the lockfile and re-verifies). + */ + cacheDir?: string + /** Absolute path of the lockfile being verified. Used by the cache's stat shortcut. */ + lockfilePath?: string +} + /** - * Policy-neutral pass that asks each resolver-supplied {@link ResolutionVerifier} - * to check every entry in a lockfile loaded from disk. Iteration runs - * before resolution decisions are touched and before any tarball is - * fetched, so a lockfile whose entries were resolved elsewhere (committed - * to the repo, restored from a cache, etc.) under a weaker or absent - * policy cannot reach the filesystem. Fresh local resolution is covered - * by the resolver's own per-version filter. + * Policy-neutral pass that asks every resolver-supplied + * {@link ResolutionVerifier} to check every entry in a lockfile loaded + * from disk. Iteration runs before resolution decisions are touched and + * before any tarball is fetched, so a lockfile whose entries were + * resolved elsewhere (committed to the repo, restored from a cache, + * etc.) under a weaker or absent policy cannot reach the filesystem. + * Fresh local resolution is covered by the resolver's own per-version + * filter. * - * Designed for fail-closed semantics at the verifier level: a verifier that - * can't confirm a resolution is expected to return `{ ok: false }` rather - * than passing silently — otherwise a registry hiccup or an unpublished - * version would re-open the bypass. + * Each verifier handles its own protocol short-circuit inside `verify` + * (returning `{ ok: true }` for resolutions outside its scope), so the + * fan-out is policy-neutral and dispatch-free at this layer. * - * No-op when `verifyResolution` is undefined (no active policies). + * Designed for fail-closed semantics at the verifier level: a verifier + * that can't confirm a resolution is expected to return `{ ok: false }` + * rather than passing silently — otherwise a registry hiccup or an + * unpublished version would re-open the bypass. + * + * No-op when `verifiers` is empty. + * + * When `options.cacheDir` and `options.lockfilePath` are both + * provided, an unchanged lockfile that has already been verified + * under the same (or stricter) policy short-circuits the registry + * round-trip entirely — see {@link tryLockfileVerificationCache} for + * the lookup logic. */ export async function verifyLockfileResolutions ( lockfile: LockfileObject, - verifyResolution: ResolutionVerifier | undefined, - options?: { concurrency?: number } + verifiers: ResolutionVerifier[], + options?: VerifyLockfileResolutionsOptions ): Promise { - if (verifyResolution == null) return + if (verifiers.length === 0) return if (!lockfile.packages) return + // Caching kicks in only when the caller surfaced both a writable + // cache directory and the lockfile's absolute path — that's the + // production wiring; unit tests that skip them get the gate without + // memoization and still exercise the same code path. + const cache = options?.cacheDir && options?.lockfilePath + ? { cacheDir: options.cacheDir, lockfilePath: options.lockfilePath } + : undefined + + // Cache lookup runs before any registry I/O — the fast path is a + // single stat() of the lockfile when the previous install already + // verified it under a policy that's at least as strict as today's. + // The content key is hashed lazily from the in-memory lockfile (not + // the file bytes) so we never read the file a second time. On a + // miss the precomputed stat+hash flow to recordVerification. + type Precomputed = ReturnType['precomputed'] + let cachePrecomputed: Precomputed | undefined + // hashObject is streaming (no "Invalid string length" on huge lockfiles) + // and key-order-stable, which JSON.stringify is not. + let cachedHash: string | undefined + const hashLockfile = (): string => { + if (cachedHash == null) cachedHash = hashObject(lockfile) + return cachedHash + } + if (cache) { + const result = tryLockfileVerificationCache(cache.cacheDir, { + lockfilePath: cache.lockfilePath, + verifiers, + hashLockfile, + }) + if (result.hit) return + cachePrecomputed = result.precomputed + } + // depPath can include peer-dependency and patch_hash suffixes (e.g. // `react@18.0.0(peer)(patch_hash=…)`); the same (name, version) pair may // therefore appear multiple times. Dedupe so we issue at most one // verification per package version. - const candidates = new Map() + // + // Include a serialization of `resolution` in the key so two entries that + // share a (name, version) but differ in *what* was resolved (e.g. one + // pinned via npm, another via a git URL under the same alias) don't + // collapse: if the wrong shape wins the dedup, a protocol-scoped + // verifier short-circuits on the surviving entry and the real one is + // never checked. + const candidates = new Map() for (const [depPath, snapshot] of Object.entries(lockfile.packages)) { const { name, version } = nameVerFromPkgSnapshot(depPath as DepPath, snapshot) if (!name || !version) continue - candidates.set(`${name}@${version}`, { name, version, resolution: snapshot.resolution }) + const key = `${name}@${version}@${JSON.stringify(snapshot.resolution)}` + candidates.set(key, { + name, + version, + resolution: snapshot.resolution as Resolution, + }) } const violations: Violation[] = [] @@ -61,14 +138,33 @@ export async function verifyLockfileResolutions ( await Promise.all( Array.from(candidates.values(), ({ name, version, resolution }) => limit(async () => { const pkgId = `${name}@${version}` - const result = await verifyResolution(resolution as Parameters[0], { name, version }) - if (!result.ok) { - violations.push({ pkgId, code: result.code, reason: result.reason }) + // Fan out across every active verifier; each handles its own + // protocol short-circuit (e.g. the npm verifier returns ok:true for + // git resolutions). We stop at the first failure per entry so a + // multi-verifier setup doesn't produce duplicate violations for the + // same (name, version). + for (const verifier of verifiers) { + // eslint-disable-next-line no-await-in-loop + const result = await verifier.verify(resolution, { name, version }) + if (!result.ok) { + violations.push({ pkgId, code: result.code, reason: result.reason }) + break + } } })) ) - if (violations.length === 0) return + if (violations.length === 0) { + // Persist the success so the next install can stat-only the lockfile. + if (cache) { + recordVerification(cache.cacheDir, { + lockfilePath: cache.lockfilePath, + verifiers, + hashLockfile, + }, cachePrecomputed) + } + return + } // Stable order so the error output is deterministic. violations.sort((a, b) => a.pkgId.localeCompare(b.pkgId)) diff --git a/installing/deps-installer/src/install/verifyLockfileResolutionsCache.ts b/installing/deps-installer/src/install/verifyLockfileResolutionsCache.ts new file mode 100644 index 0000000000..b1315577fb --- /dev/null +++ b/installing/deps-installer/src/install/verifyLockfileResolutionsCache.ts @@ -0,0 +1,428 @@ +import fs from 'node:fs' +import path from 'node:path' +import util from 'node:util' + +import { logger } from '@pnpm/logger' +import type { ResolutionVerifier } from '@pnpm/resolving.resolver-base' + +/** + * Subset of {@link ResolutionVerifier} the cache layer needs: the + * verifier's `policy` contribution plus the `canTrustPastCheck` + * comparator. `verify` is intentionally absent — the cache never runs + * verifiers, it just decides whether a previous run is still + * trustworthy. + */ +export type VerifierCacheIdentity = Pick + +/** + * On-disk cache of verifyLockfileResolutions results, keyed by lockfile + * content hash. Lets repeat installs against an unchanged lockfile skip + * the per-package registry round trips entirely — including across git + * worktrees, where the same lockfile content lives at different paths. + * + * Two indexes share the same JSONL records: + * + * - **by content hash** — the primary index. Recognizing the same + * lockfile content regardless of where it sits on disk is what makes + * worktrees and lockfile copies hit. + * - **by absolute path** — a same-machine stat shortcut. When we've + * seen this exact path before with these exact stat values, we + * trust the cached hash and skip reading the lockfile entirely + * (microseconds vs. ms-per-MB). Worktrees that get reinstalled in + * pay the hash cost once, then hit the stat fast path. + * + * All filesystem operations are synchronous: the cache is consulted + * once before verification fan-out and recorded once after — there's + * no concurrent install work to overlap with, so blocking the event + * loop for the brief read/stat/hash is fine and keeps the call sites + * straight-line. + * + * Persisted as JSON Lines: each verification appends one record; + * later records overwrite earlier ones on key collision when read. + * Appends of a single line are atomic on POSIX and NTFS, so parallel + * pnpm processes (monorepo installs, CI matrices sharing a cache) can + * write without coordination. + * + * Policy-neutral. Every active verifier's `policy` contribution merges + * into a single `policy` bag on the record; verifiers sharing a + * logical policy (same field name) share the slot — no resolver-level + * namespacing. + */ + +const CACHE_FILE_NAME = 'lockfile-verified.jsonl' + +// Cap the file before it grows large enough to slow down reads. When the +// cap is exceeded we rewrite the file keeping the N most recently +// verified entries. The number is generous — a developer machine that +// touches a thousand distinct (path, content) tuples is far past steady +// state. +const MAX_CACHE_ENTRIES = 1000 + +// Records cluster around 250–400 bytes; budget 1 KiB per entry as a +// conservative upper bound. The compaction check uses `stat().size` to +// decide whether to read+rewrite, so we never parse the file unless it +// has actually grown past the cap. +const COMPACT_TRIGGER_BYTES = MAX_CACHE_ENTRIES * 1024 * 3 / 2 + +interface CacheRecord { + lockfile: { + /** + * sha256 hex of the lockfile content — primary cache key. Computed + * from the parsed in-memory lockfile object (not the raw file + * bytes); two YAML layouts that parse to the same object share a + * hash. Same content on disk → same parsed object → same hash, so + * worktrees and CI checkouts collide here. + */ + hash: string + /** Absolute path the cache last saw this content at — secondary index for the stat fast path. */ + path: string + /** Lockfile size in bytes. */ + size: number + /** + * Lockfile mtime in nanoseconds (stringified — JSON numbers lose + * ns precision). Cross-machine values are meaningless; on a CI + * runner the fresh checkout resets mtime, so we fall back to + * hashing. + */ + mtimeNs: string + /** + * Stringified — some filesystems (e.g. large network drives) use + * inodes that exceed Number.MAX_SAFE_INTEGER, so a plain number + * would lose precision and silently invalidate the fast path. + */ + inode: string + } + /** ISO-8601 timestamp of when the verification ran. */ + verifiedAt: string + /** + * Merged policy snapshot that passed when the verification ran. Each + * active {@link VerifierCacheIdentity} contributes its fields here; + * verifiers checking the same logical policy (same field name) share + * the slot. On read, each verifier's `canTrustPastCheck` decides + * whether today's policy can still trust this snapshot. + */ + policy: Record +} + +export interface CacheLookupResult { + hit: boolean + /** + * stat + hash already computed during the lookup. When the caller + * follows up with {@link recordVerification} after running the gate, + * passing these back avoids re-stat'ing and (especially) re-hashing + * the lockfile a second time. Fields are undefined when the lookup + * couldn't (or didn't need to) compute them — `recordVerification` + * falls back to computing what's missing. + */ + precomputed: { stat?: LockfileStat, hash?: string } +} + +interface LockfileStat { + size: number + mtimeNs: string + inode: string +} + +export interface LockfileVerificationCacheKey { + lockfilePath: string + verifiers: readonly VerifierCacheIdentity[] + /** + * Lazy: returns a stable hex hash of the in-memory lockfile. The + * cache invokes this only when the stat shortcut doesn't apply (the + * lockfile is at a new path, or its stat has drifted from the + * cached record). When the stat shortcut hits, the in-memory hash is + * never computed. + */ + hashLockfile: () => string +} + +interface CacheIndexes { + /** Latest record per content hash — primary lookup. */ + byHash: Map + /** Latest record per absolute path — same-machine stat fast path. */ + byPath: Map +} + +/** + * Build two indexes over the JSONL records in one pass: by content + * hash (primary) and by absolute path (stat shortcut). Records are + * walked in file order so the last record for any key wins. + */ +function readCache (cacheDir: string): CacheIndexes { + const cacheFilePath = path.join(cacheDir, CACHE_FILE_NAME) + let contents: string + try { + contents = fs.readFileSync(cacheFilePath, 'utf8') + } catch (err: unknown) { + if (isNodeError(err) && err.code === 'ENOENT') return { byHash: new Map(), byPath: new Map() } + throw err + } + const byHash = new Map() + const byPath = new Map() + for (const line of contents.split('\n')) { + if (!line) continue + try { + const parsed = JSON.parse(line) as Partial + const hash = parsed?.lockfile?.hash + const lockfilePath = parsed?.lockfile?.path + if (typeof hash !== 'string' || typeof lockfilePath !== 'string') continue + const record = normalizeRecord(parsed) + byHash.set(hash, record) + byPath.set(lockfilePath, record) + } catch { + // Skip malformed lines; the next clean append will still work. + } + } + return { byHash, byPath } +} + +function normalizeRecord (parsed: Partial): CacheRecord { + const lockfile: Partial = parsed.lockfile ?? {} + return { + lockfile: { + hash: lockfile.hash ?? '', + path: lockfile.path ?? '', + size: lockfile.size ?? -1, + mtimeNs: lockfile.mtimeNs ?? '', + inode: lockfile.inode ?? '', + }, + verifiedAt: parsed.verifiedAt ?? '', + policy: parsed.policy && typeof parsed.policy === 'object' ? parsed.policy : {}, + } +} + +function statLockfile (lockfilePath: string): LockfileStat | null { + try { + const stat = fs.statSync(lockfilePath, { bigint: true }) + return { + size: Number(stat.size), + mtimeNs: stat.mtimeNs.toString(), + inode: stat.ino.toString(), + } + } catch (err: unknown) { + if (isNodeError(err) && err.code === 'ENOENT') return null + throw err + } +} + +function statMatches (stat: LockfileStat, lockfile: CacheRecord['lockfile']): boolean { + return stat.size === lockfile.size && + stat.mtimeNs === lockfile.mtimeNs && + stat.inode === lockfile.inode +} + +/** + * Try to confirm a cached verification covers the lockfile as it + * currently sits on disk and the policies currently in effect. Returns + * `{ hit: true }` to skip the gate; `{ hit: false }` means the caller + * should run the verifier and persist the result with + * {@link recordVerification}. + * + * Lookup order: + * + * 1. **Stat shortcut** — if we've previously verified this exact path + * with these exact stat values, trust the cached hash and skip + * reading the lockfile. + * 2. **Content lookup** — hash the lockfile and look up by hash. + * Catches the worktree case (same content, different path) and + * CI checkouts where stat fields got reset. Refreshes the + * stat-shortcut entry on hit so the next install at this path + * skips the hash. + * + * Every active verifier must agree the cached policy snapshot is still + * trustworthy under what it currently demands; if any rejects, the + * full gate runs. + */ +export function tryLockfileVerificationCache ( + cacheDir: string, + key: LockfileVerificationCacheKey +): CacheLookupResult { + let indexes: CacheIndexes + try { + indexes = readCache(cacheDir) + } catch (err: unknown) { + // A corrupt cache file should never block the install; fall + // through to verification so the gate still runs. + logger.debug({ msg: 'lockfile-verified cache: read failed', err }) + return { hit: false, precomputed: {} } + } + + const stat = statLockfile(key.lockfilePath) + if (!stat) return { hit: false, precomputed: {} } + + // Stat shortcut: same path + same stat means we trust the cached + // hash without reading the file. Microseconds. + const byPathRecord = indexes.byPath.get(key.lockfilePath) + if (byPathRecord && statMatches(stat, byPathRecord.lockfile)) { + return { + hit: everyVerifierTrustsCachedRun(byPathRecord, key.verifiers), + // The stat-match implies the file content is unchanged since the + // cached record was written, so its hash is still correct. Pass + // it through to skip hashing on the miss-then-record path. + precomputed: { stat, hash: byPathRecord.lockfile.hash }, + } + } + + // Content lookup: hash the in-memory lockfile, look up by content + // hash. Catches worktrees (same content, different path) and CI + // checkouts (same content, reset stat). On hit, refresh the + // path/stat entry so the next install at this path takes the stat + // shortcut above. + let hash: string + try { + hash = key.hashLockfile() + } catch (err: unknown) { + logger.debug({ msg: 'lockfile-verified cache: lockfile hash failed', err }) + return { hit: false, precomputed: { stat } } + } + const byHashRecord = indexes.byHash.get(hash) + if (!byHashRecord) return { hit: false, precomputed: { stat, hash } } + if (!everyVerifierTrustsCachedRun(byHashRecord, key.verifiers)) { + return { hit: false, precomputed: { stat, hash } } + } + + appendRecord(cacheDir, { + ...byHashRecord, + lockfile: { ...byHashRecord.lockfile, path: key.lockfilePath, size: stat.size, mtimeNs: stat.mtimeNs, inode: stat.inode }, + }) + return { hit: true, precomputed: { stat, hash } } +} + +function everyVerifierTrustsCachedRun (record: CacheRecord, verifiers: readonly VerifierCacheIdentity[]): boolean { + for (const verifier of verifiers) { + if (!verifier.canTrustPastCheck(record.policy)) return false + } + return true +} + +function mergePolicies (verifiers: readonly VerifierCacheIdentity[]): Record { + // Later verifiers overwrite earlier ones on conflict — a shared field + // should carry the same value across verifiers by convention; mismatch + // is a config bug and we don't try to reconcile it here. + const merged: Record = {} + for (const verifier of verifiers) { + Object.assign(merged, verifier.policy) + } + return merged +} + +/** + * Persist a successful verification. Called after the gate passes; the + * lockfile is hashed once and the resulting record is appended to the + * cache file. If the file is past {@link MAX_CACHE_ENTRIES}, it is + * rewritten keeping the most recent entries. + * + * Reuses `precomputed` values from a prior + * {@link tryLockfileVerificationCache} lookup so we don't re-stat or + * (especially) re-hash the lockfile a second time on the miss-then- + * record path. + */ +export function recordVerification ( + cacheDir: string, + key: LockfileVerificationCacheKey, + precomputed?: { stat?: LockfileStat, hash?: string } +): void { + let stat: LockfileStat | null + let hash: string + try { + stat = precomputed?.stat ?? statLockfile(key.lockfilePath) + if (!stat) return + hash = precomputed?.hash ?? key.hashLockfile() + } catch (err: unknown) { + // The gate has already passed; if we can't record the cache entry we + // just won't get the speedup next time. Not a reason to fail install. + logger.debug({ msg: 'lockfile-verified cache: could not record verification', err }) + return + } + const record: CacheRecord = { + lockfile: { + hash, + path: key.lockfilePath, + size: stat.size, + mtimeNs: stat.mtimeNs, + inode: stat.inode, + }, + verifiedAt: new Date().toISOString(), + policy: mergePolicies(key.verifiers), + } + appendRecord(cacheDir, record) +} + +function appendRecord (cacheDir: string, record: CacheRecord): void { + const cacheFilePath = path.join(cacheDir, CACHE_FILE_NAME) + const line = `${JSON.stringify(record)}\n` + try { + fs.mkdirSync(cacheDir, { recursive: true }) + fs.appendFileSync(cacheFilePath, line) + } catch (err: unknown) { + logger.debug({ msg: 'lockfile-verified cache: append failed', err }) + return + } + maybeCompactCache(cacheDir) +} + +function maybeCompactCache (cacheDir: string): void { + const cacheFilePath = path.join(cacheDir, CACHE_FILE_NAME) + // Decide whether to compact from the file size alone — avoids reading + // and parsing the file on every successful install. Records cluster + // around a few hundred bytes; the byte budget translates directly to + // the entry cap with generous slack so we don't trigger a rewrite on + // every append once we cross the line. + let size: number + try { + size = fs.statSync(cacheFilePath).size + } catch (err: unknown) { + if (isNodeError(err) && err.code === 'ENOENT') return + logger.debug({ msg: 'lockfile-verified cache: stat for compaction failed', err }) + return + } + if (size <= COMPACT_TRIGGER_BYTES) return + + let contents: string + try { + contents = fs.readFileSync(cacheFilePath, 'utf8') + } catch (err: unknown) { + if (isNodeError(err) && err.code === 'ENOENT') return + logger.debug({ msg: 'lockfile-verified cache: read for compaction failed', err }) + return + } + const lines = contents.split('\n').filter(Boolean) + + // Dedup by (path, hash) — that's the unit both indexes care about. + // Walking reverse keeps the newest record per tuple; we then trim to + // MAX_CACHE_ENTRIES and write back in original order. + const seen = new Set() + const reversed: string[] = [] + for (let i = lines.length - 1; i >= 0; i--) { + const line = lines[i] + try { + const parsed = JSON.parse(line) as Partial + const lockfilePath = parsed?.lockfile?.path + const hash = parsed?.lockfile?.hash + if (typeof lockfilePath !== 'string' || typeof hash !== 'string') continue + const tupleKey = `${lockfilePath}${hash}` + if (seen.has(tupleKey)) continue + seen.add(tupleKey) + reversed.push(line) + } catch { + // Skip malformed lines. + } + } + reversed.reverse() + const kept = reversed.slice(-MAX_CACHE_ENTRIES) + try { + // Write to a sibling tempfile + rename so a concurrent pnpm process + // can't observe a half-written file. + const tmpPath = `${cacheFilePath}.${process.pid}.tmp` + fs.writeFileSync(tmpPath, kept.map((line) => `${line}\n`).join('')) + fs.renameSync(tmpPath, cacheFilePath) + } catch (err: unknown) { + logger.debug({ msg: 'lockfile-verified cache: compaction failed', err }) + } +} + +function isNodeError (err: unknown): err is NodeJS.ErrnoException { + // `instanceof Error` is unreliable across realms (Jest's VM context), so + // route through util.types.isNativeError per the repo guideline. + return util.types.isNativeError(err) && 'code' in err +} diff --git a/installing/deps-installer/test/install/verifyLockfileResolutions.ts b/installing/deps-installer/test/install/verifyLockfileResolutions.ts index 67474c906c..a340e3261d 100644 --- a/installing/deps-installer/test/install/verifyLockfileResolutions.ts +++ b/installing/deps-installer/test/install/verifyLockfileResolutions.ts @@ -1,3 +1,7 @@ +import fs from 'node:fs' +import os from 'node:os' +import path from 'node:path' + import { expect, test } from '@jest/globals' import type { LockfileObject } from '@pnpm/lockfile.fs' import type { ResolutionVerifier } from '@pnpm/resolving.resolver-base' @@ -14,18 +18,30 @@ function makeLockfile (packages: Record ({ integrity, tarball: '' }) -const okVerifier: ResolutionVerifier = async () => ({ ok: true }) +const NOOP_SLOT = { + policy: {} as Record, + canTrustPastCheck: () => true, +} -test('no-op when verifyResolution is undefined', async () => { +function wrap ( + verify: ResolutionVerifier['verify'], + slot: Omit = NOOP_SLOT +): ResolutionVerifier { + return { ...slot, verify } +} + +const okVerifier = wrap(async () => ({ ok: true })) + +test('no-op when the verifier list is empty', async () => { const lockfile = makeLockfile({ 'fresh@1.0.0': { resolution: tarballResolution() }, }) - await expect(verifyLockfileResolutions(lockfile, undefined)).resolves.toBeUndefined() + await expect(verifyLockfileResolutions(lockfile, [])).resolves.toBeUndefined() }) test('no-op when lockfile has no packages', async () => { const lockfile = makeLockfile({}) - await expect(verifyLockfileResolutions(lockfile, okVerifier)).resolves.toBeUndefined() + await expect(verifyLockfileResolutions(lockfile, [okVerifier])).resolves.toBeUndefined() }) test('passes when every entry is verified ok', async () => { @@ -33,20 +49,20 @@ test('passes when every entry is verified ok', async () => { 'lodash@4.17.21': { resolution: tarballResolution() }, 'is-odd@0.1.0': { resolution: tarballResolution() }, }) - await expect(verifyLockfileResolutions(lockfile, okVerifier)).resolves.toBeUndefined() + await expect(verifyLockfileResolutions(lockfile, [okVerifier])).resolves.toBeUndefined() }) test('throws with the verifier-supplied code and reason on a single failure', async () => { const lockfile = makeLockfile({ 'is-odd@0.1.2': { resolution: tarballResolution() }, }) - const verifier: ResolutionVerifier = async () => ({ + const verifier = wrap(async () => ({ ok: false, code: 'MINIMUM_RELEASE_AGE_VIOLATION', reason: 'was published yesterday', - }) + })) - await expect(verifyLockfileResolutions(lockfile, verifier)).rejects.toMatchObject({ + await expect(verifyLockfileResolutions(lockfile, [verifier])).rejects.toMatchObject({ code: 'ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION', message: expect.stringMatching(/is-odd@0\.1\.2 was published yesterday/), }) @@ -57,13 +73,13 @@ test('lists violations in stable order across multiple failures', async () => { 'fresh-b@2.0.0': { resolution: tarballResolution('sha512-b') }, 'fresh-a@1.0.0': { resolution: tarballResolution('sha512-a') }, }) - const verifier: ResolutionVerifier = async (_, { name, version }) => ({ + const verifier = wrap(async (_, { name, version }) => ({ ok: false, code: 'POLICY_X', reason: `${name}@${version} failed`, - }) + })) - await expect(verifyLockfileResolutions(lockfile, verifier)) + await expect(verifyLockfileResolutions(lockfile, [verifier])) .rejects.toThrow(/fresh-a@1\.0\.0[\s\S]*fresh-b@2\.0\.0/) }) @@ -75,13 +91,13 @@ test('caps printed violations at 20 with an "…and N more" summary', async () = } } const lockfile = makeLockfile(packages) - const verifier: ResolutionVerifier = async (_, { name, version }) => ({ + const verifier = wrap(async (_, { name, version }) => ({ ok: false, code: 'POLICY_X', reason: `${name}@${version}`, - }) + })) - await expect(verifyLockfileResolutions(lockfile, verifier)) + await expect(verifyLockfileResolutions(lockfile, [verifier])) .rejects.toThrow(/25 lockfile entries failed verification[\s\S]*…and 5 more/) }) @@ -92,15 +108,38 @@ test('dedupes peer/patch-suffix variants and invokes the verifier once per (name 'react@18.0.0(patch_hash=abc)(peer-x)': { resolution: tarballResolution('sha512-a') }, }) const seen: Array<{ name: string, version: string }> = [] - const verifier: ResolutionVerifier = async (_, { name, version }) => { + const verifier = wrap(async (_, { name, version }) => { seen.push({ name, version }) return { ok: true } - } + }) - await verifyLockfileResolutions(lockfile, verifier) + await verifyLockfileResolutions(lockfile, [verifier]) expect(seen).toEqual([{ name: 'react', version: '18.0.0' }]) }) +test('does not collapse same (name, version) with different resolutions', async () => { + // Two entries sharing a name@version but pinned via different protocols + // (npm registry vs. git). If the dedup key were just `name@version` one + // would silently overwrite the other and a protocol-scoped verifier + // would short-circuit on the survivor — letting the real entry skip + // the gate. + const npmResolution = tarballResolution('sha512-a') + const gitResolution = { type: 'git', repo: 'x', commit: 'abc' } + const lockfile = makeLockfile({ + 'foo@1.0.0': { resolution: npmResolution }, + 'foo@1.0.0(peer-x)': { resolution: gitResolution }, + }) + const seenResolutions: unknown[] = [] + const verifier = wrap(async (resolution) => { + seenResolutions.push(resolution) + return { ok: true } + }) + + await verifyLockfileResolutions(lockfile, [verifier]) + expect(seenResolutions).toEqual(expect.arrayContaining([npmResolution, gitResolution])) + expect(seenResolutions).toHaveLength(2) +}) + test('the verifier sees the resolution shape verbatim', async () => { const npmResolution = tarballResolution() const gitResolution = { type: 'git', repo: 'x', commit: 'abc' } @@ -109,12 +148,12 @@ test('the verifier sees the resolution shape verbatim', async () => { 'git-pkg@1.0.0': { resolution: gitResolution }, }) const received: unknown[] = [] - const verifier: ResolutionVerifier = async (resolution) => { + const verifier = wrap(async (resolution) => { received.push(resolution) return { ok: true } - } + }) - await verifyLockfileResolutions(lockfile, verifier) + await verifyLockfileResolutions(lockfile, [verifier]) expect(received).toEqual(expect.arrayContaining([npmResolution, gitResolution])) }) @@ -123,13 +162,105 @@ test('uses the first violation\'s code when multiple verifiers fire', async () = 'a@1.0.0': { resolution: tarballResolution('sha512-a') }, 'b@1.0.0': { resolution: tarballResolution('sha512-b') }, }) - const verifier: ResolutionVerifier = async (_, { name }) => ({ + const verifier = wrap(async (_, { name }) => ({ ok: false, code: name === 'a' ? 'POLICY_A' : 'POLICY_B', reason: 'failed', - }) + })) - await expect(verifyLockfileResolutions(lockfile, verifier)).rejects.toMatchObject({ + await expect(verifyLockfileResolutions(lockfile, [verifier])).rejects.toMatchObject({ code: 'ERR_PNPM_POLICY_A', }) }) + +test('runs every active verifier per entry and stops at the first failure', async () => { + const lockfile = makeLockfile({ + 'a@1.0.0': { resolution: tarballResolution('sha512-a') }, + }) + const calls: string[] = [] + const firstOk = wrap(async () => { + calls.push('first') + return { ok: true } + }, NOOP_SLOT) + const secondFail = wrap(async () => { + calls.push('second') + return { ok: false, code: 'SECOND_POLICY', reason: 'nope' } + }, NOOP_SLOT) + + await expect(verifyLockfileResolutions(lockfile, [firstOk, secondFail])) + .rejects.toMatchObject({ code: 'ERR_PNPM_SECOND_POLICY' }) + // Both verifiers ran on the entry; ordering follows the list. + expect(calls).toEqual(['first', 'second']) +}) + +function exampleSlot (current: number): Omit { + return { + policy: { minimumReleaseAge: current }, + canTrustPastCheck: (cached) => { + const past = cached.minimumReleaseAge + return typeof past === 'number' && past >= current + }, + } +} + +test('skips the verifier when the cache holds an unchanged lockfile + matching policy', async () => { + const tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'pnpm-vlr-')) + try { + const cacheDir = path.join(tmpDir, 'cache') + const lockfilePath = path.join(tmpDir, 'pnpm-lock.yaml') + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + const lockfile = makeLockfile({ + 'a@1.0.0': { resolution: tarballResolution('sha512-a') }, + }) + + let calls = 0 + const counting = wrap(async () => { + calls++ + return { ok: true } + }, exampleSlot(60)) + + // First call has no cache record yet — verifier runs. + await verifyLockfileResolutions(lockfile, [counting], { + cacheDir, lockfilePath, + }) + expect(calls).toBe(1) + + // Second call against the same lockfile + policy — cache short-circuit. + await verifyLockfileResolutions(lockfile, [counting], { + cacheDir, lockfilePath, + }) + expect(calls).toBe(1) + } finally { + await fs.promises.rm(tmpDir, { recursive: true, force: true }) + } +}) + +test('does not write a cache record when verification rejects', async () => { + const tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'pnpm-vlr-')) + try { + const cacheDir = path.join(tmpDir, 'cache') + const lockfilePath = path.join(tmpDir, 'pnpm-lock.yaml') + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + const lockfile = makeLockfile({ + 'a@1.0.0': { resolution: tarballResolution('sha512-a') }, + }) + + const rejecting = wrap(async () => ({ + ok: false, + code: 'POLICY_X', + reason: 'failed', + }), exampleSlot(60)) + + await expect( + verifyLockfileResolutions(lockfile, [rejecting], { + cacheDir, lockfilePath, + }) + ).rejects.toThrow() + + // No record was written — a rejecting verification must rerun next install. + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + await expect(fs.promises.access(cacheFile)).rejects.toThrow() + } finally { + await fs.promises.rm(tmpDir, { recursive: true, force: true }) + } +}) diff --git a/installing/deps-installer/test/install/verifyLockfileResolutionsCache.ts b/installing/deps-installer/test/install/verifyLockfileResolutionsCache.ts new file mode 100644 index 0000000000..2c83cb36c7 --- /dev/null +++ b/installing/deps-installer/test/install/verifyLockfileResolutionsCache.ts @@ -0,0 +1,359 @@ +import fs from 'node:fs' +import os from 'node:os' +import path from 'node:path' + +import { afterEach, beforeEach, describe, expect, test } from '@jest/globals' + +import { + recordVerification, + tryLockfileVerificationCache, + type VerifierCacheIdentity, +} from '../../src/install/verifyLockfileResolutionsCache.js' + +let tmpDir!: string +let cacheDir!: string +let lockfilePath!: string + +beforeEach(async () => { + tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'pnpm-verify-cache-')) + cacheDir = path.join(tmpDir, 'cache') + lockfilePath = path.join(tmpDir, 'pnpm-lock.yaml') +}) + +afterEach(async () => { + await fs.promises.rm(tmpDir, { recursive: true, force: true }) +}) + +// Helpers — most tests use a stand-in for the npm minimumReleaseAge +// verifier. The cache layer is policy-neutral, so this could be any +// verifier shape. +function mraVerifier (current: number): VerifierCacheIdentity { + return { + policy: { minimumReleaseAge: current }, + canTrustPastCheck: (cached) => { + const past = cached.minimumReleaseAge + return typeof past === 'number' && past >= current + }, + } +} + +// The cache hashes the in-memory lockfile, not the file bytes — but for +// unit tests we don't need a real lockfile object. A stable thunk is +// enough; tests that simulate "different content" pass a different tag. +const hashLockfileFor = (tag: string) => () => `hash:${tag}` + +describe('tryLockfileVerificationCache', () => { + test('miss when the cache file does not exist', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(false) + }) + + test('miss when the lockfile path is not in the cache', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + // Seed an unrelated record. + recordVerification(cacheDir, { + lockfilePath: path.join(tmpDir, 'other-lockfile.yaml'), + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('path.join(tmpDir'), + }) + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(false) + }) + + test('stat-only hit when size, mtime, and inode all match', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(true) + }) + + test('stat shortcut bails on size mismatch and falls through to hash lookup', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('content-a') }) + + // Append bytes — file size changes, so byPath stat-match bails. + // The lookup falls through to the hash lookup. Today's content + // produces a different hash, so byHash misses → overall miss. + await fs.promises.appendFile(lockfilePath, 'extra: bytes\n') + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], + hashLockfile: hashLockfileFor('content-b'), + }) + expect(result.hit).toBe(false) + }) + + test('hash-fallback hit when size matches but mtime/inode were reset', async () => { + // Simulate a CI checkout: same content, fresh inode + mtime. + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + + // Write to a different path, unlink the original, rename in. This + // guarantees a different inode while keeping the same content. + const sibling = path.join(tmpDir, 'pnpm-lock-2.yaml') + await fs.promises.writeFile(sibling, 'lockfileVersion: \'9.0\'\n') + await fs.promises.rm(lockfilePath) + await fs.promises.rename(sibling, lockfilePath) + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(true) + }) + + test('miss when content changed even if size happens to match', async () => { + await fs.promises.writeFile(lockfilePath, 'aaaaaaaaaaaa') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('content-a') }) + + // Same byte length, different content — hash check is what rejects. + await fs.promises.rm(lockfilePath) + await fs.promises.writeFile(lockfilePath, 'bbbbbbbbbbbb') + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], + hashLockfile: hashLockfileFor('content-b'), + }) + expect(result.hit).toBe(false) + }) + + test('miss when a verifier rejects the cached policy', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + + // Today's policy is stricter than the cached one. + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(120)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(false) + }) + + test('hit when a verifier accepts the cached policy', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(1440)], hashLockfile: hashLockfileFor('lockfilePath') }) + + // Today's policy is weaker — the stricter cached run still covers it. + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(true) + }) + + test('miss when the cached policy lacks a field the current verifier reads', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + // Seed a record whose policy doesn't have minimumReleaseAge. + recordVerification(cacheDir, { + lockfilePath, + verifiers: [{ + policy: { someOther: 'value' }, + canTrustPastCheck: () => true, + }], + hashLockfile: hashLockfileFor('lockfilePath'), + }) + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], + hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(false) + }) + + test('hit when every verifier trusts its share of the merged cached policy', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + const verifiers: VerifierCacheIdentity[] = [ + mraVerifier(60), + { + policy: { trustedPublishers: ['foo-org'] }, + canTrustPastCheck: (cached) => Array.isArray(cached.trustedPublishers) && + cached.trustedPublishers.includes('foo-org'), + }, + ] + const hashLockfile = hashLockfileFor('lockfilePath') + recordVerification(cacheDir, { lockfilePath, verifiers, hashLockfile }) + + const result = tryLockfileVerificationCache(cacheDir, { lockfilePath, verifiers, hashLockfile }) + expect(result.hit).toBe(true) + }) + + test('miss when the lockfile no longer exists', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + await fs.promises.rm(lockfilePath) + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(false) + }) + + test('hit at a new path when the content matches a cached hash (worktree case)', async () => { + // Both paths represent the same lockfile content, so they share a + // hash. This is the whole point: the cache should recognize the + // content regardless of path. + const sharedHash = hashLockfileFor('shared-content') + + // First install: lockfile at `lockfilePath` gets verified and cached. + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: sharedHash }) + + // Second install: a different worktree with the same lockfile content, + // so a different absolute path but identical bytes. The stat shortcut + // misses (different path), but the content lookup hits via hash. + const worktreeLockfile = path.join(tmpDir, 'worktree', 'pnpm-lock.yaml') + await fs.promises.mkdir(path.dirname(worktreeLockfile), { recursive: true }) + await fs.promises.writeFile(worktreeLockfile, 'lockfileVersion: \'9.0\'\n') + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath: worktreeLockfile, + verifiers: [mraVerifier(60)], + hashLockfile: sharedHash, + }) + expect(result.hit).toBe(true) + + // The hit appended a refreshed record for the new path, so the next + // install on the worktree path takes the stat shortcut (no rehash). + // We can't directly observe whether the shortcut fires, but we can + // confirm the cache file now has at least one record naming the + // worktree path. + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + const raw = await fs.promises.readFile(cacheFile, 'utf8') + const paths = raw.split('\n').filter(Boolean) + .map((line) => (JSON.parse(line) as { lockfile: { path: string } }).lockfile.path) + expect(paths).toEqual(expect.arrayContaining([worktreeLockfile])) + }) + + test('latest record per path wins when the cache has multiple appends', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + + // Earlier record under a stricter cutoff. + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + // Later record under a weaker cutoff that does satisfy 120. + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(120)], hashLockfile: hashLockfileFor('lockfilePath') }) + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(120)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(true) + }) + + test('malformed lines are ignored, not propagated', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + await fs.promises.mkdir(cacheDir, { recursive: true }) + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + await fs.promises.writeFile(cacheFile, '{not json\n\n') + + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + + const result = tryLockfileVerificationCache(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + expect(result.hit).toBe(true) + }) +}) + +describe('recordVerification', () => { + test('writes a JSONL record with a merged policy bag', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + const raw = await fs.promises.readFile(cacheFile, 'utf8') + const lines = raw.split('\n').filter(Boolean) + expect(lines).toHaveLength(1) + const record = JSON.parse(lines[0]) as Record & { + lockfile: Record + } + expect(record).toMatchObject({ + lockfile: { path: lockfilePath }, + policy: { minimumReleaseAge: 60 }, + }) + expect(typeof record.lockfile.hash).toBe('string') + expect(typeof record.verifiedAt).toBe('string') + expect(typeof record.lockfile.size).toBe('number') + expect(typeof record.lockfile.mtimeNs).toBe('string') + expect(typeof record.lockfile.inode).toBe('string') + }) + + test('merges policy fields across verifiers into a single bag', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { + lockfilePath, + verifiers: [ + mraVerifier(60), + { + policy: { trustedPublishers: ['foo-org', 'pnpm'] }, + canTrustPastCheck: () => true, + }, + ], + hashLockfile: hashLockfileFor('lockfilePath'), + }) + + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + const raw = await fs.promises.readFile(cacheFile, 'utf8') + const record = JSON.parse(raw.trim()) as { policy: Record } + expect(record.policy).toEqual({ + minimumReleaseAge: 60, + trustedPublishers: ['foo-org', 'pnpm'], + }) + }) + + test('shared policy field is stored once, not duplicated', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + // Two verifiers both contribute minimumReleaseAge with the same value + // — the merged bag stores it once. + recordVerification(cacheDir, { + lockfilePath, + verifiers: [mraVerifier(60), mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath'), + }) + + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + const raw = await fs.promises.readFile(cacheFile, 'utf8') + const record = JSON.parse(raw.trim()) as { policy: Record } + expect(record.policy).toEqual({ minimumReleaseAge: 60 }) + }) + + test('silently skips when the lockfile is missing', async () => { + expect( + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + ).toBeUndefined() + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + await expect(fs.promises.access(cacheFile)).rejects.toThrow() + }) + + test('appends without rewriting previous lines', async () => { + await fs.promises.writeFile(lockfilePath, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { lockfilePath, verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('lockfilePath') }) + + const otherLockfile = path.join(tmpDir, 'other-lockfile.yaml') + await fs.promises.writeFile(otherLockfile, 'lockfileVersion: \'9.0\'\n') + recordVerification(cacheDir, { + lockfilePath: otherLockfile, + verifiers: [mraVerifier(60)], hashLockfile: hashLockfileFor('otherLockfile'), + }) + + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + const raw = await fs.promises.readFile(cacheFile, 'utf8') + const lines = raw.split('\n').filter(Boolean) + expect(lines).toHaveLength(2) + const paths = lines.map((line) => (JSON.parse(line) as { lockfile: { path: string } }).lockfile.path) + expect(paths).toEqual(expect.arrayContaining([lockfilePath, otherLockfile])) + }) +}) diff --git a/installing/deps-installer/test/utils/testDefaults.ts b/installing/deps-installer/test/utils/testDefaults.ts index 1df64f7a66..a40d1667ea 100644 --- a/installing/deps-installer/test/utils/testDefaults.ts +++ b/installing/deps-installer/test/utils/testDefaults.ts @@ -28,7 +28,7 @@ export function testDefaults ( registries: Registries storeController: StoreController storeDir: string - verifyResolution?: ResolutionVerifier + resolutionVerifiers: ResolutionVerifier[] } & T { // Forward minimumReleaseAge policy into the Client so it builds the @@ -39,7 +39,7 @@ export function testDefaults ( ...(opts?.minimumReleaseAgeStrict != null ? { minimumReleaseAgeStrict: opts.minimumReleaseAgeStrict } : {}), ...(opts?.minimumReleaseAgeExclude != null ? { minimumReleaseAgeExclude: opts.minimumReleaseAgeExclude } : {}), } - const { storeController, storeDir, cacheDir, verifyResolution } = createTempStore({ + const { storeController, storeDir, cacheDir, resolutionVerifiers } = createTempStore({ ...opts, clientOptions: { ...(opts?.registries != null ? { registries: opts.registries } : {}), @@ -57,7 +57,7 @@ export function testDefaults ( }, storeController, storeDir, - verifyResolution, + resolutionVerifiers, ...opts, } as ( InstallOptions & @@ -66,7 +66,7 @@ export function testDefaults ( registries: Registries storeController: StoreController storeDir: string - verifyResolution?: ResolutionVerifier + resolutionVerifiers: ResolutionVerifier[] } & T ) diff --git a/pnpm/test/install/minimumReleaseAge.ts b/pnpm/test/install/minimumReleaseAge.ts index c390a53746..077808fe37 100644 --- a/pnpm/test/install/minimumReleaseAge.ts +++ b/pnpm/test/install/minimumReleaseAge.ts @@ -1,3 +1,6 @@ +import fs from 'node:fs' +import path from 'node:path' + import { describe, expect, test } from '@jest/globals' import { prepare } from '@pnpm/prepare' import { writeYamlFileSync } from 'write-yaml-file' @@ -73,6 +76,49 @@ describe('lockfile minimumReleaseAge verification', () => { ) }) + test('records the verification cache so a repeat install reuses it', async () => { + // Step 1: populate the lockfile with no policy. is-positive@1.0.0 + // was published in 2014, so a 1-minute cutoff later will pass it. + prepare({ + dependencies: { 'is-positive': '1.0.0' }, + }) + await execPnpm([PUBLIC_REGISTRY, 'install']) + + // Step 2: turn the policy on. The post-resolution gate now runs + // against the existing lockfile and writes a cache record. + const cacheDir = path.resolve('pnpm-cache') + writeYamlFileSync('pnpm-workspace.yaml', { + minimumReleaseAge: 1, + minimumReleaseAgeStrict: true, + cacheDir, + }) + execPnpmSync( + [PUBLIC_REGISTRY, 'install', '--frozen-lockfile'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + expect(fs.existsSync(cacheFile)).toBe(true) + const lines = fs.readFileSync(cacheFile, 'utf8').split('\n').filter(Boolean) + expect(lines.length).toBeGreaterThanOrEqual(1) + const record = JSON.parse(lines.at(-1)!) as { + lockfile: { hash: string, path: string } + policy: Record + } + expect(record.lockfile.path).toBe(path.resolve('pnpm-lock.yaml')) + expect(record.lockfile.hash).toMatch(/^[a-z0-9+/=]+$/i) + expect(record.policy).toMatchObject({ minimumReleaseAge: 1 }) + + // Step 3: another install with the same lockfile + policy. The cache + // short-circuits the gate (asserting that requires registry-call + // instrumentation we don't have at this layer, but install + // completing cleanly is the smoke test). + execPnpmSync( + [PUBLIC_REGISTRY, 'install', '--frozen-lockfile'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + }) + test('install is unaffected by minimumReleaseAge when strict mode is explicitly off', () => { // The config reader auto-enables strict mode the moment a user // explicitly sets `minimumReleaseAge`, so opting out requires an diff --git a/resolving/default-resolver/src/index.ts b/resolving/default-resolver/src/index.ts index a1592c0567..00849f52ab 100644 --- a/resolving/default-resolver/src/index.ts +++ b/resolving/default-resolver/src/index.ts @@ -157,16 +157,21 @@ export type ResolutionVerifierFactoryOptions = } /** - * Companion to {@link createResolver}. Combines the resolver-specific - * verifier factories (today: npm) into a single {@link ResolutionVerifier}, - * dispatching by resolution shape. Returns `undefined` when none of the - * underlying resolvers have any active policy — letting callers cheaply - * decide whether to iterate at all. + * Companion to {@link createResolver}. Collects the resolver-specific + * verifier factories (today: npm) into a list. Returns an empty array + * when no policy is active — callers can cheaply decide whether to + * iterate at all by checking `verifiers.length`. + * + * Future protocols (jsr, git, attestation, etc.) plug in here by pushing + * their own `ResolutionVerifier` onto the list. Each verifier handles + * its own protocol short-circuit inside `verify` (returns `{ ok: true }` + * for resolutions outside its scope), so dispatch happens naturally at + * the install side — no combinator needed. */ -export function createResolutionVerifier ( +export function createResolutionVerifiers ( fetchFromRegistry: FetchFromRegistry, opts: ResolutionVerifierFactoryOptions -): ResolutionVerifier | undefined { +): ResolutionVerifier[] { const fetchOpts = { fetch: fetchFromRegistry, retry: opts.retry ?? {}, @@ -174,6 +179,7 @@ export function createResolutionVerifier ( fetchWarnTimeoutMs: opts.fetchWarnTimeoutMs ?? 10_000, } const getAuthHeaderValueByURI = createGetAuthHeaderByURI(opts.configByUri ?? {}, opts.registries.default) + const verifiers: ResolutionVerifier[] = [] const npmVerifier = createNpmResolutionVerifier({ minimumReleaseAge: opts.minimumReleaseAge, minimumReleaseAgeStrict: opts.minimumReleaseAgeStrict, @@ -185,16 +191,6 @@ export function createResolutionVerifier ( cacheDir: opts.cacheDir, now: opts.now, }) - // Future protocols (jsr, git, etc.) plug in here. When every sub-verifier - // is undefined, the combined verifier is too — caller short-circuits. - // - // When a second verifier lands, this combinator needs to dispatch by - // resolution shape (so e.g. a git verifier doesn't run on npm-registry - // entries and vice versa). The classification logic should live as a - // shared helper in `@pnpm/resolving.resolver-base` — `pickFetcher` in - // `fetching/pick-fetcher` already classifies the same shape today - // (resolution.type / tarball / gitHosted / integrity); reconcile both - // call sites onto one classifier rather than re-deriving it per verifier. - if (!npmVerifier) return undefined - return async (resolution, ctx) => npmVerifier(resolution, ctx) + if (npmVerifier) verifiers.push(npmVerifier) + return verifiers } diff --git a/resolving/npm-resolver/src/createNpmResolutionVerifier.ts b/resolving/npm-resolver/src/createNpmResolutionVerifier.ts index 099c9a71f0..c105d38da9 100644 --- a/resolving/npm-resolver/src/createNpmResolutionVerifier.ts +++ b/resolving/npm-resolver/src/createNpmResolutionVerifier.ts @@ -108,7 +108,9 @@ export function createNpmResolutionVerifier ( return promise } - return async (resolution, { name, version }) => { + const minimumReleaseAge = opts.minimumReleaseAge + + const verify: ResolutionVerifier['verify'] = async (resolution, { name, version }) => { if (!isNpmRegistryResolution(resolution)) return { ok: true } // Non-semver versions identify URL tarballs, file: refs, git refs, etc. // The age policy doesn't apply and a registry lookup would 404. @@ -157,6 +159,20 @@ export function createNpmResolutionVerifier ( } return { ok: true } } + return { + verify, + policy: { minimumReleaseAge }, + canTrustPastCheck: (cached) => { + // A previously cached run under a larger cutoff (stricter window) + // is trustworthy under a smaller current one — its set of + // accepted versions is a subset of today's. The reverse — + // tightening the cutoff — invalidates the cached run: versions + // that passed before may now be in-window. Non-number cached + // values come from an older record shape and aren't trusted. + const past = cached.minimumReleaseAge + return typeof past === 'number' && past >= minimumReleaseAge + }, + } } function pickRegistryForVersion ( diff --git a/resolving/resolver-base/src/index.ts b/resolving/resolver-base/src/index.ts index a7bad9ae86..3546948904 100644 --- a/resolving/resolver-base/src/index.ts +++ b/resolving/resolver-base/src/index.ts @@ -93,19 +93,41 @@ export type ResolutionVerification = | { ok: false, code: string, reason: string } /** - * Optional companion to a resolver factory. Lets each resolver enforce - * policies (e.g. minimumReleaseAge for npm) against an already-resolved - * entry from a lockfile without re-doing resolution. + * Optional companion to a resolver factory. * - * The verifier inspects the `resolution` shape to decide whether the entry + * `verify` inspects the `resolution` shape to decide whether the entry * is within its protocol; for entries outside its protocol it should - * return `{ ok: true }`. Combined verifiers (in default-resolver) dispatch - * across underlying resolver-specific verifiers. + * return `{ ok: true }`. The install side fans out across the verifier + * list rather than asking a combinator to dispatch. + * + * `policy` and `canTrustPastCheck` describe the verifier's cache + * contract. Policies from every active verifier are merged into a + * single shared bag stored alongside the lockfile hash; the + * install-side verification cache reads them to decide if a previous + * run on the same lockfile is still trustworthy under today's policy + * without re-issuing the registry round-trips that `verify` would. + * Verifiers that check the same logical policy (e.g. minimumReleaseAge + * across registries) name it the same and share the cache slot. */ -export type ResolutionVerifier = ( - resolution: Resolution, - ctx: { name: string, version: string } -) => Promise +export interface ResolutionVerifier { + verify: (resolution: Resolution, ctx: { name: string, version: string }) => Promise + /** + * Snapshot of the policy fields this verifier enforces. Merged with + * every other active verifier's `policy` into the cache record. A + * field shared across verifiers (same key) should carry the same + * value; if it doesn't, the last verifier in the list wins. + */ + policy: Record + /** + * Returns true when the previously cached policy (the merged snapshot + * from the last successful run) can be trusted to still satisfy what + * this verifier currently demands. Reads whichever fields the + * verifier owns; missing or non-conforming values (e.g. an older + * record shape) should return false. A loosened policy can trust a + * stricter cached run; a tightened policy cannot. + */ + canTrustPastCheck: (cachedPolicy: Record) => boolean +} /** Concrete platform selector used when picking a variant from a VariationsResolution. */ export interface PlatformSelector { diff --git a/store/connection-manager/src/createNewStoreController.ts b/store/connection-manager/src/createNewStoreController.ts index 76a9254761..18a610f459 100644 --- a/store/connection-manager/src/createNewStoreController.ts +++ b/store/connection-manager/src/createNewStoreController.ts @@ -63,7 +63,7 @@ export type CreateNewStoreControllerOptions = CreateResolverOptions & Pick { +): Promise<{ ctrl: StoreController, dir: string, resolutionVerifiers: ResolutionVerifier[] }> { const fullMetadata = opts.fetchFullMetadata ?? ( ( opts.resolutionMode === 'time-based' || @@ -72,7 +72,7 @@ export async function createNewStoreController ( ) await fs.mkdir(opts.storeDir, { recursive: true }) const storeIndex = new StoreIndex(opts.storeDir) - const { resolve, fetchers, clearResolutionCache, verifyResolution } = createClient({ + const { resolve, fetchers, clearResolutionCache, resolutionVerifiers } = createClient({ customResolvers: opts.hooks?.customResolvers, customFetchers: opts.hooks?.customFetchers, unsafePerm: opts.unsafePerm, @@ -145,6 +145,6 @@ export async function createNewStoreController ( storeIndex, }), dir: opts.storeDir, - verifyResolution, + resolutionVerifiers, } } diff --git a/store/connection-manager/src/index.ts b/store/connection-manager/src/index.ts index 55d6d00850..46f6728424 100644 --- a/store/connection-manager/src/index.ts +++ b/store/connection-manager/src/index.ts @@ -17,7 +17,7 @@ export type CreateStoreControllerOptions = Omit Date: Sun, 17 May 2026 06:24:24 -0500 Subject: [PATCH 002/169] fix: tolerate padded auth base64 (#11694) * fix: tolerate padded auth base64 * fix: avoid regex in auth padding normalization --- .changeset/clear-password-padding.md | 6 +++ config/reader/src/parseCreds.ts | 44 ++++++++++++++++++- config/reader/test/parseCreds.test.ts | 37 ++++++++++++++++ pacquet/crates/config/src/npmrc_auth/tests.rs | 3 ++ 4 files changed, 88 insertions(+), 2 deletions(-) create mode 100644 .changeset/clear-password-padding.md diff --git a/.changeset/clear-password-padding.md b/.changeset/clear-password-padding.md new file mode 100644 index 0000000000..fa6f7b932d --- /dev/null +++ b/.changeset/clear-password-padding.md @@ -0,0 +1,6 @@ +--- +"@pnpm/config.reader": patch +"pnpm": patch +--- + +Allow redundant trailing base64 padding in `.npmrc` auth values and report invalid auth base64 with a pnpm error. diff --git a/config/reader/src/parseCreds.ts b/config/reader/src/parseCreds.ts index 2f3d8ebe67..e9bec19d32 100644 --- a/config/reader/src/parseCreds.ts +++ b/config/reader/src/parseCreds.ts @@ -59,7 +59,7 @@ function parseBasicAuth ({ authPassword, }: Pick): BasicAuth | undefined { if (authPairBase64) { - const pair = atob(authPairBase64) + const pair = decodeBase64Credential(authPairBase64, '_auth') const colonIndex = pair.indexOf(':') if (colonIndex < 0) { throw new AuthMissingSeparatorError() @@ -70,12 +70,44 @@ function parseBasicAuth ({ } if (authUsername && authPassword) { - return { username: authUsername, password: atob(authPassword) } + return { username: authUsername, password: decodeBase64Credential(authPassword, '_password') } } return undefined } +function decodeBase64Credential (value: string, key: '_auth' | '_password'): string { + try { + return atob(value) + } catch { + const normalizedValue = normalizeBase64Padding(value) + if (normalizedValue !== value) { + try { + return atob(normalizedValue) + } catch {} + } + throw new AuthBase64DecodeError(key) + } +} + +function normalizeBase64Padding (value: string): string { + let paddingStart = value.length + while (paddingStart > 0 && value[paddingStart - 1] === '=') { + paddingStart-- + } + + const valueWithoutPadding = value.slice(0, paddingStart) + if (!valueWithoutPadding) return value + + const remainder = valueWithoutPadding.length % 4 + if (remainder === 1) return value + + return valueWithoutPadding.padEnd( + valueWithoutPadding.length + (4 - remainder) % 4, + '=' + ) +} + export class AuthMissingSeparatorError extends PnpmError { constructor () { super('AUTH_MISSING_SEPARATOR', 'No separator found in the decoded form of _auth', { @@ -84,6 +116,14 @@ export class AuthMissingSeparatorError extends PnpmError { } } +export class AuthBase64DecodeError extends PnpmError { + constructor (key: '_auth' | '_password') { + super('AUTH_INVALID_BASE64', `Failed to decode ${key} as base64`, { + hint: `${key} must contain a base64-encoded ${key === '_auth' ? ':' : 'password'} value`, + }) + } +} + /** Characters reserved for more advanced features in the future. */ const RESERVED_CHARACTERS = new Set(['$', '%', '`', '"', "'"]) diff --git a/config/reader/test/parseCreds.test.ts b/config/reader/test/parseCreds.test.ts index 2398963f64..6b520c46a5 100644 --- a/config/reader/test/parseCreds.test.ts +++ b/config/reader/test/parseCreds.test.ts @@ -1,6 +1,7 @@ import { describe, expect, test } from '@jest/globals' import { + AuthBase64DecodeError, AuthMissingSeparatorError, type Creds, parseCreds, @@ -52,6 +53,23 @@ describe('parseCreds', () => { })).toThrow(new AuthMissingSeparatorError()) }) + test('authPairBase64 allows redundant trailing padding', () => { + expect(parseCreds({ + authPairBase64: `${btoa('foo:bar')}=`, + })).toStrictEqual({ + basicAuth: { + username: 'foo', + password: 'bar', + }, + } as Creds) + }) + + test('authPairBase64 must be base64', () => { + expect(() => parseCreds({ + authPairBase64: 'foo*bar', + })).toThrow(new AuthBase64DecodeError('_auth')) + }) + test('authUsername and authPassword', () => { expect(parseCreds({ authUsername: 'foo', @@ -72,6 +90,25 @@ describe('parseCreds', () => { })).toBeUndefined() }) + test('authPassword allows redundant trailing padding', () => { + expect(parseCreds({ + authUsername: 'foo', + authPassword: `${btoa('bar')}=`, + })).toStrictEqual({ + basicAuth: { + username: 'foo', + password: 'bar', + }, + } as Creds) + }) + + test('authPassword must be base64', () => { + expect(() => parseCreds({ + authUsername: 'foo', + authPassword: 'bar*baz', + })).toThrow(new AuthBase64DecodeError('_password')) + }) + test('tokenHelper', () => { expect(parseCreds({ tokenHelper: 'example-token-helper --foo --bar baz', diff --git a/pacquet/crates/config/src/npmrc_auth/tests.rs b/pacquet/crates/config/src/npmrc_auth/tests.rs index ec841a282b..c400a25ef1 100644 --- a/pacquet/crates/config/src/npmrc_auth/tests.rs +++ b/pacquet/crates/config/src/npmrc_auth/tests.rs @@ -349,6 +349,9 @@ fn base64_decode_covers_every_alphabet_branch() { assert_eq!(base64_decode("fn5+").as_deref(), Some("~~~")); // `=` padding short-circuits the loop on a 2-byte input. assert_eq!(base64_decode("aGk=").as_deref(), Some("hi")); + // Redundant trailing padding is ignored, matching pnpm's tolerant + // credential decoder. + assert_eq!(base64_decode("aGk===").as_deref(), Some("hi")); // Invalid byte returns None so the parser keeps the raw // value verbatim. `*` is not in the alphabet. assert_eq!(base64_decode("not*base64"), None); From 5dc8be8a42257e512d95d7491b750e95620136f6 Mon Sep 17 00:00:00 2001 From: Zoltan Kochan Date: Sun, 17 May 2026 13:25:05 +0200 Subject: [PATCH 003/169] fix(graph-hasher): resolve GVS engine per-snapshot for runtime-pinned deps (#11693) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #11690. A dependency that declares `engines.runtime` in its manifest carries the desugared `dependencies.node: 'runtime:'` pin in the lockfile, and pnpm's bin linker spawns that dep's lifecycle scripts through the pinned Node downloaded into `/node_modules/node/`. The GVS hash and the side-effects-cache key prefix were still anchored to the install-wide runtime — so the pinning snapshot's slot encoded the wrong Node major, and a reinstall on the same host could read the cached side-effects under a key whose `;;node` triple disagreed with the Node the build actually ran on. Per-snapshot resolution now matches what `bins/linker` already does on a per-package basis: a snapshot's own pin wins; the install-wide value (from #11689's `findRuntimeNodeVersion`) is the fallback. ### TypeScript - `deps/graph-hasher/src/index.ts:72-77` — adds `readSnapshotRuntimePin(children)`: pulls the bare Node version from a graph node's `children.node` entry when that points at a `node@runtime:` snapshot. Factors out a small `extractRuntimeNodeVersion(snapshotKey)` parser shared with `findRuntimeNodeVersion`. - `deps/graph-hasher/src/index.ts:115-116,245-246` — `calcDepState` and `calcGraphNodeHash` consult `readSnapshotRuntimePin(graph[depPath].children)` first and only fall back to the install-wide `nodeVersion` parameter when the snapshot doesn't pin its own Node. No caller changes required — install-wide fallback continues to be computed via `findRuntimeNodeVersion(Object.keys(graph))` at each call site. - **Refactor (separate commit):** `findRuntimeNodeVersion` moved from `@pnpm/engine.runtime.system-node-version` to `@pnpm/deps.graph-hasher` (along with the new `readSnapshotRuntimePin`). `system-node-version` is about probing the *host* Node — `getSystemNodeVersion`, `engineName`. The lockfile-shape parsers fit better next to the package that actually composes the engine string. Every caller already depended on graph-hasher, so no new deps; six packages drop the now-unused dependency on `system-node-version`. ### Pacquet - `pacquet/crates/package-manager/src/install_frozen_lockfile.rs:1309-1345` — new `find_own_runtime_node_major(snapshot)` reads a snapshot's `dependencies` for a `node` entry with `Prefix::Runtime`, returning the bare major. - `pacquet/crates/package-manager/src/virtual_store_layout.rs:178-205` — `VirtualStoreLayout::new` resolves engine per-snapshot inside the hash loop via `engine_name(own_major, None, None)` when the snapshot pins, otherwise inherits the install-wide `engine` argument. ### Migration Snapshots of dependencies that declare their own `engines.runtime` re-hash under that dep's pinned Node instead of the install-wide value. Old slots become prune-eligible on next install. --- .changeset/gvs-engine-name-shell-node.md | 4 +- .../gvs-engine-per-snapshot-runtime-pin.md | 17 ++ building/after-install/package.json | 1 - building/after-install/src/index.ts | 3 +- building/after-install/tsconfig.json | 3 - building/during-install/package.json | 1 - building/during-install/src/index.ts | 3 +- building/during-install/tsconfig.json | 3 - cspell.json | 2 + deps/graph-builder/package.json | 1 - .../src/iteratePkgsForVirtualStore.ts | 2 +- deps/graph-builder/tsconfig.json | 3 - deps/graph-hasher/src/index.ts | 111 +++++++++++-- .../test/calcGraphNodeHash.test.ts | 119 +++++++++++++ deps/graph-hasher/test/index.ts | 59 ++++++- .../runtime/system-node-version/src/index.ts | 37 +---- .../test/getSystemNodeVersion.test.ts | 19 +-- exec/commands/test/dlx.e2e.ts | 4 +- installing/deps-installer/package.json | 1 - installing/deps-installer/src/install/link.ts | 3 +- installing/deps-installer/tsconfig.json | 3 - installing/deps-resolver/package.json | 1 - installing/deps-resolver/src/index.ts | 3 +- installing/deps-resolver/tsconfig.json | 3 - installing/deps-restorer/package.json | 1 - installing/deps-restorer/src/index.ts | 3 +- .../deps-restorer/src/linkHoistedModules.ts | 3 +- installing/deps-restorer/tsconfig.json | 3 - .../src/install_frozen_lockfile.rs | 99 +++++++++++ .../src/virtual_store_layout.rs | 157 ++++++++++++++++-- pnpm-lock.yaml | 18 -- 31 files changed, 551 insertions(+), 139 deletions(-) create mode 100644 .changeset/gvs-engine-per-snapshot-runtime-pin.md diff --git a/.changeset/gvs-engine-name-shell-node.md b/.changeset/gvs-engine-name-shell-node.md index da97ba3b04..9d72caf56b 100644 --- a/.changeset/gvs-engine-name-shell-node.md +++ b/.changeset/gvs-engine-name-shell-node.md @@ -19,8 +19,8 @@ Three changes: -- `@pnpm/engine.runtime.system-node-version` now exports `engineName(nodeVersion?)` and `findRuntimeNodeVersion(snapshotKeys)`. `engineName()` resolves the version in this order: explicit override → `getSystemNodeVersion()` (which already prefers `node --version` over `process.version` in SEA contexts) → `process.version`. `findRuntimeNodeVersion` scans an iterable of lockfile snapshot keys for a `node@runtime:` entry and returns its bare version string. -- `@pnpm/deps.graph-hasher`'s `calcDepState` and `calcGraphNodeHash`/`iterateHashedGraphNodes` now accept a `nodeVersion?` (in the options bag for the first, as a trailing parameter / ctx field for the others), forwarded to `engineName()`. The default (no override) preserves the pre-change behaviour. The legacy `ENGINE_NAME` constant in `@pnpm/constants` is unchanged so external consumers and existing tests keep working; in non-SEA, non-pinned contexts every value lines up. +- `@pnpm/engine.runtime.system-node-version` now exports `engineName(nodeVersion?)`. Resolves the version in this order: explicit override → `getSystemNodeVersion()` (which already prefers `node --version` over `process.version` in SEA contexts) → `process.version`. +- `@pnpm/deps.graph-hasher` now exports `findRuntimeNodeVersion(snapshotKeys)` — scans an iterable of lockfile snapshot keys for a `node@runtime:` entry and returns its bare version string. `calcDepState` and `calcGraphNodeHash`/`iterateHashedGraphNodes` accept a `nodeVersion?` (in the options bag for the first, as a trailing parameter / ctx field for the others), forwarded to `engineName()`. The default (no override) preserves the pre-change behaviour. The legacy `ENGINE_NAME` constant in `@pnpm/constants` is unchanged so external consumers and existing tests keep working; in non-SEA, non-pinned contexts every value lines up. - Every install-side caller of the graph-hasher (`@pnpm/installing.deps-resolver`, `@pnpm/installing.deps-restorer`, `@pnpm/installing.deps-installer`, `@pnpm/building.during-install`, `@pnpm/building.after-install`, `@pnpm/deps.graph-builder`) now derives the project's pinned runtime via `findRuntimeNodeVersion(Object.keys(graph))` once per invocation and threads it through. On upgrade, two one-time GVS slot churns are possible: diff --git a/.changeset/gvs-engine-per-snapshot-runtime-pin.md b/.changeset/gvs-engine-per-snapshot-runtime-pin.md new file mode 100644 index 0000000000..4fcd9f04da --- /dev/null +++ b/.changeset/gvs-engine-per-snapshot-runtime-pin.md @@ -0,0 +1,17 @@ +--- +"@pnpm/deps.graph-hasher": minor +"pnpm": patch +--- + +**fix**: resolve the GVS hash's engine portion per-snapshot when a dependency declares its own `engines.runtime`, instead of using an install-wide value. + +Pnpm's resolver desugars a dep's `engines.runtime` into `dependencies.node: 'runtime:'`, and the bin linker spawns that dep's lifecycle scripts through the pinned Node downloaded into `/node_modules/node/`. The GVS hash and the side-effects-cache key prefix were still anchored to the install-wide runtime — so a pinning snapshot's slot encoded the wrong Node major, and a reinstall on the same host could read the cached side-effects under a key whose `;;node` triple disagreed with the Node the build actually ran on. + +Per-snapshot resolution now matches what `bins/linker` already does on a per-package basis: + +- `@pnpm/deps.graph-hasher` adds `readSnapshotRuntimePin(children)` — reads the `node` entry from one snapshot's graph children and extracts the version from a `node@runtime:` value. Pairs with the existing `findRuntimeNodeVersion(snapshotKeys)` install-wide fallback (also now exported from `@pnpm/deps.graph-hasher` rather than `@pnpm/engine.runtime.system-node-version`, where it was a poor fit — `system-node-version` is about probing the host Node, not parsing lockfile-derived strings). +- `calcDepState` and `calcGraphNodeHash` consult `readSnapshotRuntimePin(graph[depPath].children)` first and only fall back to the install-wide `nodeVersion` parameter when the snapshot doesn't pin its own Node. + +Pacquet mirrors the same precedence at the `calc_graph_node_hash` call site in `package-manager/src/virtual_store_layout.rs` — a new `find_own_runtime_node_major(snapshot)` helper reads each snapshot's `dependencies` for a `node` entry with `Prefix::Runtime` and overrides the install-wide engine when present. + +On upgrade, snapshots of dependencies that declare their own `engines.runtime` re-hash under that dep's pinned Node instead of the install-wide value. The old slots become prune-eligible. Closes [#11690](https://github.com/pnpm/pnpm/issues/11690). diff --git a/building/after-install/package.json b/building/after-install/package.json index 6a69ad2172..63886a136a 100644 --- a/building/after-install/package.json +++ b/building/after-install/package.json @@ -41,7 +41,6 @@ "@pnpm/deps.graph-hasher": "workspace:*", "@pnpm/deps.graph-sequencer": "workspace:*", "@pnpm/deps.path": "workspace:*", - "@pnpm/engine.runtime.system-node-version": "workspace:*", "@pnpm/error": "workspace:*", "@pnpm/exec.lifecycle": "workspace:*", "@pnpm/installing.context": "workspace:*", diff --git a/building/after-install/src/index.ts b/building/after-install/src/index.ts index d201c3b74b..78f838b30c 100644 --- a/building/after-install/src/index.ts +++ b/building/after-install/src/index.ts @@ -10,10 +10,9 @@ import { WANTED_LOCKFILE, } from '@pnpm/constants' import { skippedOptionalDependencyLogger } from '@pnpm/core-loggers' -import { calcDepState, type DepsStateCache, lockfileToDepGraph } from '@pnpm/deps.graph-hasher' +import { calcDepState, type DepsStateCache, findRuntimeNodeVersion, lockfileToDepGraph } from '@pnpm/deps.graph-hasher' import { graphSequencer } from '@pnpm/deps.graph-sequencer' import * as dp from '@pnpm/deps.path' -import { findRuntimeNodeVersion } from '@pnpm/engine.runtime.system-node-version' import { PnpmError } from '@pnpm/error' import { runLifecycleHooksConcurrently, diff --git a/building/after-install/tsconfig.json b/building/after-install/tsconfig.json index 0216bd1f6d..3badc52cb6 100644 --- a/building/after-install/tsconfig.json +++ b/building/after-install/tsconfig.json @@ -39,9 +39,6 @@ { "path": "../../deps/path" }, - { - "path": "../../engine/runtime/system-node-version" - }, { "path": "../../exec/lifecycle" }, diff --git a/building/during-install/package.json b/building/during-install/package.json index 7612b22521..f04f482ddd 100644 --- a/building/during-install/package.json +++ b/building/during-install/package.json @@ -39,7 +39,6 @@ "@pnpm/deps.graph-hasher": "workspace:*", "@pnpm/deps.graph-sequencer": "workspace:*", "@pnpm/deps.path": "workspace:*", - "@pnpm/engine.runtime.system-node-version": "workspace:*", "@pnpm/error": "workspace:*", "@pnpm/exec.lifecycle": "workspace:*", "@pnpm/fs.hard-link-dir": "workspace:*", diff --git a/building/during-install/src/index.ts b/building/during-install/src/index.ts index 4715786d09..e9f5b2f7ac 100644 --- a/building/during-install/src/index.ts +++ b/building/during-install/src/index.ts @@ -6,8 +6,7 @@ import util from 'node:util' import { linkBins, linkBinsOfPackages } from '@pnpm/bins.linker' import { getWorkspaceConcurrency } from '@pnpm/config.reader' import { skippedOptionalDependencyLogger } from '@pnpm/core-loggers' -import { calcDepState, type DepsStateCache } from '@pnpm/deps.graph-hasher' -import { findRuntimeNodeVersion } from '@pnpm/engine.runtime.system-node-version' +import { calcDepState, type DepsStateCache, findRuntimeNodeVersion } from '@pnpm/deps.graph-hasher' import { PnpmError } from '@pnpm/error' import { runPostinstallHooks } from '@pnpm/exec.lifecycle' import { logger } from '@pnpm/logger' diff --git a/building/during-install/tsconfig.json b/building/during-install/tsconfig.json index 4d758c2486..d1a31c7d32 100644 --- a/building/during-install/tsconfig.json +++ b/building/during-install/tsconfig.json @@ -36,9 +36,6 @@ { "path": "../../deps/path" }, - { - "path": "../../engine/runtime/system-node-version" - }, { "path": "../../exec/lifecycle" }, diff --git a/cspell.json b/cspell.json index 3db4fc4b86..62d955d232 100644 --- a/cspell.json +++ b/cspell.json @@ -55,6 +55,8 @@ "denoland", "denolib", "deptype", + "desugared", + "desugars", "devextreme", "devowl", "dgimuvys", diff --git a/deps/graph-builder/package.json b/deps/graph-builder/package.json index f175da38ef..bb059fafc9 100644 --- a/deps/graph-builder/package.json +++ b/deps/graph-builder/package.json @@ -35,7 +35,6 @@ "@pnpm/core-loggers": "workspace:*", "@pnpm/deps.graph-hasher": "workspace:*", "@pnpm/deps.path": "workspace:*", - "@pnpm/engine.runtime.system-node-version": "workspace:*", "@pnpm/hooks.types": "workspace:*", "@pnpm/installing.modules-yaml": "workspace:*", "@pnpm/lockfile.fs": "workspace:*", diff --git a/deps/graph-builder/src/iteratePkgsForVirtualStore.ts b/deps/graph-builder/src/iteratePkgsForVirtualStore.ts index 5233fe5e87..4a464dce3d 100644 --- a/deps/graph-builder/src/iteratePkgsForVirtualStore.ts +++ b/deps/graph-builder/src/iteratePkgsForVirtualStore.ts @@ -4,6 +4,7 @@ import { calcGraphNodeHash, type DepsGraph, type DepsStateCache, + findRuntimeNodeVersion, type HashedDepPath, iterateHashedGraphNodes, iteratePkgMeta, @@ -11,7 +12,6 @@ import { type PkgMetaAndSnapshot, } from '@pnpm/deps.graph-hasher' import * as dp from '@pnpm/deps.path' -import { findRuntimeNodeVersion } from '@pnpm/engine.runtime.system-node-version' import type { LockfileObject } from '@pnpm/lockfile.fs' import { nameVerFromPkgSnapshot, diff --git a/deps/graph-builder/tsconfig.json b/deps/graph-builder/tsconfig.json index b1089aa3dc..f4d32f77da 100644 --- a/deps/graph-builder/tsconfig.json +++ b/deps/graph-builder/tsconfig.json @@ -24,9 +24,6 @@ { "path": "../../core/types" }, - { - "path": "../../engine/runtime/system-node-version" - }, { "path": "../../hooks/types" }, diff --git a/deps/graph-hasher/src/index.ts b/deps/graph-hasher/src/index.ts index 9e9d12dd29..d962e7f183 100644 --- a/deps/graph-hasher/src/index.ts +++ b/deps/graph-hasher/src/index.ts @@ -7,6 +7,75 @@ import { resolvePlatformSelector, selectPlatformVariant } from '@pnpm/resolving. import type { AllowBuild, DepPath, PkgIdWithPatchHash, SupportedArchitectures } from '@pnpm/types' import { familySync } from 'detect-libc' +/** + * Strip the `node@runtime:` prefix and any peer-context suffix `(...)` + * from a single snapshot key, returning the bare Node version (e.g. + * `"22.11.0"`) — or `undefined` if the key isn't a Node runtime pin. + * + * Peer-suffixed (`node@runtime:22.11.0(node@22.11.0)`) and bare + * (`node@runtime:22.11.0`) forms must reduce to the same answer; the + * pacquet side relies on the same rule for GVS-hash parity. + */ +function extractRuntimeNodeVersion (snapshotKey: string): string | undefined { + const prefix = 'node@runtime:' + if (!snapshotKey.startsWith(prefix)) return undefined + const versionWithPeers = snapshotKey.slice(prefix.length) + const parenAt = versionWithPeers.indexOf('(') + return parenAt === -1 ? versionWithPeers : versionWithPeers.slice(0, parenAt) +} + +/** + * Scan an iterable of lockfile snapshot keys for the resolved + * `engines.runtime` / `devEngines.runtime` Node version and return + * its bare version string (e.g. `"22.11.0"`), or `undefined` when + * no snapshot pins a runtime. + * + * Pnpm's runtime resolver writes the pinned Node into the lockfile as + * a snapshot with key `node@runtime:[()]` + * (see [`engine/runtime/node-resolver/src/index.ts`](https://github.com/pnpm/pnpm/blob/29a42efc3b/engine/runtime/node-resolver/src/index.ts)). + * The first such key found is treated as authoritative. This is fine + * as an install-wide fallback (project-pin in the typical case), but + * snapshots that pin their own Node still need + * {@link readSnapshotRuntimePin} to get a per-snapshot result. + * + * Callers typically pass `Object.keys(lockfile.packages ?? {})` — the + * in-memory `LockfileObject` merges the on-disk `packages:` and + * `snapshots:` sections under a single `packages` field, so its keys + * include every snapshot key the install will hash. + */ +export function findRuntimeNodeVersion (snapshotKeys: Iterable): string | undefined { + for (const key of snapshotKeys) { + const version = extractRuntimeNodeVersion(key) + if (version != null) return version + } + return undefined +} + +/** + * Read a single graph node's own `engines.runtime` Node pin from its + * `children` map. The resolver desugars `engines.runtime` declared on + * a dependency's manifest into `dependencies.node: 'runtime:'` + * (see [`installing/deps-resolver/src/resolveDependencies.ts`](https://github.com/pnpm/pnpm/blob/29a42efc3b/installing/deps-resolver/src/resolveDependencies.ts)), + * which then becomes a `children.node` entry pointing at the + * `node@runtime:[(peers)]` snapshot key. + * + * Returns the bare version (e.g. `"22.11.0"`) when this snapshot pins + * its own Node — or `undefined` when it doesn't and the caller should + * fall back to the install-wide pin / host probe. + * + * Per-snapshot resolution matters because the bin linker routes + * lifecycle-script spawns for a pinning package through *that + * package's* downloaded Node — anchoring the snapshot's GVS engine + * hash to an install-wide value would produce the wrong + * side-effects-cache key for cross-pinning installs. + */ +export function readSnapshotRuntimePin ( + children: Record | undefined +): string | undefined { + const ref = children?.node + return ref != null ? extractRuntimeNodeVersion(ref) : undefined +} + export type DepsGraph = Record> export interface DepsGraphNode { @@ -31,18 +100,20 @@ export function calcDepState ( includeDepGraphHash: boolean supportedArchitectures?: SupportedArchitectures /** - * Resolved `engines.runtime` / `devEngines.runtime` Node version - * for the project being installed (e.g. `"22.11.0"`). When set, - * the side-effects-cache key reflects this script-runner Node - * rather than the Node that pnpm itself is running on — see - * {@link engineName} for the full resolution order. Typically - * computed once per install via {@link findRuntimeNodeVersion} - * over the lockfile's snapshot keys. + * Install-wide fallback `engines.runtime` / `devEngines.runtime` + * Node version (e.g. `"22.11.0"`). Used only when the snapshot at + * `depPath` doesn't itself pin a Node: per-snapshot pins take + * precedence so the side-effects-cache key reflects the actual + * script-runner Node the bin linker would spawn for the package + * (see {@link readSnapshotRuntimePin}). Typically computed once + * per install via {@link findRuntimeNodeVersion} over the + * lockfile's snapshot keys. */ nodeVersion?: string } ): string { - let result = engineName(opts.nodeVersion) + const ownPin = readSnapshotRuntimePin(depsGraph[depPath as T]?.children) + let result = engineName(ownPin ?? opts.nodeVersion) if (opts.includeDepGraphHash) { const depGraphHash = calcDepGraphHash(depsGraph, cache, new Set(), depPath, opts.supportedArchitectures) result += `;deps=${depGraphHash}` @@ -108,13 +179,15 @@ export function * iterateHashedGraphNodes ( allowBuild?: AllowBuild, supportedArchitectures?: SupportedArchitectures, /** - * Resolved `engines.runtime` / `devEngines.runtime` Node version - * for the project being installed. Forwarded as-is into each - * snapshot's [`calcGraphNodeHash`] call so the engine portion of - * the GVS hash reflects the Node that will actually run lifecycle - * scripts — typically obtained via [`findRuntimeNodeVersion`] - * over the lockfile's snapshot keys. `undefined` falls back to - * [`engineName`]'s default (system `node --version`, with + * Install-wide fallback `engines.runtime` / `devEngines.runtime` + * Node version. Used only for snapshots that don't pin their own + * Node; pinning snapshots get resolved per-snapshot via + * {@link readSnapshotRuntimePin} so the GVS engine hash matches + * the Node the bin linker would actually spawn for each package + * (see [`bins/linker/src/index.ts`](https://github.com/pnpm/pnpm/blob/29a42efc3b/bins/linker/src/index.ts)). + * Typically obtained via {@link findRuntimeNodeVersion} over the + * lockfile's snapshot keys. `undefined` falls back to + * {@link engineName}'s default (system `node --version`, with * `process.version` as a last resort). */ nodeVersion?: string @@ -164,7 +237,13 @@ export function calcGraphNodeHash ( // so they survive Node.js upgrades and architecture changes. const includeEngine = builtDepPaths === undefined || transitivelyRequiresBuild(graph, builtDepPaths, buildRequiredCache ??= {}, depPath, new Set()) - const engine = includeEngine ? engineName(nodeVersion) : null + // A snapshot that declares `engines.runtime` carries the desugared + // `node@runtime:` pin as a child; that's the Node the bin + // linker spawns for its lifecycle scripts, so it has to drive the + // engine portion of the hash too. Non-pinning siblings fall through + // to the install-wide value. + const ownPin = readSnapshotRuntimePin(graph[depPath]?.children) + const engine = includeEngine ? engineName(ownPin ?? nodeVersion) : null const deps = calcDepGraphHash(graph, cache, new Set(), depPath, supportedArchitectures) const hexDigest = hashObjectWithoutSorting({ engine, deps }, { encoding: 'hex' }) return formatGlobalVirtualStorePath(name, version, hexDigest) diff --git a/deps/graph-hasher/test/calcGraphNodeHash.test.ts b/deps/graph-hasher/test/calcGraphNodeHash.test.ts index 2553e1c4d9..aefe6d0cd3 100644 --- a/deps/graph-hasher/test/calcGraphNodeHash.test.ts +++ b/deps/graph-hasher/test/calcGraphNodeHash.test.ts @@ -356,6 +356,125 @@ describe('calcGraphNodeHash', () => { expect(result).toMatch(/^@my-org\/my-package\/1\.2\.3\/[a-f0-9]+$/) }) + it('uses the snapshot\'s own engines.runtime pin over an install-wide fallback', () => { + // A dep that declares `engines.runtime: node@22` carries the + // desugared `node@runtime:22.11.0` DepPath as `children.node`. + // That snapshot's GVS hash has to anchor to its *own* pin — + // matching the Node the bin linker spawns for its lifecycle + // scripts (`bins/linker/src/index.ts`'s per-package + // `runtimeHasNodeDownloaded` branch) — instead of the + // install-wide `nodeVersion` fallback that PR #11689 introduced. + const graph: DepsGraph = { + ['pinned@1.0.0' as DepPath]: { + children: { node: 'node@runtime:22.11.0' as DepPath }, + fullPkgId: 'pinned@1.0.0:sha512-pinned', + }, + ['node@runtime:22.11.0' as DepPath]: { + children: {}, + fullPkgId: 'node@runtime:22.11.0:sha512-node22', + }, + } + const pkgMeta: PkgMeta = { + depPath: 'pinned@1.0.0' as DepPath, + name: 'pinned', + version: '1.0.0', + } + + const ownPinHash = calcGraphNodeHash({ graph, cache: {}, nodeVersion: '20.0.0' }, pkgMeta) + + const depsHash = hashObject({ + id: 'pinned@1.0.0:sha512-pinned', + deps: { + node: hashObject({ id: 'node@runtime:22.11.0:sha512-node22', deps: {} }), + }, + }) + const expected = hashObjectWithoutSorting( + { engine: `${process.platform};${process.arch};node22`, deps: depsHash }, + { encoding: 'hex' } + ) + expect(ownPinHash).toBe(`@/pinned/1.0.0/${expected}`) + }) + + it('falls back to the install-wide nodeVersion when the snapshot has no own pin', () => { + // A snapshot whose own children don't include a `node@runtime:` + // entry inherits the project-wide pin instead — mirrors the + // common case where only the root manifest declares + // `engines.runtime` and every transitive dep falls through. + const graph: DepsGraph = { + ['sibling@1.0.0' as DepPath]: { + children: { dep: 'dep@1.0.0' as DepPath }, + fullPkgId: 'sibling@1.0.0:sha512-sibling', + }, + ['dep@1.0.0' as DepPath]: { + children: {}, + fullPkgId: 'dep@1.0.0:sha512-dep', + }, + } + const pkgMeta: PkgMeta = { + depPath: 'sibling@1.0.0' as DepPath, + name: 'sibling', + version: '1.0.0', + } + + const fallbackHash = calcGraphNodeHash({ graph, cache: {}, nodeVersion: '20.5.0' }, pkgMeta) + + const depsHash = hashObject({ + id: 'sibling@1.0.0:sha512-sibling', + deps: { + dep: hashObject({ id: 'dep@1.0.0:sha512-dep', deps: {} }), + }, + }) + const expected = hashObjectWithoutSorting( + { engine: `${process.platform};${process.arch};node20`, deps: depsHash }, + { encoding: 'hex' } + ) + expect(fallbackHash).toBe(`@/sibling/1.0.0/${expected}`) + }) + + it('cross-pinning siblings produce distinct engine prefixes in the same install', () => { + // Two siblings with different `engines.runtime` declarations + // surface the bug this test guards: under PR #11689's + // install-wide resolution they'd share the same engine major in + // the GVS hash (whichever `findRuntimeNodeVersion` happened to + // match first), even though the bin linker would route their + // lifecycle scripts through different downloaded Nodes. + const graph: DepsGraph = { + ['pins-22@1.0.0' as DepPath]: { + children: { node: 'node@runtime:22.11.0' as DepPath }, + fullPkgId: 'pins-22@1.0.0:sha512-a', + }, + ['pins-20@1.0.0' as DepPath]: { + children: { node: 'node@runtime:20.18.0' as DepPath }, + fullPkgId: 'pins-20@1.0.0:sha512-b', + }, + ['node@runtime:22.11.0' as DepPath]: { + children: {}, + fullPkgId: 'node@runtime:22.11.0:sha512-node22', + }, + ['node@runtime:20.18.0' as DepPath]: { + children: {}, + fullPkgId: 'node@runtime:20.18.0:sha512-node20', + }, + } + const cache: DepsStateCache = {} + + const hash22 = calcGraphNodeHash( + { graph, cache, nodeVersion: '22.11.0' }, + { depPath: 'pins-22@1.0.0' as DepPath, name: 'pins-22', version: '1.0.0' } + ) + const hash20 = calcGraphNodeHash( + { graph, cache, nodeVersion: '22.11.0' }, + { depPath: 'pins-20@1.0.0' as DepPath, name: 'pins-20', version: '1.0.0' } + ) + + // The two slots must end up on different paths even though the + // install-wide fallback is the same — the engine portion of the + // hash diverges via each snapshot's own pin. + expect(hash22).not.toBe(hash20) + expect(hash22.startsWith('@/pins-22/1.0.0/')).toBe(true) + expect(hash20.startsWith('@/pins-20/1.0.0/')).toBe(true) + }) + it('should handle prerelease versions', () => { const graph: DepsGraph = { ['pkg@1.0.0-beta.1' as DepPath]: { diff --git a/deps/graph-hasher/test/index.ts b/deps/graph-hasher/test/index.ts index f1f85d4f21..7bd0b72d20 100644 --- a/deps/graph-hasher/test/index.ts +++ b/deps/graph-hasher/test/index.ts @@ -1,6 +1,6 @@ import { describe, expect, test } from '@jest/globals' import { hashObject, hashObjectWithoutSorting } from '@pnpm/crypto.object-hasher' -import { calcDepState, calcGraphNodeHash } from '@pnpm/deps.graph-hasher' +import { calcDepState, calcGraphNodeHash, findRuntimeNodeVersion, readSnapshotRuntimePin } from '@pnpm/deps.graph-hasher' import { engineName } from '@pnpm/engine.runtime.system-node-version' import type { DepPath, PkgIdWithPatchHash } from '@pnpm/types' @@ -56,6 +56,63 @@ test('calcDepState() when scripts are ignored', () => { })).toBe(ENGINE_NAME) }) +test('findRuntimeNodeVersion() pulls the pinned major from a node@runtime: snapshot key', () => { + // Mirrors pacquet's `find_runtime_node_major` helper; both must + // agree on the version-extraction rule or the two tools would + // hash GVS slots under different engine majors for the same + // project. The peer-suffixed form must reduce to the same bare + // version as the form without a peer suffix. + expect( + findRuntimeNodeVersion(['leftpad@1.3.0', 'node@runtime:22.11.0']) + ).toBe('22.11.0') + expect( + findRuntimeNodeVersion(['node@runtime:22.11.0(node@22.11.0)']) + ).toBe('22.11.0') + expect( + findRuntimeNodeVersion(['leftpad@1.3.0', 'is-positive@3.1.0']) + ).toBeUndefined() +}) + +test('readSnapshotRuntimePin() pulls the own pin from a graph node child', () => { + // The resolver desugars a dep's `engines.runtime` into + // `dependencies.node: 'runtime:'` and `refToRelative` + // encodes that into the `node@runtime:[(peers)]` DepPath + // the graph carries as `children.node`. The per-snapshot lookup + // reads back the bare version from there. Without this branch + // the GVS hash for the pinning snapshot would key under the + // install-wide Node, not the Node the bin linker spawns for it. + expect(readSnapshotRuntimePin({ node: 'node@runtime:22.11.0' })).toBe('22.11.0') + expect(readSnapshotRuntimePin({ node: 'node@runtime:22.11.0(node@22.11.0)' })).toBe('22.11.0') + expect(readSnapshotRuntimePin({ node: 'node@22.11.0' })).toBeUndefined() + expect(readSnapshotRuntimePin({ leftpad: 'leftpad@1.3.0' })).toBeUndefined() + expect(readSnapshotRuntimePin({})).toBeUndefined() + expect(readSnapshotRuntimePin(undefined)).toBeUndefined() +}) + +test('calcDepState() uses the snapshot\'s own engines.runtime pin', () => { + // A package whose graph node has `children.node = node@runtime:...` + // pinned its own Node via `engines.runtime`; the side-effects-cache + // key prefix has to encode *that* major (not the install-wide + // fallback) because the bin linker spawns lifecycle scripts on the + // package's pinned Node, not the install-wide one. + const graph = { + 'pinned@1.0.0': { + pkgIdWithPatchHash: 'pinned@1.0.0' as PkgIdWithPatchHash, + resolution: { integrity: '900' }, + children: { node: 'node@runtime:22.11.0' }, + }, + 'node@runtime:22.11.0': { + pkgIdWithPatchHash: 'node@runtime:22.11.0' as PkgIdWithPatchHash, + resolution: { integrity: '901' }, + children: {}, + }, + } + expect(calcDepState(graph, {}, 'pinned@1.0.0', { + includeDepGraphHash: false, + nodeVersion: '20.5.0', // install-wide fallback differs from own pin + })).toBe(`${process.platform};${process.arch};node22`) +}) + describe('calcGraphNodeHash', () => { const graphNodeGraph = { 'foo@1.0.0': { diff --git a/engine/runtime/system-node-version/src/index.ts b/engine/runtime/system-node-version/src/index.ts index d779812747..e1b58bb722 100644 --- a/engine/runtime/system-node-version/src/index.ts +++ b/engine/runtime/system-node-version/src/index.ts @@ -27,8 +27,9 @@ export const getSystemNodeVersion = mem(getSystemNodeVersionNonCached) * * 1. `nodeVersion` argument when provided. Callers use this to thread * a project-pinned runtime (`engines.runtime` / `devEngines.runtime`) - * through to the hash — see {@link findRuntimeNodeVersion} for the - * helper that extracts the value from a lockfile. + * through to the hash — see `findRuntimeNodeVersion` / + * `readSnapshotRuntimePin` in `@pnpm/deps.path` for the helpers + * that extract the value from a lockfile or graph node. * 2. {@link getSystemNodeVersion} — the `node` on the user's `PATH`, * or `process.version` when not SEA-bundled. * 3. `process.version` as a last-resort fallback when the host has @@ -52,35 +53,3 @@ export function engineName (nodeVersion?: string): string { const major = stripped.split('.')[0] return `${process.platform};${process.arch};node${major}` } - -/** - * Scan an iterable of lockfile snapshot keys for the resolved - * `engines.runtime` / `devEngines.runtime` Node version and return - * its bare version string (e.g. `"22.11.0"`), or `undefined` when - * the project doesn't pin a runtime. - * - * Pnpm's runtime resolver writes the pinned Node into the lockfile as - * a snapshot with key `node@runtime:[()]` - * (see [`engine/runtime/node-resolver/src/index.ts`](https://github.com/pnpm/pnpm/blob/29a42efc3b/engine/runtime/node-resolver/src/index.ts)). - * The first such key found is treated as authoritative — workspaces - * with conflicting pins across importers are pathological and the - * resolver rejects them before they reach the lockfile. - * - * Callers typically pass `Object.keys(lockfile.packages ?? {})` — the - * in-memory `LockfileObject` merges the on-disk `packages:` and - * `snapshots:` sections under a single `packages` field, so its keys - * include every snapshot key the install will hash. - */ -export function findRuntimeNodeVersion (snapshotKeys: Iterable): string | undefined { - const prefix = 'node@runtime:' - for (const key of snapshotKeys) { - if (!key.startsWith(prefix)) continue - // Strip peer-context suffix `(...)` — `node@runtime:22.11.0(node@22.11.0)` - // resolves to the same Node version as `node@runtime:22.11.0`, - // so peer-stripped and peer-bearing keys yield the same answer. - const versionWithPeers = key.slice(prefix.length) - const parenAt = versionWithPeers.indexOf('(') - return parenAt === -1 ? versionWithPeers : versionWithPeers.slice(0, parenAt) - } - return undefined -} diff --git a/engine/runtime/system-node-version/test/getSystemNodeVersion.test.ts b/engine/runtime/system-node-version/test/getSystemNodeVersion.test.ts index f6bf8e72ac..5922908f5e 100644 --- a/engine/runtime/system-node-version/test/getSystemNodeVersion.test.ts +++ b/engine/runtime/system-node-version/test/getSystemNodeVersion.test.ts @@ -11,7 +11,7 @@ jest.unstable_mockModule('execa', () => ({ })), })) -const { getSystemNodeVersionNonCached, engineName, findRuntimeNodeVersion } = await import('../lib/index.js') +const { getSystemNodeVersionNonCached, engineName } = await import('../lib/index.js') const execa = await import('execa') test('getSystemNodeVersion() executed from an executable pnpm CLI', () => { @@ -58,20 +58,3 @@ test('engineName() falls back to the host Node when no override is provided', () const major = process.version.replace(/^v/, '').split('.')[0] expect(engineName()).toBe(`${process.platform};${process.arch};node${major}`) }) - -test('findRuntimeNodeVersion() pulls the pinned major from a node@runtime: snapshot key', () => { - // Mirrors pacquet's `find_runtime_node_major` helper; both must - // agree on the version-extraction rule or the two tools would - // hash GVS slots under different engine majors for the same - // project. The peer-suffixed form must reduce to the same bare - // version as the form without a peer suffix. - expect( - findRuntimeNodeVersion(['leftpad@1.3.0', 'node@runtime:22.11.0']) - ).toBe('22.11.0') - expect( - findRuntimeNodeVersion(['node@runtime:22.11.0(node@22.11.0)']) - ).toBe('22.11.0') - expect( - findRuntimeNodeVersion(['leftpad@1.3.0', 'is-positive@3.1.0']) - ).toBeUndefined() -}) diff --git a/exec/commands/test/dlx.e2e.ts b/exec/commands/test/dlx.e2e.ts index 39289a3109..a27f692947 100644 --- a/exec/commands/test/dlx.e2e.ts +++ b/exec/commands/test/dlx.e2e.ts @@ -9,17 +9,15 @@ import { DLX_DEFAULT_OPTS as DEFAULT_OPTS } from './utils/index.js' const { getSystemNodeVersion: originalGetSystemNodeVersion, engineName: originalEngineName, - findRuntimeNodeVersion: originalFindRuntimeNodeVersion, } = await import('@pnpm/engine.runtime.system-node-version') // Re-export every public symbol the package surfaces so downstream // dynamic imports (e.g. `@pnpm/deps.graph-hasher`'s use of // `engineName` for the GVS hash) keep working under the mock. Only // `getSystemNodeVersion` is wrapped with `jest.fn` for spy-ability; -// the other two delegate straight back to the originals. +// `engineName` delegates straight back to the original. jest.unstable_mockModule('@pnpm/engine.runtime.system-node-version', () => ({ getSystemNodeVersion: jest.fn(originalGetSystemNodeVersion), engineName: originalEngineName, - findRuntimeNodeVersion: originalFindRuntimeNodeVersion, })) const installingCommands = await import('@pnpm/installing.commands') const { add: originalAdd } = installingCommands diff --git a/installing/deps-installer/package.json b/installing/deps-installer/package.json index ae26ae5ace..c5d5b20e90 100644 --- a/installing/deps-installer/package.json +++ b/installing/deps-installer/package.json @@ -75,7 +75,6 @@ "@pnpm/deps.graph-hasher": "workspace:*", "@pnpm/deps.graph-sequencer": "workspace:*", "@pnpm/deps.path": "workspace:*", - "@pnpm/engine.runtime.system-node-version": "workspace:*", "@pnpm/error": "workspace:*", "@pnpm/exec.lifecycle": "workspace:*", "@pnpm/fs.read-modules-dir": "workspace:*", diff --git a/installing/deps-installer/src/install/link.ts b/installing/deps-installer/src/install/link.ts index 6d66f8dc0e..e331733b84 100644 --- a/installing/deps-installer/src/install/link.ts +++ b/installing/deps-installer/src/install/link.ts @@ -6,8 +6,7 @@ import { stageLogger, statsLogger, } from '@pnpm/core-loggers' -import { calcDepState, type DepsStateCache } from '@pnpm/deps.graph-hasher' -import { findRuntimeNodeVersion } from '@pnpm/engine.runtime.system-node-version' +import { calcDepState, type DepsStateCache, findRuntimeNodeVersion } from '@pnpm/deps.graph-hasher' import { symlinkDependency } from '@pnpm/fs.symlink-dependency' import type { DependenciesGraph, diff --git a/installing/deps-installer/tsconfig.json b/installing/deps-installer/tsconfig.json index a838b53526..c58ab38dad 100644 --- a/installing/deps-installer/tsconfig.json +++ b/installing/deps-installer/tsconfig.json @@ -90,9 +90,6 @@ { "path": "../../deps/path" }, - { - "path": "../../engine/runtime/system-node-version" - }, { "path": "../../exec/lifecycle" }, diff --git a/installing/deps-resolver/package.json b/installing/deps-resolver/package.json index ba64f1fd4f..e4ae5774b4 100644 --- a/installing/deps-resolver/package.json +++ b/installing/deps-resolver/package.json @@ -40,7 +40,6 @@ "@pnpm/deps.graph-hasher": "workspace:*", "@pnpm/deps.path": "workspace:*", "@pnpm/deps.peer-range": "workspace:*", - "@pnpm/engine.runtime.system-node-version": "workspace:*", "@pnpm/error": "workspace:*", "@pnpm/fetching.pick-fetcher": "workspace:*", "@pnpm/hooks.types": "workspace:*", diff --git a/installing/deps-resolver/src/index.ts b/installing/deps-resolver/src/index.ts index 3558b322ee..d00287a1ba 100644 --- a/installing/deps-resolver/src/index.ts +++ b/installing/deps-resolver/src/index.ts @@ -4,9 +4,8 @@ import type { Catalogs } from '@pnpm/catalogs.types' import { packageManifestLogger, } from '@pnpm/core-loggers' -import { iterateHashedGraphNodes } from '@pnpm/deps.graph-hasher' +import { findRuntimeNodeVersion, iterateHashedGraphNodes } from '@pnpm/deps.graph-hasher' import { isRuntimeDepPath } from '@pnpm/deps.path' -import { findRuntimeNodeVersion } from '@pnpm/engine.runtime.system-node-version' import type { LockfileObject, ProjectSnapshot, diff --git a/installing/deps-resolver/tsconfig.json b/installing/deps-resolver/tsconfig.json index 6330cfcb2e..0d5b77c4b1 100644 --- a/installing/deps-resolver/tsconfig.json +++ b/installing/deps-resolver/tsconfig.json @@ -42,9 +42,6 @@ { "path": "../../deps/peer-range" }, - { - "path": "../../engine/runtime/system-node-version" - }, { "path": "../../fetching/pick-fetcher" }, diff --git a/installing/deps-restorer/package.json b/installing/deps-restorer/package.json index be1d47db77..82f12cb173 100644 --- a/installing/deps-restorer/package.json +++ b/installing/deps-restorer/package.json @@ -48,7 +48,6 @@ "@pnpm/deps.graph-builder": "workspace:*", "@pnpm/deps.graph-hasher": "workspace:*", "@pnpm/deps.path": "workspace:*", - "@pnpm/engine.runtime.system-node-version": "workspace:*", "@pnpm/error": "workspace:*", "@pnpm/exec.lifecycle": "workspace:*", "@pnpm/fs.symlink-dependency": "workspace:*", diff --git a/installing/deps-restorer/src/index.ts b/installing/deps-restorer/src/index.ts index 29a6e4b4dc..f1bad4e844 100644 --- a/installing/deps-restorer/src/index.ts +++ b/installing/deps-restorer/src/index.ts @@ -22,9 +22,8 @@ import { lockfileToDepGraph, type LockfileToDepGraphOptions, } from '@pnpm/deps.graph-builder' -import { calcDepState, type DepsStateCache } from '@pnpm/deps.graph-hasher' +import { calcDepState, type DepsStateCache, findRuntimeNodeVersion } from '@pnpm/deps.graph-hasher' import * as dp from '@pnpm/deps.path' -import { findRuntimeNodeVersion } from '@pnpm/engine.runtime.system-node-version' import { PnpmError } from '@pnpm/error' import { makeNodeRequireOption, diff --git a/installing/deps-restorer/src/linkHoistedModules.ts b/installing/deps-restorer/src/linkHoistedModules.ts index 7cd9913003..169ddb7ed1 100644 --- a/installing/deps-restorer/src/linkHoistedModules.ts +++ b/installing/deps-restorer/src/linkHoistedModules.ts @@ -10,8 +10,7 @@ import type { DependenciesGraph, DepHierarchy, } from '@pnpm/deps.graph-builder' -import { calcDepState, type DepsStateCache } from '@pnpm/deps.graph-hasher' -import { findRuntimeNodeVersion } from '@pnpm/engine.runtime.system-node-version' +import { calcDepState, type DepsStateCache, findRuntimeNodeVersion } from '@pnpm/deps.graph-hasher' import { logger } from '@pnpm/logger' import type { PackageFilesResponse, diff --git a/installing/deps-restorer/tsconfig.json b/installing/deps-restorer/tsconfig.json index 196d57dbcb..b9d2042f22 100644 --- a/installing/deps-restorer/tsconfig.json +++ b/installing/deps-restorer/tsconfig.json @@ -60,9 +60,6 @@ { "path": "../../deps/path" }, - { - "path": "../../engine/runtime/system-node-version" - }, { "path": "../../exec/lifecycle" }, diff --git a/pacquet/crates/package-manager/src/install_frozen_lockfile.rs b/pacquet/crates/package-manager/src/install_frozen_lockfile.rs index dbee982477..2fe7bdf418 100644 --- a/pacquet/crates/package-manager/src/install_frozen_lockfile.rs +++ b/pacquet/crates/package-manager/src/install_frozen_lockfile.rs @@ -1305,3 +1305,102 @@ fn find_runtime_node_major(snapshots: Option<&HashMap } None } + +/// Read one snapshot's own `engines.runtime` Node pin from its +/// `dependencies` map. Mirrors upstream's +/// [`readSnapshotRuntimePin`](https://github.com/pnpm/pnpm/blob/HEAD/engine/runtime/system-node-version/src/index.ts): +/// the resolver desugars `engines.runtime` declared on a dep's +/// manifest into `dependencies.node: 'runtime:'` (see +/// [`installing/deps-resolver/src/resolveDependencies.ts:1477-1479`](https://github.com/pnpm/pnpm/blob/29a42efc3b/installing/deps-resolver/src/resolveDependencies.ts#L1477-L1479)). +/// +/// Returns the bare major when this snapshot pins its own Node, or +/// `None` when it doesn't — callers should then fall back to the +/// install-wide pin / host probe via [`find_runtime_node_major`]. +/// +/// Per-snapshot resolution matters because pnpm's bin linker routes +/// lifecycle-script spawns for a pinning package through that +/// package's own downloaded Node (see +/// [`bins/linker/src/index.ts:229-237`](https://github.com/pnpm/pnpm/blob/29a42efc3b/bins/linker/src/index.ts#L229-L237)). +/// Anchoring the snapshot's GVS engine hash to an install-wide value +/// would produce the wrong side-effects-cache key for cross-pinning +/// installs. +pub(crate) fn find_own_runtime_node_major(snapshot: &SnapshotEntry) -> Option { + let deps = snapshot.dependencies.as_ref()?; + for (alias, dep_ref) in deps { + // Match upstream's per-snapshot extraction rule — only the + // unscoped `node` alias counts, and only when the resolved + // ref-value's prefix is `runtime:` (bun/deno runtimes don't + // contribute to the Node-shaped engine string). + if alias.scope.is_some() || alias.bare != "node" { + continue; + } + let ver_peer = dep_ref.ver_peer(); + if ver_peer.prefix() != Prefix::Runtime { + continue; + } + // Same cast as `find_runtime_node_major` above; see the + // comment there for why `u64 → u32` is lossless in practice. + return Some(ver_peer.version().major as u32); + } + None +} + +#[cfg(test)] +mod tests { + use super::find_own_runtime_node_major; + use pacquet_lockfile::{PkgName, SnapshotDepRef, SnapshotEntry}; + use pretty_assertions::assert_eq; + use std::collections::HashMap; + + /// `dependencies.node: 'runtime:'` is the desugared form pnpm's + /// resolver writes when a dep declares its own `engines.runtime` + /// (see [`installing/deps-resolver/src/resolveDependencies.ts:1477-1479`](https://github.com/pnpm/pnpm/blob/29a42efc3b/installing/deps-resolver/src/resolveDependencies.ts#L1477-L1479)). + /// The helper pulls the bare major back out. + #[test] + fn picks_up_runtime_pin_from_dependencies() { + let mut deps = HashMap::new(); + deps.insert( + PkgName::parse("node").expect("parse pkg name"), + SnapshotDepRef::Plain("runtime:22.11.0".parse().expect("parse ver-peer")), + ); + let snapshot = SnapshotEntry { dependencies: Some(deps), ..SnapshotEntry::default() }; + assert_eq!(find_own_runtime_node_major(&snapshot), Some(22)); + } + + /// A plain semver `node` dep (no `runtime:` prefix) is not an + /// `engines.runtime` pin — workspaces can depend on the `node` + /// npm package without intending it as the script runner. The + /// helper must skip these. + #[test] + fn ignores_non_runtime_node_dep() { + let mut deps = HashMap::new(); + deps.insert( + PkgName::parse("node").expect("parse pkg name"), + SnapshotDepRef::Plain("22.11.0".parse().expect("parse ver-peer")), + ); + let snapshot = SnapshotEntry { dependencies: Some(deps), ..SnapshotEntry::default() }; + assert_eq!(find_own_runtime_node_major(&snapshot), None); + } + + /// Scoped `node` (`@scope/node`) isn't pnpm's runtime alias — + /// only the bare unscoped `node` alias counts. Matches the + /// sibling [`super::find_runtime_node_major`] check. + #[test] + fn ignores_scoped_node_alias() { + let mut deps = HashMap::new(); + deps.insert( + PkgName::parse("@scope/node").expect("parse pkg name"), + SnapshotDepRef::Plain("runtime:22.11.0".parse().expect("parse ver-peer")), + ); + let snapshot = SnapshotEntry { dependencies: Some(deps), ..SnapshotEntry::default() }; + assert_eq!(find_own_runtime_node_major(&snapshot), None); + } + + /// A snapshot with no `dependencies` map yields `None` — the + /// install-wide fallback handles those. + #[test] + fn empty_dependencies_yields_none() { + let snapshot = SnapshotEntry::default(); + assert_eq!(find_own_runtime_node_major(&snapshot), None); + } +} diff --git a/pacquet/crates/package-manager/src/virtual_store_layout.rs b/pacquet/crates/package-manager/src/virtual_store_layout.rs index d83ead50cd..065734286a 100644 --- a/pacquet/crates/package-manager/src/virtual_store_layout.rs +++ b/pacquet/crates/package-manager/src/virtual_store_layout.rs @@ -20,10 +20,11 @@ //! [`PkgNameVerPeer::to_virtual_store_name`]: pacquet_lockfile::PkgNameVerPeer::to_virtual_store_name //! [`pacquet_graph_hasher::format_global_virtual_store_path`]: pacquet_graph_hasher::format_global_virtual_store_path -use crate::AllowBuildPolicy; +use crate::{AllowBuildPolicy, install_frozen_lockfile::find_own_runtime_node_major}; use pacquet_config::Config; use pacquet_graph_hasher::{ - DepsGraphNode, DepsStateCache, calc_graph_node_hash, format_global_virtual_store_path, + DepsGraphNode, DepsStateCache, calc_graph_node_hash, engine_name, + format_global_virtual_store_path, }; use pacquet_lockfile::{ LockfileResolution, PackageKey, PackageMetadata, PkgIdWithPatchHash, PkgName, SnapshotDepRef, @@ -96,10 +97,22 @@ impl VirtualStoreLayout { /// internal `HashMap` doesn't mutate after /// `new`). /// - /// `engine` is the `ENGINE_NAME`-style string that - /// [`pacquet_graph_hasher::engine_name`] produces; threaded in - /// instead of recomputed inside so the value matches whatever the - /// rest of the install (notably the side-effects cache key) uses. + /// `engine` is the install-wide fallback `ENGINE_NAME`-style + /// string that [`pacquet_graph_hasher::engine_name`] produces; + /// threaded in instead of recomputed inside so the value matches + /// whatever the rest of the install (notably the side-effects + /// cache key) uses. Snapshots that themselves pin Node via + /// `engines.runtime` (carried in the lockfile as + /// `dependencies.node: runtime:`) override the fallback + /// per-snapshot through `find_own_runtime_node_major` — the + /// engine portion of the hash then tracks the Node that pnpm's + /// bin linker would spawn for that pinning package's lifecycle + /// scripts (see + /// [`bins/linker/src/index.ts:229-237`](https://github.com/pnpm/pnpm/blob/29a42efc3b/bins/linker/src/index.ts#L229-L237)). + /// Mirrors upstream's + /// [`readSnapshotRuntimePin`](https://github.com/pnpm/pnpm/blob/HEAD/engine/runtime/system-node-version/src/index.ts) + /// branch in `@pnpm/deps.graph-hasher`. + /// /// `None` propagates straight into /// [`calc_graph_node_hash`]'s `engine` parameter — `None` and /// `Some("")` produce *different* GVS hashes (the former omits @@ -175,12 +188,26 @@ impl VirtualStoreLayout { // [`buildRequiredCache`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-hasher/src/index.ts#L113-L114). let mut build_required_cache: HashMap = HashMap::new(); let mut gvs_suffixes: HashMap = HashMap::with_capacity(snapshots.len()); - for snapshot_key in snapshots.keys() { + for (snapshot_key, snapshot) in snapshots { + // Per-snapshot engine resolution: a snapshot that declares + // its own `engines.runtime` carries the desugared + // `dependencies.node: 'runtime:'` pin, which has + // to drive the engine portion of *its* hash rather than + // the install-wide fallback. Match upstream's + // [`readSnapshotRuntimePin`](https://github.com/pnpm/pnpm/blob/HEAD/engine/runtime/system-node-version/src/index.ts) + // precedence: own pin first, install-wide fallback + // second. Default host platform / arch (`None`, `None`) + // matches whatever the caller used to format the + // fallback `engine` so the two strings remain comparable + // across snapshots in one install. + let own_engine = + find_own_runtime_node_major(snapshot).map(|major| engine_name(major, None, None)); + let snapshot_engine = own_engine.as_deref().or(engine); let hex_digest = calc_graph_node_hash( &graph, &mut cache, snapshot_key, - engine, + snapshot_engine, built_dep_paths.as_ref(), &mut build_required_cache, ); @@ -313,9 +340,10 @@ mod tests { use super::VirtualStoreLayout; use pacquet_config::Config; use pacquet_lockfile::{ - LockfileResolution, PackageKey, PackageMetadata, RegistryResolution, SnapshotEntry, + LockfileResolution, PackageKey, PackageMetadata, PkgName, RegistryResolution, + SnapshotDepRef, SnapshotEntry, }; - use pretty_assertions::assert_eq; + use pretty_assertions::{assert_eq, assert_ne}; use std::{collections::HashMap, path::PathBuf}; /// Build a `Config` test-double with the GVS-relevant fields @@ -565,4 +593,113 @@ mod tests { .slot_dir(&key); assert_ne!(darwin, linux, "builder snapshot must partition GVS slot by engine string"); } + + /// Per-snapshot `engines.runtime` resolution: two builder + /// siblings that pin *different* Node majors must land on + /// different GVS slots even when given the same install-wide + /// fallback engine. Mirrors the upstream behaviour in + /// [`@pnpm/deps.graph-hasher`'s `readSnapshotRuntimePin` + /// branch](https://github.com/pnpm/pnpm/blob/HEAD/deps/graph-hasher/src/index.ts). + /// The bin linker spawns each pinning package's lifecycle scripts + /// through its own downloaded Node, so anchoring the engine + /// portion of the hash to a single install-wide value would + /// produce the wrong side-effects-cache key for cross-pinning + /// installs. + #[test] + fn cross_pinning_siblings_get_distinct_slots() { + let config = make_config( + true, + PathBuf::from("/tmp/proj/node_modules/.pnpm"), + PathBuf::from("/tmp/store/links"), + ); + + let pins_22: PackageKey = "pins-22@1.0.0".parse().unwrap(); + let pins_20: PackageKey = "pins-20@1.0.0".parse().unwrap(); + let node22_key: PackageKey = "node@runtime:22.11.0".parse().unwrap(); + let node20_key: PackageKey = "node@runtime:20.18.0".parse().unwrap(); + + let mut packages = HashMap::new(); + let integrities = [ + ( + pins_22.clone(), + "sha512-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", + ), + ( + pins_20.clone(), + "sha512-BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB", + ), + ( + node22_key.clone(), + "sha512-CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC", + ), + ( + node20_key.clone(), + "sha512-DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD", + ), + ]; + for (key, integrity_str) in integrities { + packages.insert( + key, + PackageMetadata { + resolution: LockfileResolution::Registry(RegistryResolution { + integrity: integrity_str.parse().expect("parse integrity"), + }), + engines: None, + cpu: None, + os: None, + libc: None, + deprecated: None, + has_bin: None, + prepare: None, + bundled_dependencies: None, + peer_dependencies: None, + peer_dependencies_meta: None, + }, + ); + } + + // Two builder siblings, each with `dependencies.node: + // runtime:` — the desugared form upstream's resolver + // writes for a manifest-level `engines.runtime` declaration. + let mut pins_22_deps = HashMap::new(); + pins_22_deps.insert( + PkgName::parse("node").expect("parse pkg name"), + SnapshotDepRef::Plain("runtime:22.11.0".parse().expect("parse ver-peer")), + ); + let pins_22_snapshot = + SnapshotEntry { dependencies: Some(pins_22_deps), ..SnapshotEntry::default() }; + + let mut pins_20_deps = HashMap::new(); + pins_20_deps.insert( + PkgName::parse("node").expect("parse pkg name"), + SnapshotDepRef::Plain("runtime:20.18.0".parse().expect("parse ver-peer")), + ); + let pins_20_snapshot = + SnapshotEntry { dependencies: Some(pins_20_deps), ..SnapshotEntry::default() }; + + let mut snapshots = HashMap::new(); + snapshots.insert(pins_22.clone(), pins_22_snapshot); + snapshots.insert(pins_20.clone(), pins_20_snapshot); + snapshots.insert(node22_key.clone(), SnapshotEntry::default()); + snapshots.insert(node20_key.clone(), SnapshotEntry::default()); + + // Both siblings are approved builders so the engine portion + // of the hash isn't dropped by the engine-agnostic gating. + let allowed: std::collections::HashSet = + ["pins-22".to_string(), "pins-20".to_string()].into_iter().collect(); + let policy = crate::AllowBuildPolicy::new(allowed, std::collections::HashSet::new(), false); + + // Same install-wide fallback for both layout queries — the + // divergence has to come from the per-snapshot pin lookup. + let layout = VirtualStoreLayout::new( + &config, + Some("darwin;arm64;node24"), + Some(&snapshots), + Some(&packages), + Some(&policy), + ); + let slot_22 = layout.slot_dir(&pins_22); + let slot_20 = layout.slot_dir(&pins_20); + assert_ne!(slot_22, slot_20, "cross-pinning builders must land on distinct GVS slots"); + } } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 37c0205078..43ccb39b7e 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1748,9 +1748,6 @@ importers: '@pnpm/deps.path': specifier: workspace:* version: link:../../deps/path - '@pnpm/engine.runtime.system-node-version': - specifier: workspace:* - version: link:../../engine/runtime/system-node-version '@pnpm/error': specifier: workspace:* version: link:../../core/error @@ -1963,9 +1960,6 @@ importers: '@pnpm/deps.path': specifier: workspace:* version: link:../../deps/path - '@pnpm/engine.runtime.system-node-version': - specifier: workspace:* - version: link:../../engine/runtime/system-node-version '@pnpm/error': specifier: workspace:* version: link:../../core/error @@ -3262,9 +3256,6 @@ importers: '@pnpm/deps.path': specifier: workspace:* version: link:../path - '@pnpm/engine.runtime.system-node-version': - specifier: workspace:* - version: link:../../engine/runtime/system-node-version '@pnpm/hooks.types': specifier: workspace:* version: link:../../hooks/types @@ -5617,9 +5608,6 @@ importers: '@pnpm/deps.path': specifier: workspace:* version: link:../../deps/path - '@pnpm/engine.runtime.system-node-version': - specifier: workspace:* - version: link:../../engine/runtime/system-node-version '@pnpm/error': specifier: workspace:* version: link:../../core/error @@ -5894,9 +5882,6 @@ importers: '@pnpm/deps.peer-range': specifier: workspace:* version: link:../../deps/peer-range - '@pnpm/engine.runtime.system-node-version': - specifier: workspace:* - version: link:../../engine/runtime/system-node-version '@pnpm/error': specifier: workspace:* version: link:../../core/error @@ -6039,9 +6024,6 @@ importers: '@pnpm/deps.path': specifier: workspace:* version: link:../../deps/path - '@pnpm/engine.runtime.system-node-version': - specifier: workspace:* - version: link:../../engine/runtime/system-node-version '@pnpm/error': specifier: workspace:* version: link:../../core/error From ba2c8844c95f1b5281b441308c1f257009562447 Mon Sep 17 00:00:00 2001 From: shiminshen Date: Sun, 17 May 2026 20:47:59 +0800 Subject: [PATCH 004/169] fix(config): apply pmOnFail default to devEngines.packageManager (singular) (#11682) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(config): apply pmOnFail default to devEngines.packageManager (singular) The pnpm v11 release notes document the `pmOnFail` default as `download` (via the migration table that maps `managePackageManagerVersions: true` → `pmOnFail: download (default)`). The legacy `packageManager` field already gets that default applied at the central onFail-resolution site, but the singular form of `devEngines.packageManager` short-circuited it by setting `onFail = 'error'` inside `parseDevEnginesPackageManager`, so projects that pinned a different pnpm via `devEngines.packageManager` saw a hard version mismatch instead of an auto-download. Drop that local `?? 'error'` and let the central default apply. The array form of `devEngines.packageManager` keeps its own per-element defaults ('error' for the last entry, 'ignore' for the rest) — those reflect explicit prioritisation by the user, not a system-wide fallback. Explicit `onFail` values are still honored everywhere. Closes #11676. * chore: fix spelling (prioritisation → prioritization) cspell flagged the British spelling at pre-push. --------- Co-authored-by: Damon --- .../pmonfail-default-devengines-11676.md | 8 ++++ config/reader/src/index.ts | 16 +++++--- config/reader/test/index.ts | 41 +++++++++++++++++++ pnpm/test/packageManagerCheck.test.ts | 13 +++++- 4 files changed, 71 insertions(+), 7 deletions(-) create mode 100644 .changeset/pmonfail-default-devengines-11676.md diff --git a/.changeset/pmonfail-default-devengines-11676.md b/.changeset/pmonfail-default-devengines-11676.md new file mode 100644 index 0000000000..a25839e917 --- /dev/null +++ b/.changeset/pmonfail-default-devengines-11676.md @@ -0,0 +1,8 @@ +--- +"@pnpm/config.reader": patch +"pnpm": patch +--- + +Fix `devEngines.packageManager` (singular form, without `onFail`) defaulting to `onFail: "error"` instead of the documented `pmOnFail: "download"`. As a result, a project that pinned a different pnpm version via `devEngines.packageManager` and ran `pnpm install` from a mismatched pnpm version failed with a hard error, even though the migration table from `managePackageManagerVersions: true` to `pmOnFail: download (default)` promises the install would auto-download the wanted version [#11676](https://github.com/pnpm/pnpm/issues/11676). + +The array form of `devEngines.packageManager` keeps its existing per-element defaults (`error` for the last entry, `ignore` for the rest), since those reflect explicit prioritization by the user. Explicit `onFail` values continue to win. diff --git a/config/reader/src/index.ts b/config/reader/src/index.ts index 4b8c76572e..b0cc36a4ef 100644 --- a/config/reader/src/index.ts +++ b/config/reader/src/index.ts @@ -657,9 +657,11 @@ export async function getConfig (opts: { // The `pmOnFail` config setting overrides whatever onFail the // wantedPackageManager carried, so users (and internal callers) can force - // a specific behavior without editing the manifest. Otherwise, the legacy - // `packageManager` field defaults to `download` — `devEngines.packageManager` - // already has onFail set during parsing. + // a specific behavior without editing the manifest. Otherwise, both the + // legacy `packageManager` field and singular `devEngines.packageManager` + // fall through to `download` (the documented default for `pmOnFail`); the + // array form of `devEngines.packageManager` already has its own per-element + // defaults applied during parsing. if (pnpmConfig.wantedPackageManager) { if (pnpmConfig.pmOnFail) { pnpmConfig.wantedPackageManager.onFail = pnpmConfig.pmOnFail @@ -764,7 +766,7 @@ export function parsePackageManager (packageManager: string): { name: string, ve function parseDevEnginesPackageManager (devEngines?: DevEngines): EngineDependency | undefined { if (!devEngines?.packageManager) return undefined let pmEngine: EngineDependency | undefined - let onFail: 'ignore' | 'warn' | 'error' | 'download' + let onFail: 'ignore' | 'warn' | 'error' | 'download' | undefined if (Array.isArray(devEngines.packageManager)) { const engines = devEngines.packageManager if (engines.length === 0) return undefined @@ -781,7 +783,11 @@ function parseDevEnginesPackageManager (devEngines?: DevEngines): EngineDependen } } else { pmEngine = devEngines.packageManager - onFail = pmEngine.onFail ?? 'error' + // Singular form: leave onFail undefined when the user did not set it, so + // the central pmOnFail default ('download') applies. The array form keeps + // its own per-element defaults ('error' for the last entry, 'ignore' for + // the rest) because those reflect explicit prioritization by the user. + onFail = pmEngine.onFail } if (!pmEngine?.name) return undefined return { diff --git a/config/reader/test/index.ts b/config/reader/test/index.ts index 66c0b91a64..cee7de949f 100644 --- a/config/reader/test/index.ts +++ b/config/reader/test/index.ts @@ -166,6 +166,47 @@ test('runtimeOnFail=ignore overrides an existing onFail=download and removes nod expect(context.rootProjectManifest?.devDependencies?.node).toBeUndefined() }) +test('devEngines.packageManager without onFail resolves to the documented pmOnFail default "download" (#11676)', async () => { + prepare({ + devEngines: { + packageManager: { + name: 'pnpm', + version: '11.0.0', + }, + }, + }) + + const { context } = await getConfig({ + cliOptions: {}, + packageManager: { name: 'pnpm', version: '11.0.0' }, + }) + + expect(context.wantedPackageManager).toMatchObject({ + name: 'pnpm', + version: '11.0.0', + onFail: 'download', + }) +}) + +test('devEngines.packageManager with explicit onFail is respected (regression guard for #11676)', async () => { + prepare({ + devEngines: { + packageManager: { + name: 'pnpm', + version: '11.0.0', + onFail: 'error', + }, + }, + }) + + const { context } = await getConfig({ + cliOptions: {}, + packageManager: { name: 'pnpm', version: '11.0.0' }, + }) + + expect(context.wantedPackageManager?.onFail).toBe('error') +}) + test('throw error if --link-workspace-packages is used with --global', async () => { await expect(getConfig({ cliOptions: { diff --git a/pnpm/test/packageManagerCheck.test.ts b/pnpm/test/packageManagerCheck.test.ts index ac39c76b19..33eee61dc8 100644 --- a/pnpm/test/packageManagerCheck.test.ts +++ b/pnpm/test/packageManagerCheck.test.ts @@ -143,7 +143,7 @@ test('devEngines.packageManager with onFail=ignore should not check version', as expect(stderr.toString()).not.toContain('0.0.1') }) -test('devEngines.packageManager defaults to onFail=error', async () => { +test('devEngines.packageManager defaults to onFail=download (#11676)', async () => { prepare({ devEngines: { packageManager: { @@ -153,10 +153,19 @@ test('devEngines.packageManager defaults to onFail=error', async () => { }, }) - const { status, stderr } = execPnpmSync(['install']) + // The documented `pmOnFail` default is `download`. Run under COREPACK_ROOT + // to short-circuit the actual version switch (corepack owns version + // selection there), so the test exercises the resolved default without a + // network round-trip. Pre-fix, devEngines.packageManager defaulted to + // `error` and the corepack-specific download-fallthrough hint did NOT + // appear. Asserting on that hint pins the new behavior. + const { status, stderr } = execPnpmSync(['install'], { + env: { COREPACK_ROOT: '/fake/corepack' }, + }) expect(status).toBe(1) expect(stderr.toString()).toContain('This project is configured to use 0.0.1 of pnpm') + expect(stderr.toString()).toContain('does not switch versions when running under corepack') }) test('devEngines.packageManager with a different PM name should fail with onFail=error', async () => { From 963861cac1b6a72ae6306f626a9fc773a18df780 Mon Sep 17 00:00:00 2001 From: Zoltan Kochan Date: Sun, 17 May 2026 15:54:01 +0200 Subject: [PATCH 005/169] perf(npm-resolver): layer abbreviated meta + attestation before full metadata in the minimumReleaseAge gate (#11704) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to #11691 — item 2 from #11687, plus a related shortcut. ## What When the `minimumReleaseAge` lockfile verification gate needs to know when a version was published, it used to fetch a multi-MB full metadata document per package just to read one timestamp. This PR replaces that single-step path with a four-layer lookup that pays the cheapest viable source first: 1. **Abbreviated metadata's `modified` field** — the resolver already fetches this for resolution. If the package as a whole hasn't been modified within the policy cutoff, every version it contains is at least that old; return `modified` as a conservative upper-bound and skip the rest of the chain. 2. **Local `FULL_META_DIR` mirror** — exact per-version times if a previous verification populated it. 3. **npm attestation endpoint** (`/-/npm/v1/attestations/@`) — a tens-of-KB Sigstore bundle whose Rekor inclusion time stands in for publish time. Wins on cold cache when the package was published with provenance. 4. **Full metadata fetch** — last resort. ## Why The verification cache from #11691 made repeat installs against an unchanged lockfile effectively free. The remaining cost is the *first* verification on a fresh CI runner with no restored cache — particularly `pnpm install --frozen-lockfile`, where every locked package's publish timestamp has to be confirmed. Fetching the full metadata document for each package is wasteful when: - The resolver has usually already cached abbreviated metadata, whose `modified` field alone is enough to clear stable packages (the common case). - For recently-modified packages, the per-version attestation endpoint is orders of magnitude smaller than full metadata. ## How ### Abbreviated `modified` shortcut `fetchFullMetadataCached` is refactored to share an internal helper with a new `fetchAbbreviatedMetadataCached`. Both do conditional GETs against their respective on-disk mirrors. On a non-frozen install the abbreviated mirror is already populated by the resolver, so the shortcut hits the local cache at headers-only cost. On `--frozen-lockfile` the fetch is still cheaper than full metadata. If `Date.parse(modified) < cutoff`, return `modified` — it's an upper bound on every version's publish time in this package, and the verifier's `published < cutoff` check passes trivially. ### Attestation endpoint `fetchAttestationPublishedAt` (new module) hits `/-/npm/v1/attestations/@`, parses the response, and reads the earliest `bundle.verificationMaterial.tlogEntries[].integratedTime` across the attestation bundles. That's the Rekor inclusion time — a couple of seconds after publish, well within tolerance for a policy that operates in minutes/hours/days. Returns `undefined` on 404 / network error / malformed body so the caller falls back. ### Per-install dedup The lookup carries a `PublishedAtLookupContext` with four memo maps: abbreviated meta per (registry, name), local full meta per (registry, name), full meta network fetch per (registry, name), final published-at per (registry, name, version). Verifying many versions of the same package only pays the disk/network costs once. ## Trust model **No Sigstore signature verification on the attestation path.** The trust model is identical to reading the registry's `time` field on the full metadata document — we already trust the registry to serve correct publish timestamps for the gate's purpose. The win is purely bandwidth. Full Sigstore verification (Fulcio cert chain + Rekor inclusion proof) would harden the timestamp against a compromised registry. It pulls in the `sigstore` npm package and the TUF root — a separate dependency-surface discussion, parked as future work. ## Tests - **13 unit tests** in `resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts`: ISO timestamp extraction, URL construction (scoped + unscoped), 404 / 5xx / network error / malformed JSON / missing fields → undefined, earliest-of-multiple-attestations, defensive number-as-integratedTime, auth header forwarding, trailing-slash normalization. - Existing `minimumReleaseAge` + `verifyLockfileResolutions` integration suites (45 tests) still pass — the fallback chain preserves end-to-end behavior when the new shortcuts don't apply. --- .../attestation-first-min-release-age.md | 8 + cspell.json | 3 + .../src/createNpmResolutionVerifier.ts | 236 ++++++++++++++++-- .../src/fetchAttestationPublishedAt.ts | 127 ++++++++++ .../src/fetchFullMetadataCached.ts | 42 +++- .../test/fetchAttestationPublishedAt.test.ts | 212 ++++++++++++++++ 6 files changed, 597 insertions(+), 31 deletions(-) create mode 100644 .changeset/attestation-first-min-release-age.md create mode 100644 resolving/npm-resolver/src/fetchAttestationPublishedAt.ts create mode 100644 resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts diff --git a/.changeset/attestation-first-min-release-age.md b/.changeset/attestation-first-min-release-age.md new file mode 100644 index 0000000000..9eb53d37b4 --- /dev/null +++ b/.changeset/attestation-first-min-release-age.md @@ -0,0 +1,8 @@ +--- +"@pnpm/resolving.npm-resolver": minor +"pnpm": patch +--- + +Sped up the `minimumReleaseAge` lockfile verification gate on cold-cache installs by trying npm's `/-/npm/v1/attestations/@` endpoint before fetching the full metadata document. The attestation response is tens of KB versus the multi-MB full metadata, so `--frozen-lockfile` installs against a fleet of provenance-published packages download far less to verify timestamps. + +The publish time comes from `bundle.verificationMaterial.tlogEntries[].integratedTime` (the Rekor inclusion time, a couple of seconds after the actual publish — close enough for a policy that operates in minutes/hours/days). When the local full-metadata mirror already has the timestamp, or the attestation endpoint 404s / errors, the verifier falls back to the existing `fetchFullMetadataCached` path. Sigstore signature verification is not performed; the trust model is unchanged versus reading the registry's `time` field on the full metadata document [#11687](https://github.com/pnpm/pnpm/issues/11687). diff --git a/cspell.json b/cspell.json index 62d955d232..a5102fce0d 100644 --- a/cspell.json +++ b/cspell.json @@ -279,6 +279,7 @@ "rehoist", "reimagining", "reka", + "Rekor", "relinks", "renderable", "replit", @@ -306,6 +307,7 @@ "sigstore", "sindresorhus", "sirv", + "SLSA", "soporan", "sopts", "spdxdocs", @@ -337,6 +339,7 @@ "teambit", "tempy", "testcase", + "tlog", "TLSV", "toctou", "todomvc", diff --git a/resolving/npm-resolver/src/createNpmResolutionVerifier.ts b/resolving/npm-resolver/src/createNpmResolutionVerifier.ts index c105d38da9..0d7f093aa7 100644 --- a/resolving/npm-resolver/src/createNpmResolutionVerifier.ts +++ b/resolving/npm-resolver/src/createNpmResolutionVerifier.ts @@ -1,5 +1,6 @@ import { pickRegistryForPackage } from '@pnpm/config.pick-registry-for-package' import { createPackageVersionPolicy } from '@pnpm/config.version-policy' +import { FULL_META_DIR } from '@pnpm/constants' import { PnpmError } from '@pnpm/error' import type { Resolution, @@ -9,8 +10,14 @@ import type { PackageVersionPolicy, Registries } from '@pnpm/types' import semver from 'semver' import type { FetchMetadataFromFromRegistryOptions } from './fetch.js' -import { fetchFullMetadataCached, type FetchFullMetadataCachedOptions } from './fetchFullMetadataCached.js' +import { fetchAttestationPublishedAt } from './fetchAttestationPublishedAt.js' +import { + fetchAbbreviatedMetadataCached, + fetchFullMetadataCached, + type FetchFullMetadataCachedOptions, +} from './fetchFullMetadataCached.js' import { BUILTIN_NAMED_REGISTRIES } from './parseBareSpecifier.js' +import { getPkgMirrorPath, loadMeta } from './pickPackage.js' export interface CreateNpmResolutionVerifierOptions { /** @@ -91,21 +98,19 @@ export function createNpmResolutionVerifier ( .filter((value): value is string => value != null) .sort((a, b) => b.length - a.length) - // In-memory dedup of the time map per (registry, name) for this verifier - // instance. The on-disk conditional-GET cache is handled inside - // fetchFullMetadataCached via the resolver's shared mirror at opts.cacheDir. - const inflight = new Map | undefined>>() - const fetchTimeMap = async (registry: string, name: string): Promise | undefined> => { - const cacheKey = `${registry}\x00${name}` - const cached = inflight.get(cacheKey) - if (cached) return cached - const promise = fetchFullMetadataCached(opts.fetchOpts, name, { - registry, - authHeaderValue: opts.getAuthHeaderValueByURI(registry), - cacheDir: opts.cacheDir, - }).then((meta) => meta.time) - inflight.set(cacheKey, promise) - return promise + // Per-install dedup of every network/disk fetch the verifier issues + // (see fetchPublishedAt for the lookup order). The on-disk + // conditional-GET cache is handled inside fetch{Abbreviated,Full}MetadataCached + // via the resolver's shared mirrors at opts.cacheDir. + const lookupContext: PublishedAtLookupContext = { + fetchOpts: opts.fetchOpts, + getAuthHeaderValueByURI: opts.getAuthHeaderValueByURI, + cacheDir: opts.cacheDir, + cutoffMs: cutoff, + abbreviatedMetaCache: new Map(), + publishedAtCache: new Map(), + localMetaCache: new Map(), + fullMetaCache: new Map(), } const minimumReleaseAge = opts.minimumReleaseAge @@ -119,9 +124,9 @@ export function createNpmResolutionVerifier ( const tarballUrl = (resolution as { tarball?: string }).tarball const registry = pickRegistryForVersion(opts.registries, namedRegistryPrefixes, name, tarballUrl) - let time: Record | undefined + let published: string | undefined try { - time = await fetchTimeMap(registry, name) + published = await fetchPublishedAt(lookupContext, registry, name, version) } catch (err) { return { ok: false, @@ -129,12 +134,11 @@ export function createNpmResolutionVerifier ( reason: uncheckable(err instanceof Error ? err.message : String(err)), } } - const published = time?.[version] if (!published) { - // Full metadata is missing this version — either an unpublish or the - // registry doesn't expose per-version timestamps for it. Either way - // the release-age can't be verified, so report a violation rather - // than silently passing. + // No source — attestation, local mirror, or full metadata — + // surfaced a publish timestamp for this version. Either it's + // unpublished or the registry doesn't expose per-version + // timestamps. Report a violation rather than silently passing. return { ok: false, code: 'MINIMUM_RELEASE_AGE_VIOLATION', @@ -175,6 +179,192 @@ export function createNpmResolutionVerifier ( } } +type PublishedAtTimeMap = Record + +interface PublishedAtLookupContext { + fetchOpts: FetchMetadataFromFromRegistryOptions + getAuthHeaderValueByURI: (registry: string) => string | undefined + cacheDir?: string + /** + * The `minimumReleaseAge` cutoff converted to a unix-ms epoch. A + * version with a publish time strictly less than this passes the + * policy. Used by the abbreviated-metadata shortcut: if the + * package's last-modified time is older than the cutoff, every + * version it contains is too. + */ + cutoffMs: number + /** + * Per-(registry+name) memo of the abbreviated metadata fetch. + * Abbreviated is what the resolver populates by default, so on a + * non-frozen install the conditional GET hits the disk mirror at + * ~zero cost. Resolves to the parsed metadata or `undefined` on + * failure. + */ + abbreviatedMetaCache: Map> + /** + * Per-(registry+name+version) memo of the final published-at answer + * the verifier hands to the policy check. One install verifies each + * (name, version) pair at most once. + */ + publishedAtCache: Map> + /** + * Per-(registry+name) memo of the on-disk full-metadata mirror read. + * One disk read per package regardless of how many versions we + * verify of it. + */ + localMetaCache: Map> + /** + * Per-(registry+name) memo of the full-metadata network fetch — only + * issued when both the abbreviated-modified shortcut and the + * attestation endpoint fail to yield a timestamp. + */ + fullMetaCache: Map> +} + +/** + * Per-(registry, name, version) lookup with a layered fallback: + * + * 1. **Abbreviated metadata `modified` shortcut.** This is what the + * resolver already fetches by default; it's a small document with + * a package-level last-modified time but no per-version timestamps. + * If `modified` is older than the policy cutoff, every version in + * this package was published at least that long ago — return the + * `modified` timestamp as a conservative upper bound and skip the + * rest of the chain. Costs one conditional GET that the resolver + * has usually already paid for. + * 2. **On-disk full-metadata mirror.** If a previous verification + * populated `FULL_META_DIR`, take the per-version timestamp from + * there. + * 3. **npm attestation endpoint.** Small payload, just this version's + * Sigstore-anchored timestamp. Wins on cold cache when the package + * was published with provenance. + * 4. **Full metadata fetch.** Last resort — only paid when the + * abbreviated shortcut can't decide, the local full mirror is + * cold, and there's no attestation. + */ +async function fetchPublishedAt ( + context: PublishedAtLookupContext, + registry: string, + name: string, + version: string +): Promise { + const cacheKey = `${registry}\x00${name}\x00${version}` + let cachedPromise = context.publishedAtCache.get(cacheKey) + if (cachedPromise == null) { + cachedPromise = resolvePublishedAt(context, registry, name, version) + context.publishedAtCache.set(cacheKey, cachedPromise) + } + return cachedPromise +} + +async function resolvePublishedAt ( + context: PublishedAtLookupContext, + registry: string, + name: string, + version: string +): Promise { + const abbreviatedShortcut = await tryAbbreviatedModifiedShortcut(context, registry, name) + if (abbreviatedShortcut != null) return abbreviatedShortcut + + const localTime = await readLocalMetaTime(context, registry, name) + if (localTime?.[version]) return localTime[version] + + const attestationTime = await fetchAttestationPublishedAt(context.fetchOpts, name, version, { + registry, + authHeaderValue: context.getAuthHeaderValueByURI(registry), + }) + if (attestationTime != null) return attestationTime + + const fullMetaTime = await fetchFullMetaTime(context, registry, name) + return fullMetaTime?.[version] +} + +/** + * Returns the abbreviated metadata's `modified` timestamp **iff** it + * proves the gate would pass — i.e. modified is strictly older than + * the policy cutoff. In that case every version this package contains + * predates the cutoff, so the caller can short-circuit with `modified` + * as a conservative upper-bound publish time. + * + * Returns `undefined` otherwise (modified is too recent, the metadata + * lacks a parseable modified field, or the fetch failed) and the + * caller proceeds with per-version lookups. + */ +async function tryAbbreviatedModifiedShortcut ( + context: PublishedAtLookupContext, + registry: string, + name: string +): Promise { + const meta = await fetchAbbreviatedMeta(context, registry, name) + const modified = meta?.modified + if (typeof modified !== 'string') return undefined + const modifiedMs = Date.parse(modified) + if (Number.isNaN(modifiedMs)) return undefined + if (modifiedMs >= context.cutoffMs) return undefined + return modified +} + +function fetchAbbreviatedMeta ( + context: PublishedAtLookupContext, + registry: string, + name: string +): Promise<{ modified?: string } | undefined> { + const cacheKey = `${registry}\x00${name}` + let cachedPromise = context.abbreviatedMetaCache.get(cacheKey) + if (cachedPromise == null) { + cachedPromise = fetchAbbreviatedMetadataCached(context.fetchOpts, name, { + registry, + authHeaderValue: context.getAuthHeaderValueByURI(registry), + cacheDir: context.cacheDir, + }).catch(() => undefined) + context.abbreviatedMetaCache.set(cacheKey, cachedPromise) + } + return cachedPromise +} + +function readLocalMetaTime ( + context: PublishedAtLookupContext, + registry: string, + name: string +): Promise { + if (!context.cacheDir) return Promise.resolve(undefined) + const cacheKey = `${registry}\x00${name}` + let cachedPromise = context.localMetaCache.get(cacheKey) + if (cachedPromise == null) { + cachedPromise = loadLocalMetaTime(context.cacheDir, registry, name) + context.localMetaCache.set(cacheKey, cachedPromise) + } + return cachedPromise +} + +async function loadLocalMetaTime ( + cacheDir: string, + registry: string, + name: string +): Promise { + const pkgMirror = getPkgMirrorPath(cacheDir, FULL_META_DIR, registry, name) + const cached = await loadMeta(pkgMirror) + return cached?.time +} + +function fetchFullMetaTime ( + context: PublishedAtLookupContext, + registry: string, + name: string +): Promise { + const cacheKey = `${registry}\x00${name}` + let cachedPromise = context.fullMetaCache.get(cacheKey) + if (cachedPromise == null) { + cachedPromise = fetchFullMetadataCached(context.fetchOpts, name, { + registry, + authHeaderValue: context.getAuthHeaderValueByURI(registry), + cacheDir: context.cacheDir, + }).then((meta) => meta.time) + context.fullMetaCache.set(cacheKey, cachedPromise) + } + return cachedPromise +} + function pickRegistryForVersion ( registries: Registries, namedRegistryPrefixes: string[], diff --git a/resolving/npm-resolver/src/fetchAttestationPublishedAt.ts b/resolving/npm-resolver/src/fetchAttestationPublishedAt.ts new file mode 100644 index 0000000000..b12c223603 --- /dev/null +++ b/resolving/npm-resolver/src/fetchAttestationPublishedAt.ts @@ -0,0 +1,127 @@ +import * as retry from '@zkochan/retry' + +import type { FetchMetadataFromFromRegistryOptions } from './fetch.js' + +/** + * Per-version publish timestamp from npm's attestation endpoint — + * `/-/npm/v1/attestations/@`. + * + * The response is a small JSON document containing one or more Sigstore + * bundles. We read `bundle.verificationMaterial.tlogEntries[].integratedTime` + * (the Rekor inclusion time) and surface it as an ISO date. This is a + * couple of seconds after the actual publish — close enough for a + * release-age policy that operates in minutes/hours/days. + * + * We deliberately do **not** verify the Sigstore signature here: the + * trust model is identical to reading the registry's `time` field on + * the full metadata document. The win is bandwidth — the attestation + * payload is tens of KB versus the multi-MB full metadata document, so + * cold-cache + `--frozen-lockfile` installs against a fleet of + * provenance-published packages pay far less to verify timestamps. + * + * Returns `undefined` when: + * + * - The package has no published attestations (`404`). + * - The response is malformed or missing the timestamp. + * - The request itself fails (network error, registry 5xx). + * + * In all of those cases the caller falls back to fetching full metadata. + */ +export interface FetchAttestationOptions { + registry: string + authHeaderValue?: string +} + +export async function fetchAttestationPublishedAt ( + fetchOpts: FetchMetadataFromFromRegistryOptions, + pkgName: string, + version: string, + opts: FetchAttestationOptions +): Promise { + const url = `${opts.registry.replace(/\/$/, '')}/-/npm/v1/attestations/${pkgName}@${version}` + const retryOperation = retry.operation(fetchOpts.retry) + return new Promise((resolve) => { + retryOperation.attempt(async () => { + let response: Response + try { + response = await fetchOpts.fetch(url, { + authHeaderValue: opts.authHeaderValue, + retry: fetchOpts.retry, + timeout: fetchOpts.timeout, + }) + } catch { + // Network errors fall through to the full-metadata path; the + // caller's `fetchFullMetadataCached` has its own retry policy. + resolve(undefined) + return + } + // 404 = package never published attestations. Other 4xx/5xx also + // mean "can't get an answer from this endpoint, fall back." + if (response.status >= 400) { + resolve(undefined) + return + } + let body: unknown + try { + body = await response.json() + } catch { + resolve(undefined) + return + } + resolve(extractPublishedAt(body)) + }) + }) +} + +/** + * Pull the earliest `integratedTime` across every attestation bundle in + * the response and convert it to an ISO timestamp. Earliest is the + * conservative choice: if two attestations disagree (e.g. publish + * v0.1 vs SLSA provenance v1), we attribute the publish to the older + * Rekor entry. The Rekor timestamp is what tells us when the artifact + * existed in a transparency log — that's the floor on publish time. + */ +function extractPublishedAt (body: unknown): string | undefined { + if (!body || typeof body !== 'object') return undefined + const attestations = (body as { attestations?: unknown }).attestations + if (!Array.isArray(attestations)) return undefined + + let earliestSeconds: number | undefined + for (const attestation of attestations) { + const seconds = readEarliestIntegratedTime(attestation) + if (seconds == null) continue + if (earliestSeconds == null || seconds < earliestSeconds) { + earliestSeconds = seconds + } + } + if (earliestSeconds == null) return undefined + return new Date(earliestSeconds * 1000).toISOString() +} + +function readEarliestIntegratedTime (attestation: unknown): number | undefined { + if (!attestation || typeof attestation !== 'object') return undefined + const bundle = (attestation as { bundle?: unknown }).bundle + if (!bundle || typeof bundle !== 'object') return undefined + const verificationMaterial = (bundle as { verificationMaterial?: unknown }).verificationMaterial + if (!verificationMaterial || typeof verificationMaterial !== 'object') return undefined + const tlogEntries = (verificationMaterial as { tlogEntries?: unknown }).tlogEntries + if (!Array.isArray(tlogEntries)) return undefined + + let earliest: number | undefined + for (const entry of tlogEntries) { + if (!entry || typeof entry !== 'object') continue + const rawIntegratedTime = (entry as { integratedTime?: unknown }).integratedTime + // npm serializes integratedTime as a string ("1778583836") to avoid + // JSON precision loss; accept either string or number defensively. + const seconds = parseIntegratedTimeSeconds(rawIntegratedTime) + if (seconds == null) continue + if (earliest == null || seconds < earliest) earliest = seconds + } + return earliest +} + +function parseIntegratedTimeSeconds (raw: unknown): number | undefined { + const seconds = typeof raw === 'string' ? Number(raw) : typeof raw === 'number' ? raw : NaN + if (!Number.isFinite(seconds) || seconds <= 0) return undefined + return seconds +} diff --git a/resolving/npm-resolver/src/fetchFullMetadataCached.ts b/resolving/npm-resolver/src/fetchFullMetadataCached.ts index d1d51400d9..86082da87b 100644 --- a/resolving/npm-resolver/src/fetchFullMetadataCached.ts +++ b/resolving/npm-resolver/src/fetchFullMetadataCached.ts @@ -1,23 +1,24 @@ -import { FULL_META_DIR } from '@pnpm/constants' +import { ABBREVIATED_META_DIR, FULL_META_DIR } from '@pnpm/constants' import { PnpmError } from '@pnpm/error' import type { PackageMeta } from '@pnpm/resolving.registry.types' import { fetchMetadataFromFromRegistry, type FetchMetadataFromFromRegistryOptions } from './fetch.js' import { getPkgMirrorPath, loadMeta, loadMetaHeaders, prepareJsonForDisk, saveMeta } from './pickPackage.js' -export interface FetchFullMetadataCachedOptions { +export interface FetchMetadataCachedOptions { registry: string authHeaderValue?: string /** * pnpm's on-disk cache directory. When set, the call issues a conditional - * GET against the same `FULL_META_DIR` mirror the resolver populates: a - * 304 Not Modified response serves the body from disk, a 200 writes the - * new body back. Omit to disable caching — every call re-fetches the - * full manifest. + * GET against the matching mirror the resolver populates: a 304 Not + * Modified response serves the body from disk, a 200 writes the new body + * back. Omit to disable caching — every call re-fetches. */ cacheDir?: string } +export type FetchFullMetadataCachedOptions = FetchMetadataCachedOptions + /** * Fetch a full registry metadata document for `pkgName`, reusing pnpm's * shared on-disk metadata mirror when `cacheDir` is supplied. Built for the @@ -30,15 +31,40 @@ export async function fetchFullMetadataCached ( fetchOpts: FetchMetadataFromFromRegistryOptions, pkgName: string, opts: FetchFullMetadataCachedOptions +): Promise { + return fetchMetadataCached(fetchOpts, pkgName, { ...opts, fullMetadata: true, metaDir: FULL_META_DIR }) +} + +/** + * Sibling of {@link fetchFullMetadataCached} that hits the abbreviated + * metadata endpoint (`Accept: application/vnd.npm.install-v1+json`) and + * caches under `ABBREVIATED_META_DIR` — the same mirror the resolver + * populates by default. Used by the lockfile verification gate as a + * cheap upper-bound check: if the package's `modified` field is older + * than the policy cutoff, every version in it predates the cutoff and + * no per-version timestamp lookup is needed. + */ +export async function fetchAbbreviatedMetadataCached ( + fetchOpts: FetchMetadataFromFromRegistryOptions, + pkgName: string, + opts: FetchMetadataCachedOptions +): Promise { + return fetchMetadataCached(fetchOpts, pkgName, { ...opts, fullMetadata: false, metaDir: ABBREVIATED_META_DIR }) +} + +async function fetchMetadataCached ( + fetchOpts: FetchMetadataFromFromRegistryOptions, + pkgName: string, + opts: FetchMetadataCachedOptions & { fullMetadata: boolean, metaDir: string } ): Promise { const pkgMirror = opts.cacheDir != null - ? getPkgMirrorPath(opts.cacheDir, FULL_META_DIR, opts.registry, pkgName) + ? getPkgMirrorPath(opts.cacheDir, opts.metaDir, opts.registry, pkgName) : null const cacheHeaders = pkgMirror != null ? await loadMetaHeaders(pkgMirror) : null const result = await fetchMetadataFromFromRegistry(fetchOpts, pkgName, { registry: opts.registry, authHeaderValue: opts.authHeaderValue, - fullMetadata: true, + fullMetadata: opts.fullMetadata, etag: cacheHeaders?.etag, modified: cacheHeaders?.modified, }) diff --git a/resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts b/resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts new file mode 100644 index 0000000000..ad96099663 --- /dev/null +++ b/resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts @@ -0,0 +1,212 @@ +import { describe, expect, test } from '@jest/globals' + +import { fetchAttestationPublishedAt } from '../src/fetchAttestationPublishedAt.js' + +type StubFetch = (url: string, opts?: unknown) => Promise<{ + status: number + json: () => Promise +}> + +function makeFetchOpts (fetch: StubFetch) { + return { + fetch: fetch as never, + retry: { retries: 0 }, + timeout: 10_000, + fetchWarnTimeoutMs: 10_000, + } +} + +const REGISTRY = 'https://registry.npmjs.org/' + +function attestationResponse (...integratedTimes: Array): unknown { + return { + attestations: integratedTimes.map((t) => ({ + predicateType: 'https://github.com/npm/attestation/tree/main/specs/publish/v0.1', + bundle: { + verificationMaterial: { + tlogEntries: [{ integratedTime: t }], + }, + }, + })), + } +} + +describe('fetchAttestationPublishedAt', () => { + test('returns an ISO timestamp built from tlogEntries[].integratedTime', async () => { + // 1778583836 (Unix seconds) → 2026-05-12T05:43:56.000Z + const fetch: StubFetch = async () => ({ + status: 200, + json: async () => attestationResponse('1778583836'), + }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBe(new Date(1778583836 * 1000).toISOString()) + }) + + test('hits /-/npm/v1/attestations/@ with a literal name@version spec', async () => { + const seenUrls: string[] = [] + const fetch: StubFetch = async (url) => { + seenUrls.push(url) + return { status: 404, json: async () => null } + } + await fetchAttestationPublishedAt(makeFetchOpts(fetch), 'pnpm', '11.1.1', { registry: REGISTRY }) + expect(seenUrls).toEqual(['https://registry.npmjs.org/-/npm/v1/attestations/pnpm@11.1.1']) + }) + + test('scoped package name passes through unencoded (slashes are path separators)', async () => { + const seenUrls: string[] = [] + const fetch: StubFetch = async (url) => { + seenUrls.push(url) + return { status: 404, json: async () => null } + } + await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + '@pnpm/exe', + '11.1.1', + { registry: REGISTRY } + ) + expect(seenUrls).toEqual(['https://registry.npmjs.org/-/npm/v1/attestations/@pnpm/exe@11.1.1']) + }) + + test('returns undefined when the registry has no attestations for the package (404)', async () => { + const fetch: StubFetch = async () => ({ status: 404, json: async () => null }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBeUndefined() + }) + + test('returns undefined on 5xx — caller falls back to full metadata', async () => { + const fetch: StubFetch = async () => ({ status: 503, json: async () => null }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBeUndefined() + }) + + test('returns undefined when the fetch itself throws (network error)', async () => { + const fetch: StubFetch = async () => { + throw new Error('ECONNREFUSED') + } + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBeUndefined() + }) + + test('returns undefined when the body is malformed JSON', async () => { + const fetch: StubFetch = async () => ({ + status: 200, + json: async () => { + throw new SyntaxError('bad') + }, + }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBeUndefined() + }) + + test('returns undefined when the response has no attestations array', async () => { + const fetch: StubFetch = async () => ({ status: 200, json: async () => ({}) }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBeUndefined() + }) + + test('returns undefined when no tlogEntry carries a usable integratedTime', async () => { + const fetch: StubFetch = async () => ({ + status: 200, + json: async () => ({ + attestations: [{ bundle: { verificationMaterial: { tlogEntries: [{}] } } }], + }), + }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBeUndefined() + }) + + test('picks the earliest integratedTime across multiple attestations', async () => { + // SLSA provenance is signed slightly before npm publish attestation; + // earlier integratedTime is the conservative pick. + const fetch: StubFetch = async () => ({ + status: 200, + json: async () => attestationResponse('1778583836', '1778583833'), + }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBe(new Date(1778583833 * 1000).toISOString()) + }) + + test('accepts integratedTime as a number too (defensive against schema drift)', async () => { + const fetch: StubFetch = async () => ({ + status: 200, + json: async () => attestationResponse(1778583836), + }) + const result = await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY } + ) + expect(result).toBe(new Date(1778583836 * 1000).toISOString()) + }) + + test('forwards the auth header to the fetch call', async () => { + let seenAuth: string | undefined + const fetch: StubFetch = async (_url, opts) => { + seenAuth = (opts as { authHeaderValue?: string } | undefined)?.authHeaderValue + return { status: 404, json: async () => null } + } + await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: REGISTRY, authHeaderValue: 'Bearer secret' } + ) + expect(seenAuth).toBe('Bearer secret') + }) + + test('strips a trailing slash on the registry URL', async () => { + const seenUrls: string[] = [] + const fetch: StubFetch = async (url) => { + seenUrls.push(url) + return { status: 404, json: async () => null } + } + await fetchAttestationPublishedAt( + makeFetchOpts(fetch), + 'pnpm', + '11.1.1', + { registry: 'https://registry.npmjs.org/' } + ) + expect(seenUrls[0]).toBe('https://registry.npmjs.org/-/npm/v1/attestations/pnpm@11.1.1') + }) +}) From 06d2d3deb22d29de8f5ff182f011b4e04ebebc08 Mon Sep 17 00:00:00 2001 From: shiminshen Date: Sun, 17 May 2026 22:35:59 +0800 Subject: [PATCH 006/169] fix: write packageManagerDependencies to lockfile when devEngines.packageManager is set (#11681) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When `devEngines.packageManager.pnpm` is set without `onFail: "download"`, `pnpm install` ran `syncEnvLockfile` instead of `switchCliVersion`. That sync returned early whenever the env lockfile did not already record a `packageManagerDependencies.pnpm` entry, so the resolved pnpm version was never recorded on first install — contradicting the documented behavior ("The resolved version is stored in pnpm-lock.yaml") and forcing users to add `onFail: "download"` purely to trigger the lockfile write. Drop the two early-returns that only fired when the env lockfile was missing or empty. The resolution proceeds whenever (a) the project pins a pnpm version via `devEngines.packageManager` (or a v12+ `packageManager` field) and (b) the running pnpm satisfies that pin. The existing "already-resolved" no-op path still skips work when the lockfile already records a satisfying version, so steady-state installs don't churn. Closes #11674 (part 1). Part 3 (pruning `@pnpm/exe` platform entries when `onFail: "download"` is removed) is left for a follow-up — it needs a state-transition signal the codebase doesn't yet track. Co-authored-by: Damon --- .../sync-env-lockfile-when-missing-11674.md | 5 ++++ pnpm/src/syncEnvLockfile.test.ts | 19 +++++++++++--- pnpm/src/syncEnvLockfile.ts | 25 ++++++++++--------- 3 files changed, 33 insertions(+), 16 deletions(-) create mode 100644 .changeset/sync-env-lockfile-when-missing-11674.md diff --git a/.changeset/sync-env-lockfile-when-missing-11674.md b/.changeset/sync-env-lockfile-when-missing-11674.md new file mode 100644 index 0000000000..0550a6f3c3 --- /dev/null +++ b/.changeset/sync-env-lockfile-when-missing-11674.md @@ -0,0 +1,5 @@ +--- +"pnpm": patch +--- + +Fix `devEngines.packageManager` not writing `packageManagerDependencies` to `pnpm-lock.yaml` when the lockfile lacks an env-doc entry. Previously the lockfile sync skipped resolution unless an existing `packageManagerDependencies.pnpm` entry needed refreshing, so a fresh install without `onFail: "download"` left the resolved pnpm version unrecorded — contradicting the documented behavior that the resolved version is stored in `pnpm-lock.yaml` [#11674](https://github.com/pnpm/pnpm/issues/11674). diff --git a/pnpm/src/syncEnvLockfile.test.ts b/pnpm/src/syncEnvLockfile.test.ts index 40eaeb86f4..e2c8705f27 100644 --- a/pnpm/src/syncEnvLockfile.test.ts +++ b/pnpm/src/syncEnvLockfile.test.ts @@ -84,21 +84,32 @@ test('no-op when running pnpm does not satisfy wanted range', async () => { expect(resolvePackageManagerIntegrities).not.toHaveBeenCalled() }) -test('no-op when no env lockfile exists', async () => { +test('writes packageManagerDependencies when no env lockfile exists yet (#11674)', async () => { const dir = tempDir() await syncEnvLockfile(baseConfig, makeContext(dir, { wantedPackageManager: { name: 'pnpm', version: packageManager.version, fromDevEngines: true }, })) - expect(resolvePackageManagerIntegrities).not.toHaveBeenCalled() + expect(resolvePackageManagerIntegrities).toHaveBeenCalledTimes(1) + const updated = await readEnvLockfile(dir) + expect(updated).not.toBeNull() + expect(updated!.importers['.'].packageManagerDependencies?.['pnpm']).toEqual({ + specifier: packageManager.version, + version: packageManager.version, + }) }) -test('no-op when lockfile has no packageManagerDependencies for pnpm', async () => { +test('writes packageManagerDependencies when env lockfile exists but lacks pnpm entry (#11674)', async () => { const dir = tempDir() writeEnvLockfileWithoutPmDeps(dir) await syncEnvLockfile(baseConfig, makeContext(dir, { wantedPackageManager: { name: 'pnpm', version: packageManager.version, fromDevEngines: true }, })) - expect(resolvePackageManagerIntegrities).not.toHaveBeenCalled() + expect(resolvePackageManagerIntegrities).toHaveBeenCalledTimes(1) + const updated = await readEnvLockfile(dir) + expect(updated!.importers['.'].packageManagerDependencies?.['pnpm']).toEqual({ + specifier: packageManager.version, + version: packageManager.version, + }) }) test('no-op when lockfile already records a satisfying version', async () => { diff --git a/pnpm/src/syncEnvLockfile.ts b/pnpm/src/syncEnvLockfile.ts index 9373f9684d..f347be6e15 100644 --- a/pnpm/src/syncEnvLockfile.ts +++ b/pnpm/src/syncEnvLockfile.ts @@ -8,14 +8,17 @@ import semver from 'semver' import { shouldPersistLockfile } from './shouldPersistLockfile.js' /** - * Refreshes the env lockfile's `packageManagerDependencies` entry when it - * records a pnpm version that no longer satisfies the wanted - * `devEngines.packageManager` range. The currently running pnpm version - * (already verified to satisfy the wanted range by checkPackageManager) is - * recorded as the new resolution. + * Records the currently running pnpm version in the env lockfile's + * `packageManagerDependencies` entry when the project opts in to + * lockfile-pinned versioning (via `devEngines.packageManager`, or a v12+ + * `packageManager` pin) and the lockfile doesn't already record a version + * that satisfies the wanted range. * - * No-op when the project does not pin a pnpm version, when no env lockfile - * exists yet, or when the recorded version still satisfies the wanted range. + * The currently running pnpm version has already been verified by + * checkPackageManager to satisfy the wanted range, so recording it is safe. + * + * No-op when the project does not pin a pnpm version or when the recorded + * version still satisfies the wanted range. */ export async function syncEnvLockfile (config: Config, context: ConfigContext): Promise { const pm = context.wantedPackageManager @@ -27,15 +30,13 @@ export async function syncEnvLockfile (config: Config, context: ConfigContext): if (!semver.satisfies(packageManager.version, pm.version, { includePrerelease: true })) return const envLockfile = await readEnvLockfile(context.rootProjectManifestDir) - if (envLockfile == null) return - const lockedVersion = envLockfile.importers['.'].packageManagerDependencies?.['pnpm']?.version - if (lockedVersion == null) return - if (semver.satisfies(lockedVersion, pm.version, { includePrerelease: true })) return + const lockedVersion = envLockfile?.importers['.'].packageManagerDependencies?.['pnpm']?.version + if (lockedVersion != null && semver.satisfies(lockedVersion, pm.version, { includePrerelease: true })) return const store = await createStoreController({ ...config, ...context }) try { await resolvePackageManagerIntegrities(packageManager.version, { - envLockfile, + envLockfile: envLockfile ?? undefined, registries: config.registries, rootDir: context.rootProjectManifestDir, storeController: store.ctrl, From 8df408c9017609088da9a5f77ea3fd669c75425f Mon Sep 17 00:00:00 2001 From: shiminshen Date: Mon, 18 May 2026 06:13:49 +0800 Subject: [PATCH 007/169] fix(config): warn when package.json has a legacy "pnpm" field with migrated settings (#11680) * fix(config): warn when package.json has a legacy "pnpm" field In v11, pnpm stopped reading settings from the `pnpm` field of package.json (#10086). Most former pnpm-field settings now live in `pnpm-workspace.yaml`; a few (e.g. `onlyBuiltDependencies`, `executionEnv`) were removed entirely. Until now the old field was silently ignored, so users upgrading from v10 had no signal that their overrides or patched dependencies had stopped taking effect. Emit a warning whenever the `pnpm` field contains any key that pnpm no longer reads from package.json. The check is an allowlist (only `pnpm.app`, consumed by `pnpm pack-app`, is still active), so the warning won't go stale as new settings are added or removed in future versions. The message points users at https://pnpm.io/settings rather than prescribing a single fix, since the new home depends on the key. Closes #11677. * fix(config): only warn for migrated pnpm-field keys, not unrelated ones Previously the warning fired for every key under `pnpm` except `app`, which would surface false positives for third-party tooling that piggybacks on the `pnpm` namespace. Switch to an explicit denylist of the v10 settings that moved to pnpm-workspace.yaml, matching the PR's stated contract. --------- Co-authored-by: Damon Co-authored-by: Zoltan Kochan --- .../warn-deprecated-pnpm-field-11677.md | 6 +++ config/reader/src/index.ts | 36 ++++++++++++++++ .../pkg-with-legacy-pnpm-field/package.json | 10 +++++ .../pkg-with-pnpm-app-field/package.json | 7 ++++ .../pkg-with-unknown-pnpm-field/package.json | 5 +++ config/reader/test/index.ts | 41 +++++++++++++++++++ 6 files changed, 105 insertions(+) create mode 100644 .changeset/warn-deprecated-pnpm-field-11677.md create mode 100644 config/reader/test/fixtures/pkg-with-legacy-pnpm-field/package.json create mode 100644 config/reader/test/fixtures/pkg-with-pnpm-app-field/package.json create mode 100644 config/reader/test/fixtures/pkg-with-unknown-pnpm-field/package.json diff --git a/.changeset/warn-deprecated-pnpm-field-11677.md b/.changeset/warn-deprecated-pnpm-field-11677.md new file mode 100644 index 0000000000..0c177c42ea --- /dev/null +++ b/.changeset/warn-deprecated-pnpm-field-11677.md @@ -0,0 +1,6 @@ +--- +"@pnpm/config.reader": patch +"pnpm": patch +--- + +Warn when `package.json` contains a legacy `pnpm` field with settings pnpm no longer reads from `package.json` (e.g. `pnpm.overrides`, `pnpm.patchedDependencies`). Previously these were silently ignored after the upgrade from v10, leaving users unaware that their overrides/patched dependencies had stopped taking effect [#11677](https://github.com/pnpm/pnpm/issues/11677). diff --git a/config/reader/src/index.ts b/config/reader/src/index.ts index b0cc36a4ef..1d8eb41f37 100644 --- a/config/reader/src/index.ts +++ b/config/reader/src/index.ts @@ -404,6 +404,10 @@ export async function getConfig (opts: { if (pnpmConfig.rootProjectManifest.workspaces?.length && !pnpmConfig.workspaceDir) { warnings.push('The "workspaces" field in package.json is not supported by pnpm. Create a "pnpm-workspace.yaml" file instead.') } + const ignoredPnpmFieldKeys = getIgnoredPnpmFieldKeys(pnpmConfig.rootProjectManifest) + if (ignoredPnpmFieldKeys.length > 0) { + warnings.push(`The "pnpm" field in package.json is no longer read by pnpm. The following keys were ignored: ${ignoredPnpmFieldKeys.map(k => `"pnpm.${k}"`).join(', ')}. See https://pnpm.io/settings for the new home of each setting.`) + } const wantedPmResult = getWantedPackageManager(pnpmConfig.rootProjectManifest) if (wantedPmResult.pm) { pnpmConfig.wantedPackageManager = wantedPmResult.pm @@ -750,6 +754,38 @@ function getWantedPackageManager (manifest: ProjectManifest): { pm?: WantedPacka return { warnings } } +// Settings that used to be read from the `pnpm` field of `package.json` in v10 +// but moved to `pnpm-workspace.yaml` in v11. Keys not in this set (e.g. `app`, +// or anything set by third-party tooling that piggybacks on the `pnpm` namespace) +// are left alone to avoid false-positive warnings. +const MIGRATED_PNPM_FIELD_KEYS = new Set([ + 'allowBuilds', + 'allowedDeprecatedVersions', + 'allowUnusedPatches', + 'auditConfig', + 'configDependencies', + 'executionEnv', + 'ignoredOptionalDependencies', + 'neverBuiltDependencies', + 'onlyBuiltDependencies', + 'onlyBuiltDependenciesFile', + 'overrides', + 'packageExtensions', + 'patchedDependencies', + 'peerDependencyRules', + 'requiredScripts', + 'supportedArchitectures', + 'updateConfig', +]) + +function getIgnoredPnpmFieldKeys (manifest: ProjectManifest): string[] { + const legacyField = (manifest as { pnpm?: unknown }).pnpm + if (legacyField == null || typeof legacyField !== 'object' || Array.isArray(legacyField)) { + return [] + } + return Object.keys(legacyField as Record).filter(k => MIGRATED_PNPM_FIELD_KEYS.has(k)) +} + export function parsePackageManager (packageManager: string): { name: string, version: string | undefined } { if (!packageManager.includes('@')) return { name: packageManager, version: undefined } const [name, pmReference] = packageManager.split('@') diff --git a/config/reader/test/fixtures/pkg-with-legacy-pnpm-field/package.json b/config/reader/test/fixtures/pkg-with-legacy-pnpm-field/package.json new file mode 100644 index 0000000000..adcad06407 --- /dev/null +++ b/config/reader/test/fixtures/pkg-with-legacy-pnpm-field/package.json @@ -0,0 +1,10 @@ +{ + "pnpm": { + "overrides": { + "lodash": "^4.17.21" + }, + "patchedDependencies": { + "is-odd": "patches/is-odd.patch" + } + } +} diff --git a/config/reader/test/fixtures/pkg-with-pnpm-app-field/package.json b/config/reader/test/fixtures/pkg-with-pnpm-app-field/package.json new file mode 100644 index 0000000000..9cfaa842a7 --- /dev/null +++ b/config/reader/test/fixtures/pkg-with-pnpm-app-field/package.json @@ -0,0 +1,7 @@ +{ + "pnpm": { + "app": { + "entry": "dist/index.js" + } + } +} diff --git a/config/reader/test/fixtures/pkg-with-unknown-pnpm-field/package.json b/config/reader/test/fixtures/pkg-with-unknown-pnpm-field/package.json new file mode 100644 index 0000000000..3e90c1aada --- /dev/null +++ b/config/reader/test/fixtures/pkg-with-unknown-pnpm-field/package.json @@ -0,0 +1,5 @@ +{ + "pnpm": { + "foo": "bar" + } +} diff --git a/config/reader/test/index.ts b/config/reader/test/index.ts index cee7de949f..a4403fc924 100644 --- a/config/reader/test/index.ts +++ b/config/reader/test/index.ts @@ -1668,6 +1668,47 @@ test('do not return a warning if a package.json has workspaces field and there i expect(warnings).toStrictEqual([]) }) +test('return a warning if a package.json has a legacy "pnpm" field with ignored settings', async () => { + const prefix = f.find('pkg-with-legacy-pnpm-field') + const { warnings } = await getConfig({ + cliOptions: { dir: prefix }, + packageManager: { + name: 'pnpm', + version: '1.0.0', + }, + }) + + expect(warnings).toStrictEqual([ + 'The "pnpm" field in package.json is no longer read by pnpm. The following keys were ignored: "pnpm.overrides", "pnpm.patchedDependencies". See https://pnpm.io/settings for the new home of each setting.', + ]) +}) + +test('do not return a warning if a package.json "pnpm" field only contains keys that are still actively read (e.g. "pnpm.app")', async () => { + const prefix = f.find('pkg-with-pnpm-app-field') + const { warnings } = await getConfig({ + cliOptions: { dir: prefix }, + packageManager: { + name: 'pnpm', + version: '1.0.0', + }, + }) + + expect(warnings).toStrictEqual([]) +}) + +test('do not return a warning if a package.json "pnpm" field only contains keys unrelated to migrated settings (e.g. set by third-party tooling)', async () => { + const prefix = f.find('pkg-with-unknown-pnpm-field') + const { warnings } = await getConfig({ + cliOptions: { dir: prefix }, + packageManager: { + name: 'pnpm', + version: '1.0.0', + }, + }) + + expect(warnings).toStrictEqual([]) +}) + test('read PNPM_HOME defined in environment variables', async () => { const oldEnv = process.env const homeDir = './specified-dir' From f46757d732ec448253891425c19872bb15a59aed Mon Sep 17 00:00:00 2001 From: Zoltan Kochan Date: Mon, 18 May 2026 01:20:45 +0200 Subject: [PATCH 008/169] fix(cmd-shim): symlink node runtime binary instead of cmd-shimming it (#11707) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(cmd-shim): symlink node runtime binary instead of cmd-shimming it Ports pnpm v11's `cmd.name === 'node'` short-circuit from `bins/linker/src/index.ts:281-308`: on Unix, symlink `.bin/node` directly at the runtime binary; on Windows, hardlink (or copy on hardlink failure) the source to `.exe` when the source ends in `.exe`, otherwise fall through to the cmd-shim path. Adds two `LinkBinsError` variants (`RemoveStaleBin`, `LinkNodeBin`). Without the short-circuit, the node binary was wrapped in a `/bin/sh "$basedir/../node/bin/node"` shim. On any subsequent run that read it back as a shim source, `search_script_runtime` parsed the `#!/bin/sh` shebang and emitted a wrapper around the wrapper, with a self-referencing relative target (`node/bin/../node/bin/node` — the `node` segment appears twice and resolves to a non-existent path). `remove_file`-before-link, rather than `fs::write` truncation, also prevents the corrupt-the-source-binary failure mode that arises when the existing dirent is hardlinked into the GVS slot from the store. Tests mirror `bins/linker/test/index.ts:643-735` and add a regression test (`link_node_bin_does_not_corrupt_hardlinked_target`) that pre-hardlinks the bin slot to the source and verifies the binary content survives. --- Written by an agent (Claude Code, claude-opus-4-7). * chore(cmd-shim): rewrite "mis-handle" to "mishandle" for typos check The hyphenated form trips `typos` ("mis" flagged as "miss"/"mist"). No behavior change. --- Written by an agent (Claude Code, claude-opus-4-7). --- pacquet/crates/cmd-shim/src/link_bins.rs | 119 ++++++++++ .../crates/cmd-shim/src/link_bins/tests.rs | 223 ++++++++++++++++++ 2 files changed, 342 insertions(+) diff --git a/pacquet/crates/cmd-shim/src/link_bins.rs b/pacquet/crates/cmd-shim/src/link_bins.rs index d8fa919d8e..da6ae5c6d3 100644 --- a/pacquet/crates/cmd-shim/src/link_bins.rs +++ b/pacquet/crates/cmd-shim/src/link_bins.rs @@ -155,6 +155,23 @@ pub enum LinkBinsError { #[error(source)] error: io::Error, }, + + #[display("Failed to remove stale bin at {path:?}: {error}")] + #[diagnostic(code(pacquet_cmd_shim::remove_stale_bin))] + RemoveStaleBin { + path: PathBuf, + #[error(source)] + error: io::Error, + }, + + #[display("Failed to link node runtime binary {src:?} -> {dst:?}: {error}")] + #[diagnostic(code(pacquet_cmd_shim::link_node_bin))] + LinkNodeBin { + src: PathBuf, + dst: PathBuf, + #[error(source)] + error: io::Error, + }, } /// Read `/package.json` for each entry under `modules_dir` and link @@ -402,6 +419,31 @@ fn write_shim(target_path: &Path, shim_path: &Path) -> Result<(), LinkBinsE where Api: FsReadString + FsReadHead + FsWrite + FsSetExecutable + FsEnsureExecutableBits, { + // The node runtime binary is special: never wrap it in a shell + // shim. Mirrors pnpm v11's `cmd.name === 'node'` short-circuit in + // [`bins/linker/src/index.ts`](https://github.com/pnpm/pnpm/blob/06d2d3deb2/bins/linker/src/index.ts#L281-L308), + // which symlinks the binary on Unix and hardlinks `node.exe` on + // Windows. + // + // Two reasons this matters: + // + // 1. Parity. pnpm install in the same workspace symlinks `.bin/node` + // to the runtime binary; pacquet must do the same so the + // `same_global_virtual_store_layout_*` checks see the same + // dirent shape. + // 2. Robustness against accidental shim-wrapping. The node binary + // itself has no shebang, but a prior bad install may leave a + // cmd-shim text file with `#!/bin/sh` at `/bin/node`. If + // pacquet then cmd-shims that file, `search_script_runtime` + // parses the shebang as `prog: "/bin/sh"` and emits a shim + // whose target resolves to a non-existent path + // (`$basedir/../node/bin/../node/bin/node` — the `node` segment + // appears twice). A direct symlink / hardlink bypasses the + // parser entirely. + if is_node_bin_name(shim_path) && link_node_bin(target_path, shim_path)? { + return Ok(()); + } + let runtime = search_script_runtime::(target_path).map_err(|error| { LinkBinsError::ProbeShimSource { path: target_path.to_path_buf(), error } })?; @@ -491,6 +533,83 @@ where Ok(()) } +/// Whether `shim_path`'s file name is exactly `node` — the trigger for the +/// node-runtime short-circuit in [`write_shim`]. Lifted out so the check +/// is unit-testable and the call site reads as a predicate. +fn is_node_bin_name(shim_path: &Path) -> bool { + matches!(shim_path.file_name().and_then(|s| s.to_str()), Some("node")) +} + +/// Link the node runtime binary `target_path` into the bin slot +/// `shim_path` directly, without a cmd-shim wrapper. Returns `Ok(true)` +/// when the special case took effect (the caller must skip the regular +/// shim-writing path) and `Ok(false)` when it didn't apply and the +/// caller should fall through (Windows non-`.exe` source). +/// +/// Mirrors the two halves of pnpm's `cmd.name === 'node'` branch: +/// +/// - **Unix** symlinks `shim_path` → absolute `target_path`. The +/// existing dirent (if any) is removed first because `fs::symlink` +/// rejects with `AlreadyExists` and we don't want to silently leave +/// a stale shim in place. +/// - **Windows** hardlinks `target_path` to `.exe`, falling +/// back to `fs::copy` on hardlink failure (cross-device, ACL deny, +/// …). The source must end in `.exe`; otherwise pnpm falls through +/// to the cmd-shim path and so do we. +/// +/// `remove_file` rather than `Api::write`-style truncation is +/// load-bearing on both platforms: if `shim_path` is currently a +/// regular file hardlinked to the source binary (a state an earlier +/// pacquet revision could leave behind), truncating through the +/// hardlink would corrupt the binary itself. Removing the dirent +/// leaves the hardlinked content intact. +#[cfg(unix)] +fn link_node_bin(target_path: &Path, shim_path: &Path) -> Result { + use std::os::unix::fs::symlink; + remove_stale_bin(shim_path)?; + symlink(target_path, shim_path).map_err(|error| LinkBinsError::LinkNodeBin { + src: target_path.to_path_buf(), + dst: shim_path.to_path_buf(), + error, + })?; + Ok(true) +} + +#[cfg(windows)] +fn link_node_bin(target_path: &Path, shim_path: &Path) -> Result { + use std::fs; + let is_exe = target_path + .extension() + .and_then(|s| s.to_str()) + .is_some_and(|ext| ext.eq_ignore_ascii_case("exe")); + if !is_exe { + return Ok(false); + } + let exe_path = with_extension_appended(shim_path, "exe"); + remove_stale_bin(&exe_path)?; + if fs::hard_link(target_path, &exe_path).is_err() { + fs::copy(target_path, &exe_path).map_err(|error| LinkBinsError::LinkNodeBin { + src: target_path.to_path_buf(), + dst: exe_path, + error, + })?; + } + Ok(true) +} + +/// Remove an existing dirent at `path`, swallowing `NotFound`. Used by +/// [`link_node_bin`] to clear any prior shim / symlink / hardlink +/// before laying down the new one. Any other IO error (PermissionDenied, +/// EROFS, AppArmor deny, …) surfaces as [`LinkBinsError::RemoveStaleBin`] +/// so a real failure isn't hidden behind a silent skip. +fn remove_stale_bin(path: &Path) -> Result<(), LinkBinsError> { + match std::fs::remove_file(path) { + Ok(()) => Ok(()), + Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(()), + Err(error) => Err(LinkBinsError::RemoveStaleBin { path: path.to_path_buf(), error }), + } +} + /// Append `` to `path` as a *new* extension segment (`foo` becomes /// `foo.cmd`), regardless of any existing extension. `Path::with_extension` /// would *replace* the existing extension, which is wrong for our case. diff --git a/pacquet/crates/cmd-shim/src/link_bins/tests.rs b/pacquet/crates/cmd-shim/src/link_bins/tests.rs index 3f392e225a..d7818a9aea 100644 --- a/pacquet/crates/cmd-shim/src/link_bins/tests.rs +++ b/pacquet/crates/cmd-shim/src/link_bins/tests.rs @@ -1058,3 +1058,226 @@ fn hoisted_origin_loses_to_existing_direct() { "Direct incumbent must shut out Hoisted candidate, got body:\n{body}", ); } + +/// Mirrors pnpm's `linkBinsOfPackages() symlinks node binary directly +/// instead of creating a shell shim` at +/// . +/// +/// The `node` bin must land as a symlink to the real binary, never a +/// `/bin/sh`-wrapped shim. Wrapping is the recursion trap described in +/// [`super::link_node_bin`]'s doc comment. +#[cfg(unix)] +#[test] +fn link_node_bin_symlinks_directly_instead_of_writing_shim() { + let tmp = tempdir().unwrap(); + let bin_target = tmp.path().join("bin_target"); + let node_dir = tmp.path().join("node_pkg"); + let node_bin_dir = node_dir.join("bin"); + create_dir_all(&node_bin_dir).unwrap(); + write_file(node_bin_dir.join("node"), "fake-node-binary").unwrap(); + write_file( + node_dir.join("package.json"), + json!({"name": "node", "version": "20.0.0", "bin": {"node": "bin/node"}}).to_string(), + ) + .unwrap(); + + let manifest: Value = + serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); + link_bins_of_packages::( + &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], + &bin_target, + ) + .unwrap(); + + let bin_location = bin_target.join("node"); + let meta = std::fs::symlink_metadata(&bin_location).unwrap(); + assert!(meta.file_type().is_symlink(), "node bin must be a symlink, not a shim file"); + assert_eq!( + std::fs::canonicalize(&bin_location).unwrap(), + std::fs::canonicalize(node_bin_dir.join("node")).unwrap(), + "symlink must resolve to the real binary", + ); + // The target binary must not have been mutated to a sh-shim text. + assert_eq!( + read_to_string(node_bin_dir.join("node")).unwrap(), + "fake-node-binary", + "node bin special case must not rewrite the underlying binary", + ); +} + +/// Mirrors pnpm's `linkBinsOfPackages() replaces a dangling symlink +/// when linking node binary` at +/// . +/// +/// A previous install can leave a dangling symlink at `bin/node` when +/// the prior store entry was pruned. The next install must overwrite +/// it; `fs::symlink` would otherwise error with `AlreadyExists`. +#[cfg(unix)] +#[test] +fn link_node_bin_replaces_dangling_symlink() { + use std::os::unix::fs::symlink; + let tmp = tempdir().unwrap(); + let bin_target = tmp.path().join("bin_target"); + create_dir_all(&bin_target).unwrap(); + let dangling_target = tmp.path().join("does_not_exist"); + symlink(&dangling_target, bin_target.join("node")).unwrap(); + assert!( + std::fs::metadata(bin_target.join("node")).is_err(), + "precondition: symlink must be dangling", + ); + + let node_dir = tmp.path().join("node_pkg"); + let node_bin_dir = node_dir.join("bin"); + create_dir_all(&node_bin_dir).unwrap(); + write_file(node_bin_dir.join("node"), "fake-node-binary").unwrap(); + write_file( + node_dir.join("package.json"), + json!({"name": "node", "version": "20.0.0", "bin": {"node": "bin/node"}}).to_string(), + ) + .unwrap(); + + let manifest: Value = + serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); + link_bins_of_packages::( + &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], + &bin_target, + ) + .unwrap(); + + let stat = std::fs::symlink_metadata(bin_target.join("node")).unwrap(); + assert!(stat.file_type().is_symlink()); + assert_eq!( + std::fs::canonicalize(bin_target.join("node")).unwrap(), + std::fs::canonicalize(node_bin_dir.join("node")).unwrap(), + ); +} + +/// Regression test for the corruption pattern that motivated the +/// node-bin short-circuit. Without the special case, if `bin/node` is +/// hardlinked into a pacquet slot and `/node` is also a +/// regular file hardlinked to the same inode (e.g. a prior pacquet +/// revision left it that way), then `fs::write` truncating the dirent +/// would rewrite the underlying node binary as a 459-byte +/// `/bin/sh`-wrapper text file — propagating to every project that +/// reflinks from the same store. +/// +/// The fix is `remove_file` followed by `fs::symlink`. `remove_file` +/// drops only the dirent, leaving the hardlinked content intact. +#[cfg(unix)] +#[test] +fn link_node_bin_does_not_corrupt_hardlinked_target() { + let tmp = tempdir().unwrap(); + let bin_target = tmp.path().join("bin_target"); + create_dir_all(&bin_target).unwrap(); + + let node_dir = tmp.path().join("node_pkg"); + let node_bin_dir = node_dir.join("bin"); + create_dir_all(&node_bin_dir).unwrap(); + write_file(node_bin_dir.join("node"), "fake-node-binary").unwrap(); + // Hardlink the binary into the would-be bin slot, simulating the + // disk state that produced the upstream corruption. + std::fs::hard_link(node_bin_dir.join("node"), bin_target.join("node")).unwrap(); + + write_file( + node_dir.join("package.json"), + json!({"name": "node", "version": "20.0.0", "bin": {"node": "bin/node"}}).to_string(), + ) + .unwrap(); + + let manifest: Value = + serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); + link_bins_of_packages::( + &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], + &bin_target, + ) + .unwrap(); + + assert_eq!( + read_to_string(node_bin_dir.join("node")).unwrap(), + "fake-node-binary", + "real node binary must not be rewritten by the bin linker", + ); +} + +/// Mirrors pnpm's `linkBinsOfPackages() hardlinks node.exe instead of +/// creating a cmd-shim` at +/// . +/// +/// On Windows the canonical bin dirent for the node runtime is +/// `/node.exe` — a hardlink (or copy fallback) of the source +/// `.exe`. No `.cmd` or `.ps1` shim is emitted, because npm's cmd +/// shims call `node.exe` from `IF EXIST` blocks that mishandle a +/// `.cmd` redirection. +#[cfg(windows)] +#[test] +fn link_node_bin_hardlinks_node_exe_on_windows() { + let tmp = tempdir().unwrap(); + let bin_target = tmp.path().join("bin_target"); + let node_dir = tmp.path().join("node_pkg"); + create_dir_all(&node_dir).unwrap(); + write_file(node_dir.join("node.exe"), "fake-node-binary").unwrap(); + write_file( + node_dir.join("package.json"), + json!({"name": "node", "version": "20.0.0", "bin": {"node": "node.exe"}}).to_string(), + ) + .unwrap(); + + let manifest: Value = + serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); + link_bins_of_packages::( + &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], + &bin_target, + ) + .unwrap(); + + let exe = bin_target.join("node.exe"); + assert!(exe.exists(), "node.exe must be created in the bin dir"); + assert_eq!(read_to_string(&exe).unwrap(), "fake-node-binary"); + // No canonical shim, .cmd, or .ps1 should be written. + assert!( + !bin_target.join("node").exists(), + "canonical shim must not be written for the node special case", + ); + assert!( + !bin_target.join("node.cmd").exists(), + ".cmd shim must not be written for the node special case", + ); + assert!( + !bin_target.join("node.ps1").exists(), + ".ps1 shim must not be written for the node special case", + ); +} + +/// Windows-only: when the node manifest declares a non-`.exe` source +/// (uncommon but possible — e.g. a wrapper script), pnpm falls through +/// to the regular cmd-shim path. Pacquet must too. +#[cfg(windows)] +#[test] +fn link_node_bin_falls_through_to_cmd_shim_when_source_is_not_exe() { + let tmp = tempdir().unwrap(); + let bin_target = tmp.path().join("bin_target"); + let node_dir = tmp.path().join("node_pkg"); + create_dir_all(node_dir.join("bin")).unwrap(); + write_file(node_dir.join("bin/node"), "#!/usr/bin/env node\nconsole.log(1)\n").unwrap(); + write_file( + node_dir.join("package.json"), + json!({"name": "node", "version": "20.0.0", "bin": {"node": "bin/node"}}).to_string(), + ) + .unwrap(); + + let manifest: Value = + serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); + link_bins_of_packages::( + &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], + &bin_target, + ) + .unwrap(); + + // The non-`.exe` node source falls through to the cmd-shim path, + // so the canonical sh shim, `.cmd`, and `.ps1` siblings all land + // exactly as for any other bin. + assert!(bin_target.join("node").exists()); + assert!(bin_target.join("node.cmd").exists()); + assert!(bin_target.join("node.ps1").exists()); + assert!(!bin_target.join("node.exe").exists()); +} From 247d70b40c5f0abefd13a3263ee33cea5c45e1c7 Mon Sep 17 00:00:00 2001 From: Rayan Salhab Date: Mon, 18 May 2026 02:57:46 +0300 Subject: [PATCH 009/169] fix: silence verify-deps auto-install output (#11679) * fix: silence verify-deps auto-install output * fix: pass reporter to dependency status install --------- Co-authored-by: cyphercodes --- .changeset/fix-verify-deps-silent-install.md | 7 +++++++ exec/commands/src/exec.ts | 1 + exec/commands/src/runDepsStatusCheck.ts | 5 +++-- exec/pnpm-cli-runner/src/index.ts | 14 ++++++++++---- pnpm/test/exec.ts | 15 +++++++++++++++ pnpm/test/run.ts | 18 ++++++++++++++++++ 6 files changed, 54 insertions(+), 6 deletions(-) create mode 100644 .changeset/fix-verify-deps-silent-install.md diff --git a/.changeset/fix-verify-deps-silent-install.md b/.changeset/fix-verify-deps-silent-install.md new file mode 100644 index 0000000000..32306f6c5e --- /dev/null +++ b/.changeset/fix-verify-deps-silent-install.md @@ -0,0 +1,7 @@ +--- +"@pnpm/exec.commands": patch +"@pnpm/exec.pnpm-cli-runner": patch +"pnpm": patch +--- + +Honor `--silent` when `verifyDepsBeforeRun: install` auto-installs dependencies before `pnpm run` or `pnpm exec`, preventing install output from being written to stdout [#11636](https://github.com/pnpm/pnpm/issues/11636). diff --git a/exec/commands/src/exec.ts b/exec/commands/src/exec.ts index 330f0ad89e..b11736395d 100644 --- a/exec/commands/src/exec.ts +++ b/exec/commands/src/exec.ts @@ -156,6 +156,7 @@ export type ExecOpts = Required> & | 'nodeOptions' | 'pnpmHomeDir' | 'recursive' +| 'reporter' | 'reporterHidePrefix' | 'userAgent' | 'verifyDepsBeforeRun' diff --git a/exec/commands/src/runDepsStatusCheck.ts b/exec/commands/src/runDepsStatusCheck.ts index 900fcc7189..136db411ba 100644 --- a/exec/commands/src/runDepsStatusCheck.ts +++ b/exec/commands/src/runDepsStatusCheck.ts @@ -1,4 +1,4 @@ -import type { VerifyDepsBeforeRun } from '@pnpm/config.reader' +import type { Config, VerifyDepsBeforeRun } from '@pnpm/config.reader' import { checkDepsStatus, type CheckDepsStatusOptions, type WorkspaceStateSettings } from '@pnpm/deps.status' import { PnpmError } from '@pnpm/error' import { runPnpmCli } from '@pnpm/exec.pnpm-cli-runner' @@ -7,6 +7,7 @@ import enquirer from 'enquirer' export interface RunDepsStatusCheckOptions extends CheckDepsStatusOptions { dir: string + reporter?: Config['reporter'] verifyDepsBeforeRun?: VerifyDepsBeforeRun } @@ -20,7 +21,7 @@ export async function runDepsStatusCheck (opts: RunDepsStatusCheckOptions): Prom if (upToDate) return const command = ['install', ...createInstallArgs(workspaceState?.settings)] - const install = runPnpmCli.bind(null, command, { cwd: opts.dir }) + const install = runPnpmCli.bind(null, command, { cwd: opts.dir, reporter: opts.reporter }) switch (opts.verifyDepsBeforeRun) { case 'install': diff --git a/exec/pnpm-cli-runner/src/index.ts b/exec/pnpm-cli-runner/src/index.ts index 9161af10bd..9df02756f4 100644 --- a/exec/pnpm-cli-runner/src/index.ts +++ b/exec/pnpm-cli-runner/src/index.ts @@ -2,17 +2,23 @@ import path from 'node:path' import { sync as execSync } from 'execa' -export function runPnpmCli (command: string[], { cwd }: { cwd: string }): void { +export interface RunPnpmCliOptions { + cwd: string + reporter?: string +} + +export function runPnpmCli (command: string[], { cwd, reporter }: RunPnpmCliOptions): void { const execOpts = { cwd, stdio: 'inherit' as const, } + const cliCommand = reporter ? [...command, `--reporter=${reporter}`] : command const execFileName = path.basename(process.execPath).toLowerCase() if (execFileName === 'pnpm' || execFileName === 'pnpm.exe') { - execSync(process.execPath, command, execOpts) + execSync(process.execPath, cliCommand, execOpts) } else if (path.basename(process.argv[1]) === 'pnpm.mjs') { - execSync(process.execPath, [process.argv[1], ...command], execOpts) + execSync(process.execPath, [process.argv[1], ...cliCommand], execOpts) } else { - execSync('pnpm', command, execOpts) + execSync('pnpm', cliCommand, execOpts) } } diff --git a/pnpm/test/exec.ts b/pnpm/test/exec.ts index eb4cc883b1..08023d63b9 100644 --- a/pnpm/test/exec.ts +++ b/pnpm/test/exec.ts @@ -3,6 +3,7 @@ import path from 'node:path' import { expect, test } from '@jest/globals' import { prepare } from '@pnpm/prepare' +import { writeYamlFileSync } from 'write-yaml-file' import { execPnpm, execPnpmSync } from './utils/index.js' @@ -30,3 +31,17 @@ test("exec should respect the caller's current working directory", async () => { expect(fs.readFileSync(cmdFilePath, 'utf8')).toBe(subdirPath) }) + +test('silent exec does not print verifyDepsBeforeRun install output', async () => { + prepare({}) + writeYamlFileSync('pnpm-workspace.yaml', { + verifyDepsBeforeRun: 'install', + }) + + const result = execPnpmSync(['--silent', 'exec', 'node', '-e', 'process.stdout.write("hi")'], { + expectSuccess: true, + omitEnvDefaults: ['pnpm_config_silent'], + }) + + expect(result.stdout.toString()).toBe('hi') +}) diff --git a/pnpm/test/run.ts b/pnpm/test/run.ts index 5a2fa4e9fd..e531c7d5de 100644 --- a/pnpm/test/run.ts +++ b/pnpm/test/run.ts @@ -140,6 +140,24 @@ test('silent run only prints the output of the child process', async () => { expect(result.stdout.toString().trim()).toBe('hi') }) +test('silent run does not print verifyDepsBeforeRun install output', async () => { + prepare({ + scripts: { + hi: 'echo hi', + }, + }) + writeYamlFileSync('pnpm-workspace.yaml', { + verifyDepsBeforeRun: 'install', + }) + + const result = execPnpmSync(['run', '--silent', 'hi'], { + expectSuccess: true, + omitEnvDefaults: ['pnpm_config_silent'], + }) + + expect(result.stdout.toString().trim()).toBe('hi') +}) + testOnPosix('pnpm run with preferSymlinkedExecutables true', async () => { prepare({ scripts: { From 02f8138f1368a884d8bced31b6e6354546e2000b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kh=E1=BA=A3i?= Date: Mon, 18 May 2026 13:08:23 +0700 Subject: [PATCH 010/169] refactor(pacquet): optional DI pattern + documentation (#11708) Co-authored-by: Claude --- pacquet/AGENTS.md | 14 ++ pacquet/CODE_STYLE_GUIDE.md | 182 ++++++++++++++++++ pacquet/crates/cli/src/cli_args.rs | 6 +- pacquet/crates/cli/tests/install.rs | 4 +- pacquet/crates/cmd-shim/src/bin_resolver.rs | 8 +- .../crates/cmd-shim/src/bin_resolver/tests.rs | 48 ++--- pacquet/crates/cmd-shim/src/capabilities.rs | 30 +-- pacquet/crates/cmd-shim/src/link_bins.rs | 62 +++--- .../crates/cmd-shim/src/link_bins/tests.rs | 71 ++++--- pacquet/crates/cmd-shim/src/shim.rs | 12 +- pacquet/crates/cmd-shim/src/shim/tests.rs | 37 ++-- pacquet/crates/config/src/api.rs | 45 ++--- pacquet/crates/config/src/env_replace.rs | 14 +- pacquet/crates/config/src/lib.rs | 50 ++--- pacquet/crates/config/src/npmrc_auth.rs | 24 +-- pacquet/crates/modules-yaml/src/lib.rs | 30 +-- pacquet/crates/modules-yaml/tests/index.rs | 16 +- pacquet/crates/modules-yaml/tests/real_fs.rs | 20 +- pacquet/crates/package-manager/src/install.rs | 6 +- .../package-manager/src/install/tests.rs | 14 +- .../src/install_frozen_lockfile.rs | 4 +- .../src/install_without_lockfile.rs | 4 +- .../crates/package-manager/src/link_bins.rs | 66 +++---- .../package-manager/src/link_bins/tests.rs | 4 +- pacquet/crates/reporter/src/lib.rs | 21 +- pacquet/crates/reporter/src/tests.rs | 18 +- 26 files changed, 506 insertions(+), 304 deletions(-) diff --git a/pacquet/AGENTS.md b/pacquet/AGENTS.md index c285e2d8ec..114267a5d9 100644 --- a/pacquet/AGENTS.md +++ b/pacquet/AGENTS.md @@ -42,6 +42,20 @@ Before writing code for a feature, bug fix, or behavior change: [Reporter / log events](./CODE_STYLE_GUIDE.md#reporter--log-events) in the style guide for the convention (channel mapping, threading `R: Reporter`, emit-site placement, recording-fake tests). +7. **Prefer real fixtures; reach for the dependency-injection seam + only when they can't cover the branch.** Most happy paths and + error paths should be tested with a `tempfile::TempDir`, the + mocked registry, or an integration test that spawns the actual + binary. Use the DI seam — a capability trait on the `Host` + provider, threaded as `Sys: ` — only for branches a real + fixture can't reach portably: filesystem error kinds + (`PermissionDenied`, `ENOSPC`, …), deterministic time, or the + external-service happy paths in features like `pnpm login` (2FA) + and `pnpm publish` (OIDC / provenance) when those land. See + [Dependency injection for tests](./CODE_STYLE_GUIDE.md#dependency-injection-for-tests) + in the style guide for the gating rule, the names (`Sys`, `Host`, + `Fs*`, `Clock`, `EnvVar`, …), the eight principles, and the + `modules-yaml` worked example. If the pnpm behavior is unclear or looks wrong, stop and ask the user rather than guessing. diff --git a/pacquet/CODE_STYLE_GUIDE.md b/pacquet/CODE_STYLE_GUIDE.md index 9d20b8f02f..8261487d28 100644 --- a/pacquet/CODE_STYLE_GUIDE.md +++ b/pacquet/CODE_STYLE_GUIDE.md @@ -641,6 +641,188 @@ The above code is still valid code, and the Rust compiler doesn't error, but it **Readability:** The generic `.clone()` or `Clone::clone` often implies an expensive operation (for example: cloning a `Vec`), but `Arc` and `Rc` are not as expensive as the generic `.clone()`. Explicitly marking the cloned type aids future refactoring. +### Dependency injection for tests + +Side-effecting code — filesystem access, environment variables, network calls, time, process state — has two testing routes. **The default route is a real fixture:** a `tempfile::TempDir` for filesystem work, the mocked registry (`just registry-mock`) for HTTP, an integration test that spawns the actual pacquet binary in a scratch directory for end-to-end flows. Real fixtures keep tests close to what users see and scale with the codebase without per-call-site plumbing; they are the right tool everywhere except the cases enumerated below. + +The dependency-injection seam described below is the **narrow second route**. Reach for it only when one of the following applies: + +- **Filesystem error branches the host OS won't reproduce portably.** `PermissionDenied`, `ENOSPC`, a directory that disappears mid-walk, a chmod that fails after the file exists — provoking these on real disks is platform-specific, racy, or both. A fake that returns the exact `io::ErrorKind` is the only portable way to drive the branch. +- **Deterministic time.** Asserting that `prunedAt` equals a specific HTTP-date (RFC 7231 IMF-fixdate, what `httpdate::fmt_http_date` emits), or that a throttled emitter fires on the second sample, needs the clock to be a known value. The real `SystemTime::now` makes those assertions flaky. +- **External-service happy paths that can't be staged in CI.** Upstream pnpm has features whose *normal* flow depends on real external systems — `pnpm login`'s 2FA prompt round-trip, the OIDC token exchange and provenance attestation in `pnpm publish`, and similar — where the happy path itself is what needs faking, not just the error path. When pacquet ports those features, DI is the right tool for their tests too. (As of writing, these are not yet ported; this exception is documented so the convention is in place when they land.) +- **Unreachable-by-design preconditions.** When a function declares a capability bound but a specific test exercises a branch that never reaches that capability, the fake satisfies the bound with `unreachable!` and documents the precondition. See the worked example below. + +A function that takes a `` generic but is only ever exercised via real fixtures is a smell — either the DI branches are missing coverage, or the generic is over-design. Either add the tests that justify the seam, or drop the generic and let the real fixture cover everything. + +The rest of this section is the convention that applies *when DI is the right tool*. + +#### Names + +- The generic type parameter is named **`Sys`** — short for "system seam," the slot in the function signature that selects between the real OS and the test fake. A single short name makes a generic call site instantly recognisable as the DI seam. +- The production provider struct is named **`Host`** — unqualified, because the production implementation is the default. Fakes carry behaviour-based names that describe what they do (`FailingRead`, `EmptyRead`, `PermissionDenied`, `FakeHostName`), not what category of thing they are. +- Capability traits use the form ``: filesystem capabilities are `Fs*` (`FsReadToString`, `FsCreateDirAll`, `FsWrite`, `FsReadDir`, `FsWalkFiles`, `FsSetExecutable`, `FsEnsureExecutableBits`); environment-variable lookup is `EnvVar`; clock reads are `Clock`; hostname lookup is `GetHostName`. The domain prefix lets a reader of a generic bound see which side effect the function reaches for without chasing definitions. Method names mirror their `std` equivalents so the trait is a thin seam over `std::fs::*` / `std::env::var` / `SystemTime::now`, not a re-imagining. + +#### Eight principles + +1. **Single-purpose traits.** Each capability gets its own trait — one method per side effect, no umbrella trait that bundles `read`, `write`, `create_dir_all` into one bag. A function then binds only the capabilities it actually consumes, and a test fake implements only the methods the function under test exercises. + +2. **One generic parameter with multiple bounds.** Compose bounds on a single `Sys`, never introduce a second type parameter per capability: + + ```rust + // Good: one parameter, composed bounds + pub fn read_modules_manifest(modules_dir: &Path) -> Result<...> + where + Sys: FsReadToString + Clock, + { /* ... */ } + + // Bad: a parameter per capability — every call site has to satisfy two slots + pub fn read_modules_manifest(modules_dir: &Path) -> Result<...> + where Fs: FsReadToString, C: Clock { /* ... */ } + ``` + + One parameter keeps turbofish call sites short (`read_modules_manifest::(dir)`) and makes the fake's job obvious: implement every trait in the bound list, no more. + +3. **Static methods, not `&self`.** Capability methods are associated functions (no `&self` receiver). The provider is a unit struct that carries no data: + + ```rust + pub trait FsReadToString { + fn read_to_string(path: &Path) -> io::Result; + } + + pub struct Host; + impl FsReadToString for Host { + fn read_to_string(path: &Path) -> io::Result { fs::read_to_string(path) } + } + ``` + + Stateful fakes (a fake clock that returns a fixed `SystemTime`, a recording fake that captures every call) store their state in an interior-mutable `static` declared inside the `#[test]` body, so the trait shape doesn't have to change to accommodate state. This keeps the production impl free of `&self` plumbing the test-only fake would otherwise force on it. + +4. **Associated types for data operations.** When a capability operates over a domain data type, expose the data type as an associated type rather than threading an instance through every call. The provider chooses the concrete type, and fakes can pick a stub-friendly stand-in. + +5. **Capability traits on the implementor.** `impl FsReadToString for Host` lives on the provider, not on the data type. The data types stay free of test-shim conditional impls; the seam is the provider. + +6. **Domain-neutral provider, domain-scoped traits.** The generic is `Sys`, the production type is `Host` (or whatever your crate exports as its provider), and the trait names carry the domain prefix (`Fs*`, `Env*`, `Clock`, `GetHostName`, …). A reader of `Sys: FsReadToString + Clock + EnvVar` knows immediately which side effects the function reaches for. + +7. **Explicit turbofish in production.** Production call sites name the provider: + + ```rust + read_modules_manifest::(modules_dir) + write_modules_manifest::(modules_dir, manifest) + Config::current::(env::current_dir, home::home_dir, Default::default) + ``` + + The turbofish makes the production choice visible at the call site instead of relying on type inference; if a future caller wants to swap in a different provider (a test driver, a dry-run shim), the spot to change is obvious. + +8. **Capabilities are primitives, not algorithms.** Each trait names a leaf-level effect that maps to a single `std` function (`read_to_string`, `create_dir_all`, `write`, `var`, …). Higher-level guarantees — atomic write, retry loops, walk-with-options — become free functions composed on top of those primitives, not new trait methods. That keeps the fake surface dead-simple: a fake declares one method per capability, never a knob-laden builder. + +#### Worked example: `modules-yaml` + +`crates/modules-yaml` reads and writes `node_modules/.modules.yaml`. The read path is generic over `FsReadToString + Clock` because it needs to read the file and stamp `prunedAt` from the wall clock; the write path is generic over `FsCreateDirAll + FsWrite` because it needs to ensure the parent directory exists before writing the serialized manifest: + +```rust +pub trait FsReadToString { + fn read_to_string(path: &Path) -> io::Result; +} + +pub trait FsCreateDirAll { + fn create_dir_all(path: &Path) -> io::Result<()>; +} + +pub trait FsWrite { + fn write(path: &Path, contents: &[u8]) -> io::Result<()>; +} + +pub trait Clock { + fn now() -> SystemTime; +} + +pub struct Host; + +impl FsReadToString for Host { + fn read_to_string(path: &Path) -> io::Result { fs::read_to_string(path) } +} +impl FsCreateDirAll for Host { + fn create_dir_all(path: &Path) -> io::Result<()> { fs::create_dir_all(path) } +} +impl FsWrite for Host { + fn write(path: &Path, contents: &[u8]) -> io::Result<()> { fs::write(path, contents) } +} +impl Clock for Host { + fn now() -> SystemTime { SystemTime::now() } +} + +pub fn read_modules_manifest(modules_dir: &Path) -> Result, ReadModulesError> +where + Sys: FsReadToString + Clock, +{ + let content = match Sys::read_to_string(&manifest_path) { /* ... */ }; + // ... + manifest.pruned_at = httpdate::fmt_http_date(Sys::now()); + Ok(Some(manifest)) +} +``` + +A test that wants to drive the `PermissionDenied` branch declares a unit-struct fake inside the `#[test]` body, implementing only the capability the function touches: + +```rust +#[test] +fn read_propagates_non_not_found_io_error() { + struct FailingRead; + impl FsReadToString for FailingRead { + fn read_to_string(_: &Path) -> io::Result { + Err(io::Error::new(io::ErrorKind::PermissionDenied, "mocked")) + } + } + // `read_modules_manifest`'s bound list is `FsReadToString + Clock`, + // so every fake must satisfy both bounds at the type level — Rust + // doesn't know that the `prunedAt` branch is unreachable for this + // input. The convention for capabilities the test won't exercise + // is a trivial impl whose body is `unreachable!`: the bound is + // satisfied, and the panic message documents the precondition the + // test relies on. + impl Clock for FailingRead { + fn now() -> SystemTime { + unreachable!("clock must not be called when read_to_string fails"); + } + } + let err = read_modules_manifest::(Path::new("/")).unwrap_err(); + assert!(matches!(err, ReadModulesError::ReadFile { .. })); +} +``` + +Stateful fakes (deterministic clock, recording reads) hold their state in a `static` inside the test fn: + +```rust +#[test] +fn read_fills_in_pruned_at_when_missing() { + static FAKE_NOW: SystemTime = SystemTime::UNIX_EPOCH; + struct FakeClock; + impl Clock for FakeClock { + fn now() -> SystemTime { FAKE_NOW } + } + // The `static` lives in this fn's scope, so other tests get + // independent storage and never race on it. The provider type + // stays an empty unit struct; the state lives next to the test + // that needs it. Use `SystemTime::UNIX_EPOCH` (a `const`) for a + // const-initialised static, or `LazyLock` when the desired value + // needs a runtime constructor. + // ... +} +``` + +#### Cross-domain composition + +When a function needs capabilities from more than one domain, list them inline on `Sys`: + +```rust +fn write_shim(target_path: &Path, shim_path: &Path) -> Result<(), LinkBinsError> +where + Sys: FsReadToString + FsReadHead + FsWrite + FsSetExecutable + FsEnsureExecutableBits, +{ /* ... */ } +``` + +The provider implements each trait independently, so adding a domain to an existing `Sys` is one more `impl X for Host` block — no churn on the production type beyond the new line, and no churn on existing tests beyond the ones whose fakes now need the new method. + ### Reporter / log events Pacquet's user-facing output mirrors pnpm's: every channel pnpm fires must fire from the corresponding pacquet site, with the same payload shape and the same firing cadence. The reporter lives in `crates/reporter` (the `Reporter` capability trait, the `LogEvent` enum, the `NdjsonReporter` and `SilentReporter` sinks); this section is the convention for porting emissions into ported functions. diff --git a/pacquet/crates/cli/src/cli_args.rs b/pacquet/crates/cli/src/cli_args.rs index 21305fc3fe..8af3ab24c9 100644 --- a/pacquet/crates/cli/src/cli_args.rs +++ b/pacquet/crates/cli/src/cli_args.rs @@ -9,7 +9,7 @@ use add::AddArgs; use clap::{Parser, Subcommand, ValueEnum}; use install::InstallArgs; use miette::{Context, IntoDiagnostic}; -use pacquet_config::{Config, RealApi}; +use pacquet_config::{Config, Host}; use pacquet_executor::execute_shell; use pacquet_package_manifest::PackageManifest; use pacquet_reporter::{NdjsonReporter, SilentReporter}; @@ -94,12 +94,12 @@ impl CliArgs { // builds its `localPrefix` from `cliOptions.dir`, not `cwd`) — // see [`loadNpmrcConfig`](https://github.com/pnpm/pnpm/blob/1819226b51/config/reader/src/loadNpmrcFiles.ts#L48-L50). // - // Production callers turbofish `RealApi` explicitly so the + // Production callers turbofish `Host` explicitly so the // dependency-injection plumbing is visible at the call site. // See [pnpm/pacquet#339](https://github.com/pnpm/pacquet/issues/339) // for the pattern and rationale. let config = || -> miette::Result<&'static mut Config> { - Config::current::( + Config::current::( || Ok::<_, std::convert::Infallible>(dir.clone()), home::home_dir, Default::default, diff --git a/pacquet/crates/cli/tests/install.rs b/pacquet/crates/cli/tests/install.rs index aa3d992bc8..0d339a6d1e 100644 --- a/pacquet/crates/cli/tests/install.rs +++ b/pacquet/crates/cli/tests/install.rs @@ -253,7 +253,7 @@ fn should_install_circular_dependencies() { /// End-to-end coverage for `${VAR}` substitution in `.npmrc`. /// -/// `::var` (the `std::env::var` bridge in +/// `::var` (the `std::env::var` bridge in /// `crates/config/src/api.rs`) is unreachable by every other test /// because `add_mocked_registry` writes literal values, so /// `env_replace` short-circuits at the no-`$` branch. @@ -264,7 +264,7 @@ fn should_install_circular_dependencies() { /// upstream's [`installing/deps-installer/test/install/auth.ts`](https://github.com/pnpm/pnpm/blob/601317e7a3/installing/deps-installer/test/install/auth.ts) /// is not exercised here. The mock registry doesn't gate on auth, so /// substituting the registry URL is the smallest scenario that drives -/// `::var` end-to-end. Token-substitution coverage +/// `::var` end-to-end. Token-substitution coverage /// belongs in a test against a registry that actually validates the /// header. #[test] diff --git a/pacquet/crates/cmd-shim/src/bin_resolver.rs b/pacquet/crates/cmd-shim/src/bin_resolver.rs index 941fa2d7c2..3621bf01ab 100644 --- a/pacquet/crates/cmd-shim/src/bin_resolver.rs +++ b/pacquet/crates/cmd-shim/src/bin_resolver.rs @@ -63,7 +63,7 @@ pub fn pkg_owns_bin(bin_name: &str, pkg_name: &str) -> bool { /// single-character `$`. This is the path-traversal guard. /// - Bin path must resolve under `pkg_path`. Prevents a malicious manifest /// from writing shims that exec a sibling package. -pub fn get_bins_from_package_manifest( +pub fn get_bins_from_package_manifest( manifest: &Value, pkg_path: &Path, ) -> Vec { @@ -74,7 +74,7 @@ pub fn get_bins_from_package_manifest( if let Some(bin_dir_rel) = manifest.get("directories").and_then(|d| d.get("bin")).and_then(Value::as_str) { - return commands_from_directories_bin::(bin_dir_rel, pkg_path); + return commands_from_directories_bin::(bin_dir_rel, pkg_path); } Vec::new() } @@ -87,7 +87,7 @@ pub fn get_bins_from_package_manifest( /// Symlinks are not followed; pnpm uses `tinyglobby` with /// `followSymbolicLinks: false`. Missing directory degrades to an empty /// list (pnpm's `ENOENT` short-circuit). -fn commands_from_directories_bin( +fn commands_from_directories_bin( bin_dir_rel: &str, pkg_path: &Path, ) -> Vec { @@ -99,7 +99,7 @@ fn commands_from_directories_bin( // tinyglobby ENOENT short-circuit. The trait's production impl // already drops per-entry errors inside its iterator, so an `Err` // here only fires when the walker can't even open `bin_dir`. - let Ok(paths) = Api::walk_files(&bin_dir) else { + let Ok(paths) = Sys::walk_files(&bin_dir) else { return Vec::new(); }; let mut commands = Vec::new(); diff --git a/pacquet/crates/cmd-shim/src/bin_resolver/tests.rs b/pacquet/crates/cmd-shim/src/bin_resolver/tests.rs index 0e4cba87a8..4b873ae943 100644 --- a/pacquet/crates/cmd-shim/src/bin_resolver/tests.rs +++ b/pacquet/crates/cmd-shim/src/bin_resolver/tests.rs @@ -1,5 +1,5 @@ use super::{get_bins_from_package_manifest, pkg_owns_bin}; -use crate::{capabilities::RealApi, path_util::lexical_normalize}; +use crate::{capabilities::Host, path_util::lexical_normalize}; use pipe_trait::Pipe; use serde_json::json; use std::{ @@ -11,7 +11,7 @@ use tempfile::tempdir; #[test] fn bin_as_string_uses_package_name() { let manifest = json!({"name": "foo", "bin": "cli.js"}); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/foo")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/foo")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "foo"); assert_eq!(commands[0].path, Path::new("/pkg/foo/cli.js")); @@ -20,7 +20,7 @@ fn bin_as_string_uses_package_name() { #[test] fn bin_as_string_strips_scope() { let manifest = json!({"name": "@scope/foo", "bin": "cli.js"}); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/foo")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/foo")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "foo"); } @@ -34,7 +34,7 @@ fn bin_as_object_keeps_keys_and_strips_scope() { "@scope/extra": "bin/extra.js", }, }); - let mut commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); + let mut commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); commands.sort_by(|a, b| a.name.cmp(&b.name)); assert_eq!(commands.len(), 2); assert_eq!(commands[0].name, "extra"); @@ -52,7 +52,7 @@ fn rejects_unsafe_bin_names() { "$": "dollar.js", }, }); - let mut names: Vec<_> = get_bins_from_package_manifest::(&manifest, Path::new("/p")) + let mut names: Vec<_> = get_bins_from_package_manifest::(&manifest, Path::new("/p")) .into_iter() .map(|c| c.name) .collect(); @@ -66,14 +66,14 @@ fn rejects_path_traversal_outside_package_root() { "name": "x", "bin": {"x": "../../../etc/passwd"}, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/x")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/x")); assert!(commands.is_empty(), "must reject `..`-escapes from pkg root"); } #[test] fn no_bin_field_returns_empty() { let manifest = json!({"name": "x"}); - assert!(get_bins_from_package_manifest::(&manifest, Path::new("/p")).is_empty()); + assert!(get_bins_from_package_manifest::(&manifest, Path::new("/p")).is_empty()); } #[test] @@ -101,7 +101,7 @@ fn dollar_is_allowed_as_command_name() { "version": "1.0.0", "bin": {"$": "./undollar.js"}, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "$"); } @@ -122,7 +122,7 @@ fn skip_dangerous_bin_names() { "~/bad": "./bad", }, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "good"); } @@ -140,7 +140,7 @@ fn skip_dangerous_bin_locations() { "good": "./good", }, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/foo")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/pkg/foo")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "good"); } @@ -155,7 +155,7 @@ fn scoped_bin_name_strips_scope_prefix() { "version": "1.0.0", "bin": {"@foo/a": "./a"}, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "a"); } @@ -176,7 +176,7 @@ fn skip_scoped_bin_names_with_path_traversal() { "@scope/legit": "./good.js", }, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "legit"); } @@ -189,7 +189,7 @@ fn malformed_bin_type_returns_empty() { for shape in [json!(42), json!(["a", "b"]), json!(null), json!(true)] { let manifest = json!({"name": "x", "version": "1.0.0", "bin": shape}); assert!( - get_bins_from_package_manifest::(&manifest, Path::new("/p")).is_empty(), + get_bins_from_package_manifest::(&manifest, Path::new("/p")).is_empty(), "malformed bin shape must be tolerated", ); } @@ -202,7 +202,7 @@ fn malformed_bin_type_returns_empty() { #[test] fn bin_string_with_missing_package_name_returns_empty() { let manifest = json!({"bin": "cli.js"}); - assert!(get_bins_from_package_manifest::(&manifest, Path::new("/p")).is_empty()); + assert!(get_bins_from_package_manifest::(&manifest, Path::new("/p")).is_empty()); } /// Object-form bin entries whose values aren't strings (number, null, etc.) @@ -219,7 +219,7 @@ fn bin_object_with_non_string_value_is_skipped() { "bad-null": null, }, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "good"); } @@ -242,7 +242,7 @@ fn directories_bin_walks_files_recursively() { "version": "1.0.0", "directories": {"bin": "bin-dir"}, }); - let mut commands = get_bins_from_package_manifest::(&manifest, &pkg); + let mut commands = get_bins_from_package_manifest::(&manifest, &pkg); commands.sort_by(|a, b| a.name.cmp(&b.name)); assert_eq!(commands.len(), 2); assert_eq!(commands[0].name, "rootBin.js"); @@ -277,7 +277,7 @@ fn directories_bin_rejects_path_traversal() { "directories": {"bin": "../siblings"}, }); assert!( - get_bins_from_package_manifest::(&manifest, &pkg).is_empty(), + get_bins_from_package_manifest::(&manifest, &pkg).is_empty(), "is_subdir guard must reject `..`-escapes from the pkg root, even \ when the resolved directory exists and has files", ); @@ -306,7 +306,7 @@ fn directories_bin_rejects_real_path_traversal() { "version": "1.0.0", "directories": {"bin": "../secret"}, }); - assert!(get_bins_from_package_manifest::(&manifest, &pkg).is_empty()); + assert!(get_bins_from_package_manifest::(&manifest, &pkg).is_empty()); } /// `directories.bin` pointing at a non-existent subdirectory must @@ -321,7 +321,7 @@ fn directories_bin_missing_directory_returns_empty() { "version": "1.0.0", "directories": {"bin": "missing-dir"}, }); - assert!(get_bins_from_package_manifest::(&manifest, &pkg).is_empty()); + assert!(get_bins_from_package_manifest::(&manifest, &pkg).is_empty()); } /// `directories.bin` filters out files whose basename fails the @@ -342,7 +342,7 @@ fn directories_bin_filters_unsafe_file_names() { "version": "1.0.0", "directories": {"bin": "bin"}, }); - let mut commands = get_bins_from_package_manifest::(&manifest, &pkg); + let mut commands = get_bins_from_package_manifest::(&manifest, &pkg); commands.sort_by(|a, b| a.name.cmp(&b.name)); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "good"); @@ -357,7 +357,7 @@ fn empty_bin_key_is_rejected() { "version": "1.0.0", "bin": {"": "ok.js", "good": "ok.js"}, }); - let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); + let commands = get_bins_from_package_manifest::(&manifest, Path::new("/p")); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "good"); } @@ -413,7 +413,7 @@ fn directories_bin_accepts_excess_parent_dirs_that_resolve_inside_pkg() { "version": "1.0.0", "directories": {"bin": "x/../../pkg/bin-dir"}, }); - let commands = get_bins_from_package_manifest::(&manifest, &pkg); + let commands = get_bins_from_package_manifest::(&manifest, &pkg); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "cli"); } @@ -434,7 +434,7 @@ fn directories_bin_handles_curdir_in_relative_path() { "version": "1.0.0", "directories": {"bin": "./bin-dir"}, }); - let commands = get_bins_from_package_manifest::(&manifest, &pkg); + let commands = get_bins_from_package_manifest::(&manifest, &pkg); assert_eq!(commands.len(), 1); assert_eq!(commands[0].name, "cli"); } @@ -497,7 +497,7 @@ fn bin_field_takes_precedence_over_directories_bin() { "bin": "primary.js", "directories": {"bin": "legacy-bin"}, }); - let commands = get_bins_from_package_manifest::(&manifest, &pkg); + let commands = get_bins_from_package_manifest::(&manifest, &pkg); assert_eq!(commands.len(), 1, "bin field wins, directories.bin is ignored"); assert_eq!(commands[0].name, "tool"); } diff --git a/pacquet/crates/cmd-shim/src/capabilities.rs b/pacquet/crates/cmd-shim/src/capabilities.rs index c4d824e7ac..ab2130e773 100644 --- a/pacquet/crates/cmd-shim/src/capabilities.rs +++ b/pacquet/crates/cmd-shim/src/capabilities.rs @@ -1,5 +1,5 @@ //! Per-capability dependency-injection traits and the production -//! [`RealApi`] provider. Mirrors the pattern documented at +//! [`Host`] provider. Mirrors the pattern documented at //! : //! //! 1. One trait per capability. @@ -27,7 +27,7 @@ use std::{ /// generic over this trait, so test fakes do not have to grow. /// /// The trait makes no claim about how many syscalls a particular -/// impl will use — the production `RealApi` impl opens the file, +/// impl will use — the production `Host` impl opens the file, /// seeks to `offset` (if non-zero), and reads, which is more than /// one. What it does promise is the semantic contract: read up to /// `buf.len()` bytes starting at `offset` into `buf`. @@ -48,7 +48,7 @@ pub trait FsReadFile { /// Read the entire contents of a file into a `String`. Used by /// [`crate::link_bins_of_packages`] to short-circuit on warm reinstalls /// where the existing shim already targets the same bin file. -pub trait FsReadString { +pub trait FsReadToString { fn read_to_string(path: &Path) -> io::Result; } @@ -145,9 +145,9 @@ pub trait FsEnsureExecutableBits { /// The production filesystem provider. Every method delegates straight /// to `std::fs`. -pub struct RealApi; +pub struct Host; -impl FsReadHead for RealApi { +impl FsReadHead for Host { fn read_head(path: &Path, offset: u64, buf: &mut [u8]) -> io::Result { use std::io::{Read, Seek, SeekFrom}; let mut file = std::fs::File::open(path)?; @@ -158,19 +158,19 @@ impl FsReadHead for RealApi { } } -impl FsReadFile for RealApi { +impl FsReadFile for Host { fn read_file(path: &Path) -> io::Result> { std::fs::read(path) } } -impl FsReadString for RealApi { +impl FsReadToString for Host { fn read_to_string(path: &Path) -> io::Result { std::fs::read_to_string(path) } } -impl FsReadDir for RealApi { +impl FsReadDir for Host { fn read_dir(path: &Path) -> io::Result> { // `flatten()` silently drops per-entry errors. This matches the // prior collect-then-flatten shape and the `tinyglobby`-style @@ -179,7 +179,7 @@ impl FsReadDir for RealApi { } } -impl FsWalkFiles for RealApi { +impl FsWalkFiles for Host { fn walk_files(path: &Path) -> io::Result> { // `flatten()` silently drops per-entry errors and matches // pnpm's `tinyglobby` ENOENT-on-subtree behaviour. The @@ -196,20 +196,20 @@ impl FsWalkFiles for RealApi { } } -impl FsCreateDirAll for RealApi { +impl FsCreateDirAll for Host { fn create_dir_all(path: &Path) -> io::Result<()> { std::fs::create_dir_all(path) } } -impl FsWrite for RealApi { +impl FsWrite for Host { fn write(path: &Path, bytes: &[u8]) -> io::Result<()> { std::fs::write(path, bytes) } } #[cfg(unix)] -impl FsSetExecutable for RealApi { +impl FsSetExecutable for Host { fn set_executable(path: &Path) -> io::Result<()> { use std::os::unix::fs::PermissionsExt; std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o755)) @@ -217,14 +217,14 @@ impl FsSetExecutable for RealApi { } #[cfg(not(unix))] -impl FsSetExecutable for RealApi { +impl FsSetExecutable for Host { fn set_executable(_path: &Path) -> io::Result<()> { Ok(()) } } #[cfg(unix)] -impl FsEnsureExecutableBits for RealApi { +impl FsEnsureExecutableBits for Host { fn ensure_executable_bits(path: &Path) -> io::Result<()> { use std::os::unix::fs::PermissionsExt; let metadata = std::fs::metadata(path)?; @@ -234,7 +234,7 @@ impl FsEnsureExecutableBits for RealApi { } #[cfg(not(unix))] -impl FsEnsureExecutableBits for RealApi { +impl FsEnsureExecutableBits for Host { fn ensure_executable_bits(_path: &Path) -> io::Result<()> { Ok(()) } diff --git a/pacquet/crates/cmd-shim/src/link_bins.rs b/pacquet/crates/cmd-shim/src/link_bins.rs index da6ae5c6d3..46345244f6 100644 --- a/pacquet/crates/cmd-shim/src/link_bins.rs +++ b/pacquet/crates/cmd-shim/src/link_bins.rs @@ -1,7 +1,7 @@ use crate::{ bin_resolver::{Command, get_bins_from_package_manifest, pkg_owns_bin}, capabilities::{ - FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadString, + FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadToString, FsSetExecutable, FsWalkFiles, FsWrite, }, shim::{ @@ -187,11 +187,11 @@ pub enum LinkBinsError { /// /// Scoped packages are recursed: `node_modules/@scope/foo` becomes one /// candidate. This mirrors `binNamesAndPaths` in upstream `linkBins`. -pub fn link_bins(modules_dir: &Path, bins_dir: &Path) -> Result<(), LinkBinsError> +pub fn link_bins(modules_dir: &Path, bins_dir: &Path) -> Result<(), LinkBinsError> where - Api: FsReadDir + Sys: FsReadDir + FsReadFile - + FsReadString + + FsReadToString + FsReadHead + FsCreateDirAll + FsWalkFiles @@ -199,19 +199,19 @@ where + FsSetExecutable + FsEnsureExecutableBits, { - let packages = collect_packages_in_modules_dir::(modules_dir)?; - link_bins_of_packages::(&packages, bins_dir) + let packages = collect_packages_in_modules_dir::(modules_dir)?; + link_bins_of_packages::(&packages, bins_dir) } -fn collect_packages_in_modules_dir( +fn collect_packages_in_modules_dir( modules_dir: &Path, ) -> Result, LinkBinsError> where - Api: FsReadDir + FsReadFile, + Sys: FsReadDir + FsReadFile, { let mut packages = Vec::new(); - let entries = match Api::read_dir(modules_dir) { + let entries = match Sys::read_dir(modules_dir) { Ok(entries) => entries, Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(packages), Err(error) => { @@ -236,7 +236,7 @@ where // scope, so surface them as `ReadModulesDir`. Matches // the policy the per-`modules_dir` read above already // uses. - let scope_entries = match Api::read_dir(&path) { + let scope_entries = match Sys::read_dir(&path) { Ok(entries) => entries, Err(error) if error.kind() == io::ErrorKind::NotFound => continue, Err(error) => { @@ -244,14 +244,14 @@ where } }; for sub_path in scope_entries { - if let Some(pkg) = read_package::(&sub_path)? { + if let Some(pkg) = read_package::(&sub_path)? { packages.push(pkg); } } continue; } - if let Some(pkg) = read_package::(&path)? { + if let Some(pkg) = read_package::(&path)? { packages.push(pkg); } } @@ -259,11 +259,11 @@ where Ok(packages) } -fn read_package( +fn read_package( location: &Path, ) -> Result, LinkBinsError> { let manifest_path = location.join("package.json"); - let bytes = match Api::read_file(&manifest_path) { + let bytes = match Sys::read_file(&manifest_path) { Ok(bytes) => bytes, Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None), Err(error) => return Err(LinkBinsError::ReadManifest { path: manifest_path, error }), @@ -292,12 +292,12 @@ fn read_package( /// conflicts via semver (a feature upstream uses for hoisting), since the /// virtual-store layout means each bin source is a unique /// `(package, version)` slot already. -pub fn link_bins_of_packages( +pub fn link_bins_of_packages( packages: &[PackageBinSource], bins_dir: &Path, ) -> Result<(), LinkBinsError> where - Api: FsReadString + Sys: FsReadToString + FsReadHead + FsCreateDirAll + FsWalkFiles @@ -309,7 +309,7 @@ where for pkg in packages { let pkg_name = pkg.manifest.get("name").and_then(Value::as_str).unwrap_or(""); - let commands = get_bins_from_package_manifest::(&pkg.manifest, &pkg.location); + let commands = get_bins_from_package_manifest::(&pkg.manifest, &pkg.location); for command in commands { match chosen.get(&command.name) { None => { @@ -336,7 +336,7 @@ where return Ok(()); } - Api::create_dir_all(bins_dir) + Sys::create_dir_all(bins_dir) .map_err(|error| LinkBinsError::CreateBinDir { dir: bins_dir.to_path_buf(), error })?; // Each shim's read-shebang + write-file + chmod sequence is independent @@ -344,7 +344,7 @@ where // The hot path is per-package-bin; without parallelism the per-shim // file I/O serialised across the whole `chosen` map. chosen.par_iter().try_for_each(|(bin_name, (command, _pkg))| { - write_shim::(&command.path, &bins_dir.join(bin_name)) + write_shim::(&command.path, &bins_dir.join(bin_name)) })?; Ok(()) @@ -415,9 +415,9 @@ fn pick_winner( /// they are no-ops (Windows has no equivalent permission concept), so /// the call sites stay portable and don't need their own /// `#[cfg(unix)]` gating. -fn write_shim(target_path: &Path, shim_path: &Path) -> Result<(), LinkBinsError> +fn write_shim(target_path: &Path, shim_path: &Path) -> Result<(), LinkBinsError> where - Api: FsReadString + FsReadHead + FsWrite + FsSetExecutable + FsEnsureExecutableBits, + Sys: FsReadToString + FsReadHead + FsWrite + FsSetExecutable + FsEnsureExecutableBits, { // The node runtime binary is special: never wrap it in a shell // shim. Mirrors pnpm v11's `cmd.name === 'node'` short-circuit in @@ -444,7 +444,7 @@ where return Ok(()); } - let runtime = search_script_runtime::(target_path).map_err(|error| { + let runtime = search_script_runtime::(target_path).map_err(|error| { LinkBinsError::ProbeShimSource { path: target_path.to_path_buf(), error } })?; @@ -478,18 +478,18 @@ where // bodies are stable across pacquet versions (only the `` // segment moves), so byte equality is a sound equivalence check. let sh_marker_ok = matches!( - Api::read_to_string(shim_path), + Sys::read_to_string(shim_path), Ok(existing) if is_shim_pointing_at(&existing, target_path), ); let windows_ok = match &windows_shims { None => true, Some((cmd_path, cmd_body, ps1_path, ps1_body)) => { let cmd_ok = matches!( - Api::read_to_string(cmd_path), + Sys::read_to_string(cmd_path), Ok(existing) if &existing == cmd_body, ); let ps1_ok = matches!( - Api::read_to_string(ps1_path), + Sys::read_to_string(ps1_path), Ok(existing) if &existing == ps1_body, ); cmd_ok && ps1_ok @@ -498,17 +498,17 @@ where let already_correct = sh_marker_ok && windows_ok; if !already_correct { - Api::write(shim_path, sh_body.as_bytes()) + Sys::write(shim_path, sh_body.as_bytes()) .map_err(|error| LinkBinsError::WriteShim { path: shim_path.to_path_buf(), error })?; if let Some((cmd_path, cmd_body, ps1_path, ps1_body)) = &windows_shims { - Api::write(cmd_path, cmd_body.as_bytes()) + Sys::write(cmd_path, cmd_body.as_bytes()) .map_err(|error| LinkBinsError::WriteShim { path: cmd_path.clone(), error })?; - Api::write(ps1_path, ps1_body.as_bytes()) + Sys::write(ps1_path, ps1_body.as_bytes()) .map_err(|error| LinkBinsError::WriteShim { path: ps1_path.clone(), error })?; } } - Api::set_executable(shim_path) + Sys::set_executable(shim_path) .map_err(|error| LinkBinsError::Chmod { path: shim_path.to_path_buf(), error })?; // Make the underlying script executable too. pnpm calls // `fixBin(cmd.path, 0o755)` to do this; we apply the same minimum @@ -522,7 +522,7 @@ where // AppArmor deny, foreign uid) surfaces as `LinkBinsError::Chmod` // so real failures don't disappear silently. Mirrors pnpm's // `fixBin` ENOENT guard. - match Api::ensure_executable_bits(target_path) { + match Sys::ensure_executable_bits(target_path) { Ok(()) => {} Err(error) if error.kind() == io::ErrorKind::NotFound => {} Err(error) => { @@ -557,7 +557,7 @@ fn is_node_bin_name(shim_path: &Path) -> bool { /// …). The source must end in `.exe`; otherwise pnpm falls through /// to the cmd-shim path and so do we. /// -/// `remove_file` rather than `Api::write`-style truncation is +/// `remove_file` rather than `Sys::write`-style truncation is /// load-bearing on both platforms: if `shim_path` is currently a /// regular file hardlinked to the source binary (a state an earlier /// pacquet revision could leave behind), truncating through the diff --git a/pacquet/crates/cmd-shim/src/link_bins/tests.rs b/pacquet/crates/cmd-shim/src/link_bins/tests.rs index d7818a9aea..3439144fa6 100644 --- a/pacquet/crates/cmd-shim/src/link_bins/tests.rs +++ b/pacquet/crates/cmd-shim/src/link_bins/tests.rs @@ -1,8 +1,8 @@ use super::{BinOrigin, LinkBinsError, PackageBinSource, link_bins, link_bins_of_packages}; use crate::{ capabilities::{ - FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadString, - FsSetExecutable, FsWalkFiles, FsWrite, RealApi, + FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadToString, + FsSetExecutable, FsWalkFiles, FsWrite, Host, }, shim::is_shim_pointing_at, }; @@ -45,7 +45,7 @@ fn writes_shim_flavors_matching_host_platform() { let bins_dir = tmp.path().join("node_modules/.bin"); let manifest_value: Value = serde_json::from_slice(&read_file(pkg_dir.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::( + link_bins_of_packages::( &[PackageBinSource::new(pkg_dir.clone(), Arc::new(manifest_value))], &bins_dir, ) @@ -94,7 +94,7 @@ fn writes_shim_for_bin_string() { let bins_dir = tmp.path().join("node_modules/.bin"); let manifest_value: Value = serde_json::from_slice(&read_file(pkg_dir.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::( + link_bins_of_packages::( &[PackageBinSource::new(pkg_dir.clone(), Arc::new(manifest_value))], &bins_dir, ) @@ -122,7 +122,7 @@ fn writes_shim_for_bin_string() { } } -/// [`link_bins::`](link_bins) walks every package and its scoped +/// [`link_bins::`](link_bins) walks every package and its scoped /// children. Both regular and `@scope/...` packages must contribute their /// bins. #[test] @@ -146,7 +146,7 @@ fn link_bins_walks_modules_and_scopes() { create_dir_all(modules.join("not-a-package")).unwrap(); let bins = modules.join(".bin"); - link_bins::(&modules, &bins).unwrap(); + link_bins::(&modules, &bins).unwrap(); assert!(bins.join("foo").exists(), "foo shim must exist"); assert!(bins.join("bar").exists(), "scoped @s/bar shim must use bare name `bar`"); @@ -159,8 +159,7 @@ fn link_bins_walks_modules_and_scopes() { fn link_bins_handles_missing_modules_dir() { let tmp = tempdir().unwrap(); let bins_dir = tmp.path().join(".bin"); - link_bins::(&tmp.path().join("missing"), &bins_dir) - .expect("missing modules dir is Ok"); + link_bins::(&tmp.path().join("missing"), &bins_dir).expect("missing modules dir is Ok"); assert!(!bins_dir.exists(), "no shims means no bin dir created"); } @@ -176,7 +175,7 @@ fn link_bins_of_packages_no_op_when_no_bins() { let bins = tmp.path().join(".bin"); let manifest: Value = serde_json::from_slice(&read_file(pkg.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::(&[PackageBinSource::new(pkg, Arc::new(manifest))], &bins) + link_bins_of_packages::(&[PackageBinSource::new(pkg, Arc::new(manifest))], &bins) .unwrap(); assert!(!bins.exists(), "bins dir must not be created when nothing to link"); } @@ -212,7 +211,7 @@ fn lexical_compare_breaks_tie_when_neither_owns() { let bins = tmp.path().join(".bin"); // Order beta-then-alpha to verify the choice doesn't depend on // discovery order. - link_bins_of_packages::( + link_bins_of_packages::( &[ PackageBinSource::new(beta.clone(), Arc::new(manifest_beta)), PackageBinSource::new(alpha.clone(), Arc::new(manifest_alpha)), @@ -238,7 +237,7 @@ fn link_bins_propagates_parse_manifest_error() { write_file(modules.join("broken/package.json"), "{ this is not json").unwrap(); let bins = modules.join(".bin"); - let err = link_bins::(&modules, &bins).expect_err("invalid manifest must surface"); + let err = link_bins::(&modules, &bins).expect_err("invalid manifest must surface"); assert!( matches!(err, LinkBinsError::ParseManifest { .. }), "expected ParseManifest, got {err:?}", @@ -260,14 +259,14 @@ fn link_bins_skips_existing_shim_with_matching_marker() { write_file(modules.join("foo/f.js"), "#!/usr/bin/env node\n").unwrap(); let bins = modules.join(".bin"); - link_bins::(&modules, &bins).unwrap(); + link_bins::(&modules, &bins).unwrap(); let original = read_to_string(bins.join("foo")).unwrap(); // Append a sentinel. If the second pass rewrites the shim, the // sentinel disappears. let sentinel = format!("{original}\n# SENTINEL"); write_file(bins.join("foo"), &sentinel).unwrap(); - link_bins::(&modules, &bins).unwrap(); + link_bins::(&modules, &bins).unwrap(); assert_eq!(read_to_string(bins.join("foo")).unwrap(), sentinel); } @@ -294,7 +293,7 @@ fn link_bins_rewrites_when_only_canonical_flavor_exists() { write_file(modules.join("foo/f.js"), "#!/usr/bin/env node\n").unwrap(); let bins = modules.join(".bin"); - link_bins::(&modules, &bins).unwrap(); + link_bins::(&modules, &bins).unwrap(); // Simulate the partial-write / older-pacquet state: delete the // .cmd and .ps1 siblings, leaving only the canonical shim with its @@ -302,17 +301,17 @@ fn link_bins_rewrites_when_only_canonical_flavor_exists() { remove_file(bins.join("foo.cmd")).unwrap(); remove_file(bins.join("foo.ps1")).unwrap(); - link_bins::(&modules, &bins).unwrap(); + link_bins::(&modules, &bins).unwrap(); assert!(bins.join("foo").exists(), "canonical shim must remain"); assert!(bins.join("foo.cmd").exists(), ".cmd sibling must be re-created on second pass"); assert!(bins.join("foo.ps1").exists(), ".ps1 sibling must be re-created on second pass"); } -/// [`link_bins_of_packages`] propagates a non-`NotFound` `read_dir` -/// error from the calling context. Use a fake `Api` that fails the -/// initial `create_dir_all` to cover the [`LinkBinsError::CreateBinDir`] -/// error variant that real fs can't trigger portably. +/// [`link_bins_of_packages`] propagates a `create_dir_all` failure on +/// the destination bins directory as [`LinkBinsError::CreateBinDir`]. +/// Use a fake `Sys` that fails the initial `create_dir_all` to drive +/// the variant, since the real fs can't trigger it portably. #[test] fn link_bins_propagates_create_bin_dir_error_via_di() { use std::io; @@ -328,7 +327,7 @@ fn link_bins_propagates_create_bin_dir_error_via_di() { unreachable!("not called when chosen is empty") } } - impl FsReadString for FailingCreateDir { + impl FsReadToString for FailingCreateDir { fn read_to_string(_: &Path) -> io::Result { unreachable!() } @@ -397,7 +396,7 @@ fn link_bins_propagates_write_shim_error_via_di() { unreachable!() } } - impl FsReadString for FailingWrite { + impl FsReadToString for FailingWrite { fn read_to_string(_: &Path) -> io::Result { // Pretend no existing shim, forcing the writer path. Err(io::Error::from(io::ErrorKind::NotFound)) @@ -466,7 +465,7 @@ fn link_bins_propagates_chmod_error_via_di() { unreachable!() } } - impl FsReadString for FailingChmod { + impl FsReadToString for FailingChmod { fn read_to_string(_: &Path) -> io::Result { Err(io::Error::from(io::ErrorKind::NotFound)) } @@ -538,7 +537,7 @@ fn link_bins_propagates_target_chmod_error_via_di() { unreachable!() } } - impl FsReadString for FailingTargetChmod { + impl FsReadToString for FailingTargetChmod { fn read_to_string(_: &Path) -> io::Result { Err(io::Error::from(io::ErrorKind::NotFound)) } @@ -612,7 +611,7 @@ fn link_bins_swallows_target_chmod_not_found_via_di() { unreachable!() } } - impl FsReadString for NotFoundTargetChmod { + impl FsReadToString for NotFoundTargetChmod { fn read_to_string(_: &Path) -> io::Result { Err(io::Error::from(io::ErrorKind::NotFound)) } @@ -682,7 +681,7 @@ fn link_bins_propagates_probe_shim_source_error_via_di() { unreachable!() } } - impl FsReadString for FailingProbe { + impl FsReadToString for FailingProbe { fn read_to_string(_: &Path) -> io::Result { unreachable!() } @@ -751,7 +750,7 @@ fn link_bins_propagates_read_manifest_error_via_di() { Err(io::Error::from(io::ErrorKind::PermissionDenied)) } } - impl FsReadString for DenyManifestRead { + impl FsReadToString for DenyManifestRead { fn read_to_string(_: &Path) -> io::Result { unreachable!() } @@ -828,7 +827,7 @@ fn ownership_breaks_bin_conflicts_when_existing_owns() { // Order npm-first; this exercises the (true, false) arm because // `npm` (existing) owns and `aaa-other` (candidate) doesn't. let bins = tmp.path().join(".bin"); - link_bins_of_packages::( + link_bins_of_packages::( &[ PackageBinSource::new(npm.clone(), Arc::new(manifest_npm)), PackageBinSource::new(aaa_other.clone(), Arc::new(manifest_other)), @@ -859,7 +858,7 @@ fn link_bins_propagates_modules_dir_read_error_via_di() { unreachable!() } } - impl FsReadString for FailingModulesRead { + impl FsReadToString for FailingModulesRead { fn read_to_string(_: &Path) -> io::Result { unreachable!() } @@ -934,7 +933,7 @@ fn ownership_breaks_bin_conflicts() { serde_json::from_slice(&read_file(aaa_other.join("package.json")).unwrap()).unwrap(); let bins = tmp.path().join(".bin"); - link_bins_of_packages::( + link_bins_of_packages::( &[ PackageBinSource::new(aaa_other.clone(), Arc::new(manifest_other)), PackageBinSource::new(npm.clone(), Arc::new(manifest_npm)), @@ -988,7 +987,7 @@ fn direct_origin_wins_over_hoisted_regardless_of_lexical() { serde_json::from_slice(&read_file(direct.join("package.json")).unwrap()).unwrap(); let bins = tmp.path().join(".bin"); - link_bins_of_packages::( + link_bins_of_packages::( &[ PackageBinSource::new(hoisted.clone(), Arc::new(manifest_hoisted)) .with_origin(BinOrigin::Hoisted), @@ -1041,7 +1040,7 @@ fn hoisted_origin_loses_to_existing_direct() { let bins = tmp.path().join(".bin"); // Direct goes first so it's the incumbent when the Hoisted // candidate is processed second. - link_bins_of_packages::( + link_bins_of_packages::( &[ PackageBinSource::new(direct.clone(), Arc::new(manifest_direct)) .with_origin(BinOrigin::Direct), @@ -1083,7 +1082,7 @@ fn link_node_bin_symlinks_directly_instead_of_writing_shim() { let manifest: Value = serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::( + link_bins_of_packages::( &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], &bin_target, ) @@ -1138,7 +1137,7 @@ fn link_node_bin_replaces_dangling_symlink() { let manifest: Value = serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::( + link_bins_of_packages::( &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], &bin_target, ) @@ -1186,7 +1185,7 @@ fn link_node_bin_does_not_corrupt_hardlinked_target() { let manifest: Value = serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::( + link_bins_of_packages::( &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], &bin_target, ) @@ -1224,7 +1223,7 @@ fn link_node_bin_hardlinks_node_exe_on_windows() { let manifest: Value = serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::( + link_bins_of_packages::( &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], &bin_target, ) @@ -1267,7 +1266,7 @@ fn link_node_bin_falls_through_to_cmd_shim_when_source_is_not_exe() { let manifest: Value = serde_json::from_slice(&read_file(node_dir.join("package.json")).unwrap()).unwrap(); - link_bins_of_packages::( + link_bins_of_packages::( &[PackageBinSource::new(node_dir.clone(), Arc::new(manifest))], &bin_target, ) diff --git a/pacquet/crates/cmd-shim/src/shim.rs b/pacquet/crates/cmd-shim/src/shim.rs index 84e9651cde..9b021b0227 100644 --- a/pacquet/crates/cmd-shim/src/shim.rs +++ b/pacquet/crates/cmd-shim/src/shim.rs @@ -44,10 +44,10 @@ fn extension_program(extension: &str) -> Option<&'static str> { /// doesn't fail the whole install. Other IO errors propagate, since pacquet /// has already verified the bin path resolves under the package root by /// this point and a real failure deserves to surface. -pub fn search_script_runtime(path: &Path) -> io::Result> { +pub fn search_script_runtime(path: &Path) -> io::Result> { let extension = path.extension().and_then(|s| s.to_str()).unwrap_or(""); - let runtime_from_shebang = read_shebang::(path)?; + let runtime_from_shebang = read_shebang::(path)?; if let Some(rt) = runtime_from_shebang { return Ok(Some(rt)); } @@ -59,9 +59,9 @@ pub fn search_script_runtime(path: &Path) -> io::Result(path: &Path) -> io::Result> { +fn read_shebang(path: &Path) -> io::Result> { let mut buffer = [0u8; 512]; - let read = match read_head_filled::(path, &mut buffer) { + let read = match read_head_filled::(path, &mut buffer) { Ok(read) => read, Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None), Err(error) => return Err(error), @@ -85,10 +85,10 @@ fn read_shebang(path: &Path) -> io::Result(path: &Path, buf: &mut [u8]) -> io::Result { +pub fn read_head_filled(path: &Path, buf: &mut [u8]) -> io::Result { let mut total = 0; while total < buf.len() { - match Api::read_head(path, total as u64, &mut buf[total..])? { + match Sys::read_head(path, total as u64, &mut buf[total..])? { 0 => break, // EOF n => total += n, } diff --git a/pacquet/crates/cmd-shim/src/shim/tests.rs b/pacquet/crates/cmd-shim/src/shim/tests.rs index e331dd8d93..f779d67450 100644 --- a/pacquet/crates/cmd-shim/src/shim/tests.rs +++ b/pacquet/crates/cmd-shim/src/shim/tests.rs @@ -3,7 +3,7 @@ use super::{ is_shim_pointing_at, parse_shebang, parse_shebang_from_bytes, read_head_filled, relative_target, search_script_runtime, }; -use crate::capabilities::{FsReadHead, RealApi}; +use crate::capabilities::{FsReadHead, Host}; use crate::path_util::lexical_normalize; use std::{ io, @@ -250,7 +250,7 @@ fn search_script_runtime_reads_shebang_from_real_file() { let tmp = tempdir().unwrap(); let path = tmp.path().join("script"); std::fs::write(&path, "#!/usr/bin/env node\nbody\n").unwrap(); - let rt = search_script_runtime::(&path).unwrap().expect("runtime detected"); + let rt = search_script_runtime::(&path).unwrap().expect("runtime detected"); assert_eq!(rt.prog.as_deref(), Some("node")); } @@ -259,7 +259,7 @@ fn search_script_runtime_reads_shebang_from_real_file() { #[test] fn search_script_runtime_returns_none_for_missing_file() { let nonexistent = Path::new("/definitely/not/a/real/path/cli"); - assert_eq!(search_script_runtime::(nonexistent).unwrap(), None); + assert_eq!(search_script_runtime::(nonexistent).unwrap(), None); } /// [`search_script_runtime`] falls through to extension lookup when the @@ -271,7 +271,7 @@ fn search_script_runtime_falls_back_to_extension() { let tmp = tempdir().unwrap(); let path = tmp.path().join("script.js"); std::fs::write(&path, "console.log('no shebang')\n").unwrap(); - let rt = search_script_runtime::(&path).unwrap().expect("extension fallback"); + let rt = search_script_runtime::(&path).unwrap().expect("extension fallback"); assert_eq!(rt.prog.as_deref(), Some("node")); } @@ -283,7 +283,7 @@ fn search_script_runtime_returns_none_when_runtime_unknown() { let tmp = tempdir().unwrap(); let path = tmp.path().join("script.unknown_ext"); std::fs::write(&path, "no shebang here\n").unwrap(); - assert_eq!(search_script_runtime::(&path).unwrap(), None); + assert_eq!(search_script_runtime::(&path).unwrap(), None); } /// [`search_script_runtime`] propagates IO errors that aren't `NotFound`. @@ -292,13 +292,13 @@ fn search_script_runtime_returns_none_when_runtime_unknown() { /// . #[test] fn search_script_runtime_propagates_non_not_found_io_errors() { - struct PermissionDeniedApi; - impl FsReadHead for PermissionDeniedApi { + struct PermissionDenied; + impl FsReadHead for PermissionDenied { fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result { Err(io::Error::from(io::ErrorKind::PermissionDenied)) } } - let err = search_script_runtime::(Path::new("any")) + let err = search_script_runtime::(Path::new("any")) .expect_err("non-NotFound IO error must propagate"); assert_eq!(err.kind(), io::ErrorKind::PermissionDenied); } @@ -309,23 +309,22 @@ fn search_script_runtime_propagates_non_not_found_io_errors() { /// compatible with the no-shebang case. #[test] fn search_script_runtime_reads_zero_bytes_then_falls_through() { - struct EmptyReadApi; - impl FsReadHead for EmptyReadApi { + struct EmptyRead; + impl FsReadHead for EmptyRead { fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result { Ok(0) } } // `.js` extension still resolves to `node` even with empty content. - let rt = - search_script_runtime::(Path::new("/x.js")).unwrap().expect("ext fallback"); + let rt = search_script_runtime::(Path::new("/x.js")).unwrap().expect("ext fallback"); assert_eq!(rt.prog.as_deref(), Some("node")); // No extension and no shebang → Ok(None). - let rt = search_script_runtime::(Path::new("/x")).unwrap(); + let rt = search_script_runtime::(Path::new("/x")).unwrap(); assert_eq!(rt, None); } -/// [`RealApi::read_head`](RealApi) is the production capability. Tests +/// [`Host::read_head`](Host) is the production capability. Tests /// that exercise it indirectly cover most paths; this one pins the /// contract directly. #[test] @@ -335,18 +334,18 @@ fn real_fs_read_head_reads_up_to_buffer_size() { let path = tmp.path().join("data"); std::fs::write(&path, "hello world").unwrap(); let mut buf = [0u8; 1024]; - let read = RealApi::read_head(&path, 0, &mut buf).unwrap(); + let read = Host::read_head(&path, 0, &mut buf).unwrap(); assert_eq!(read, 11); assert_eq!(&buf[..read], b"hello world"); } -/// [`RealApi::read_head`](RealApi) propagates `NotFound` so the shebang reader can +/// [`Host::read_head`](Host) propagates `NotFound` so the shebang reader can /// distinguish a missing file from a real IO error and degrade to /// `Ok(None)`. #[test] fn real_fs_read_head_propagates_not_found() { let mut buf = [0u8; 16]; - let err = RealApi::read_head(Path::new("/no/such/file"), 0, &mut buf).unwrap_err(); + let err = Host::read_head(Path::new("/no/such/file"), 0, &mut buf).unwrap_err(); assert_eq!(err.kind(), io::ErrorKind::NotFound); } @@ -362,7 +361,7 @@ fn read_head_filled_real_fs_long_file_fills_buffer() { std::fs::write(&path, &payload).unwrap(); let mut buf = [0u8; 256]; - let read = read_head_filled::(&path, &mut buf).unwrap(); + let read = read_head_filled::(&path, &mut buf).unwrap(); assert_eq!(read, 256); assert_eq!(&buf[..], &payload[..256]); } @@ -377,7 +376,7 @@ fn read_head_filled_real_fs_short_file_returns_partial() { std::fs::write(&path, "#!/bin/sh\n").unwrap(); let mut buf = [0u8; 256]; - let read = read_head_filled::(&path, &mut buf).unwrap(); + let read = read_head_filled::(&path, &mut buf).unwrap(); assert_eq!(read, 10); assert_eq!(&buf[..read], b"#!/bin/sh\n"); } diff --git a/pacquet/crates/config/src/api.rs b/pacquet/crates/config/src/api.rs index a58669a043..5c7a477b91 100644 --- a/pacquet/crates/config/src/api.rs +++ b/pacquet/crates/config/src/api.rs @@ -1,22 +1,22 @@ -//! Capability traits and the project-wide [`RealApi`] provider. +//! Capability traits and the [`Host`] provider for this crate. //! -//! Mirrors the dependency-injection pattern documented in -//! [pnpm/pacquet#339](https://github.com/pnpm/pacquet/issues/339): one -//! trait per capability, one provider gathering every capability impl -//! used across the codebase, all methods static. Production callers -//! turbofish the real provider explicitly -//! (e.g. `Config::current::(...)`); tests substitute a -//! per-test unit struct that implements only the bounds the function -//! actually declares, with any per-test scenario data stored in a -//! `static` inside the test fn. +//! Each crate that needs to thread a side-effecting capability through +//! a generic seam declares its own capability traits and its own +//! `Host` provider; this is the one for `pacquet-config`. Production +//! callers turbofish the real provider explicitly +//! (e.g. `Config::current::(...)`); tests substitute a per-test +//! unit struct that implements only the bounds the function actually +//! declares, with any per-test scenario data stored in a `static` +//! inside the test fn. //! -//! Today the provider only exposes [`EnvVar`]. As more side-effecting -//! capabilities are introduced (filesystem, disk inspection, time, -//! …) their `impl … for RealApi` blocks land here too, so callers -//! never juggle multiple providers. Trait names keep their domain -//! prefix (`Fs*`, `GetDisk*`, `Env*`, …) so a reader can identify -//! which domain a generic bound belongs to without chasing -//! definitions. +//! Today this provider only exposes [`EnvVar`]. As more side-effecting +//! capabilities are introduced into `pacquet-config` (filesystem reads +//! for `.npmrc`, network probes for auth, …) their `impl … for Host` +//! blocks land here too. Trait names keep their domain prefix (`Fs*`, +//! `GetDisk*`, `Env*`, …) so a reader can identify which domain a +//! generic bound belongs to without chasing definitions. See the +//! [Dependency injection for tests](../../../CODE_STYLE_GUIDE.md#dependency-injection-for-tests) +//! section of the style guide for the full convention. /// Capability: read a process environment variable. /// @@ -34,18 +34,19 @@ pub trait EnvVar { fn var(name: &str) -> Option; } -/// Project-wide capability provider. Production code threads -/// `RealApi` through generic call sites with an explicit turbofish: +/// Production provider for the capability traits in this crate. +/// Production code threads `Host` through generic call sites with an +/// explicit turbofish: /// /// ```ignore -/// let config = Config::current::(env::current_dir, home::home_dir, Default::default); +/// let config = Config::current::(env::current_dir, home::home_dir, Default::default); /// ``` /// /// Tests substitute their own zero-sized struct that implements only /// the trait bounds the function under test declares. -pub struct RealApi; +pub struct Host; -impl EnvVar for RealApi { +impl EnvVar for Host { fn var(name: &str) -> Option { std::env::var(name).ok() } diff --git a/pacquet/crates/config/src/env_replace.rs b/pacquet/crates/config/src/env_replace.rs index b377894e28..3ad0266e9d 100644 --- a/pacquet/crates/config/src/env_replace.rs +++ b/pacquet/crates/config/src/env_replace.rs @@ -22,7 +22,7 @@ //! pnpm's behaviour even though plain shell `${VAR:-default}` would //! also use the default for the empty case. //! -//! Production callers thread `RealApi` (which delegates to +//! Production callers thread `Host` (which delegates to //! `std::env::var`) through the turbofish slot. Tests provide their //! own per-test unit struct, per the DI pattern from //! [pnpm/pacquet#339](https://github.com/pnpm/pacquet/issues/339). @@ -30,7 +30,7 @@ use crate::api::EnvVar; /// Replace every `${VAR}` (or `${VAR:-default}`) placeholder in `text` with -/// the value [`Api::var`] returns. Placeholders that have no value and no +/// the value [`Sys::var`] returns. Placeholders that have no value and no /// default become `""` (the literal `${...}` never reaches the caller) and /// are recorded in the returned `Vec` so the caller can surface each one as /// a warning. @@ -43,8 +43,8 @@ use crate::api::EnvVar; /// same string still expand normally — only the unresolved bare ones are /// dropped to `""`. /// -/// [`Api::var`]: EnvVar::var -pub(crate) fn env_replace_lossy(text: &str) -> (String, Vec) { +/// [`Sys::var`]: EnvVar::var +pub(crate) fn env_replace_lossy(text: &str) -> (String, Vec) { let bytes = text.as_bytes(); let mut output = String::with_capacity(text.len()); let mut unresolved = Vec::new(); @@ -92,7 +92,7 @@ pub(crate) fn env_replace_lossy(text: &str) -> (String, Vec Some(separator) => (&inside[..separator], Some(&inside[separator + 2..])), None => (inside, None), }; - let value = Api::var(var_name).filter(|value| !value.is_empty()); + let value = Sys::var(var_name).filter(|value| !value.is_empty()); match (value, default) { (Some(value), _) => output.push_str(&value), (None, Some(default)) => output.push_str(default), @@ -140,8 +140,8 @@ mod tests { /// Run [`env_replace_lossy`] and assert no placeholder went unresolved. /// Used by tests that exercise paths where every placeholder must expand. - fn replace_clean(text: &str) -> String { - let (value, unresolved) = env_replace_lossy::(text); + fn replace_clean(text: &str) -> String { + let (value, unresolved) = env_replace_lossy::(text); assert_eq!(unresolved, Vec::::new(), "unexpected unresolved placeholders"); value } diff --git a/pacquet/crates/config/src/lib.rs b/pacquet/crates/config/src/lib.rs index e161151257..bd59b1deb9 100644 --- a/pacquet/crates/config/src/lib.rs +++ b/pacquet/crates/config/src/lib.rs @@ -5,7 +5,7 @@ pub mod matcher; mod npmrc_auth; mod workspace_yaml; -pub use crate::api::{EnvVar, RealApi}; +pub use crate::api::{EnvVar, Host}; use indexmap::IndexMap; use pacquet_patching::{PatchGroupRecord, ResolvePatchedDependenciesError, resolve_and_group}; @@ -767,13 +767,13 @@ impl Config { /// `pnpm-workspace.yaml` cannot be read or parsed, matching pnpm's /// [`readWorkspaceManifest`](https://github.com/pnpm/pnpm/blob/8eb1be4988/workspace/workspace-manifest-reader/src/index.ts). /// A missing file is not an error. - pub fn current( + pub fn current( current_dir: CurrentDir, home_dir: HomeDir, default: Default, ) -> Result where - Api: EnvVar, + Sys: EnvVar, CurrentDir: FnOnce() -> Result, HomeDir: FnOnce() -> Option, Default: FnOnce() -> Config, @@ -811,7 +811,7 @@ impl Config { .and_then(|dir| read_npmrc(dir)) .or_else(|| home_dir().and_then(|dir| read_npmrc(&dir))); let mut npmrc_auth = auth_source - .map(|text| crate::npmrc_auth::NpmrcAuth::from_ini::(&text)) + .map(|text| crate::npmrc_auth::NpmrcAuth::from_ini::(&text)) .unwrap_or_default(); npmrc_auth.apply_registry_and_warn(&mut config); // Proxy cascade fires unconditionally — even when no `.npmrc` @@ -819,7 +819,7 @@ impl Config { // [`config/reader/src/index.ts:591-600`](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/index.ts#L591-L600) // is a normalization step on the resolved config, not a // function of `.npmrc` presence. - npmrc_auth.apply_proxy_cascade::(&mut config); + npmrc_auth.apply_proxy_cascade::(&mut config); // TLS + local-address are sourced from `.npmrc` only — pnpm // does not honor env vars (`NODE_EXTRA_CA_CERTS`, // `NODE_TLS_REJECT_UNAUTHORIZED`, etc.) for these keys @@ -952,7 +952,7 @@ mod tests { use pretty_assertions::assert_eq; use tempfile::tempdir; - use super::{Config, NodeLinker, PackageImportMethod, RealApi, fs}; + use super::{Config, Host, NodeLinker, PackageImportMethod, fs}; use crate::defaults::default_store_dir; use pacquet_store_dir::StoreDir; use pacquet_testing_utils::env_guard::EnvGuard; @@ -1024,7 +1024,7 @@ mod tests { let tmp = tempdir().unwrap(); fs::write(tmp.path().join(".npmrc"), "registry=https://cwd.example") .expect("write to .npmrc"); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || unreachable!("shouldn't reach home dir"), Config::new, @@ -1042,7 +1042,7 @@ mod tests { let non_auth_ini = "symlink=false\nlockfile=true\nhoist=false\nnode-linker=hoisted\n"; fs::write(tmp.path().join(".npmrc"), non_auth_ini).expect("write to .npmrc"); let defaults = Config::new(); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1066,7 +1066,7 @@ mod tests { let ini = "fetch-retries=99\nfetch-retry-factor=99\nfetch-retry-mintimeout=99\nfetch-retry-maxtimeout=99\n"; fs::write(tmp.path().join(".npmrc"), ini).expect("write to .npmrc"); let defaults = Config::new(); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1083,7 +1083,7 @@ mod tests { let tmp = tempdir().unwrap(); // write invalid utf-8 value to npmrc fs::write(tmp.path().join(".npmrc"), b"Hello \xff World").expect("write to .npmrc"); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1098,7 +1098,7 @@ mod tests { let home_dir = tempdir().unwrap(); fs::write(home_dir.path().join(".npmrc"), "registry=https://home.example") .expect("write to .npmrc"); - let config = Config::current::( + let config = Config::current::( || current_dir.path().to_path_buf().pipe(Ok::<_, ()>), || home_dir.path().to_path_buf().pipe(Some), Config::new, @@ -1117,7 +1117,7 @@ mod tests { .expect("write to .npmrc"); fs::write(tmp.path().join("pnpm-workspace.yaml"), "registry: https://from-yaml.test\n") .expect("write to pnpm-workspace.yaml"); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || unreachable!("shouldn't reach home dir"), Config::new, @@ -1135,7 +1135,7 @@ mod tests { .expect("write to pnpm-workspace.yaml"); // No `.npmrc` anywhere, but a parent dir has `pnpm-workspace.yaml` — // the yaml should still be applied. - let config = Config::current::( + let config = Config::current::( || nested.clone().pipe(Ok::<_, ()>), || None, Config::new, @@ -1148,7 +1148,7 @@ mod tests { pub fn test_current_folder_fallback_to_default() { let current_dir = tempdir().unwrap(); let home_dir = tempdir().unwrap(); - let config = Config::current::( + let config = Config::current::( || current_dir.path().to_path_buf().pipe(Ok::<_, ()>), || home_dir.path().to_path_buf().pipe(Some), || Config { symlink: false, ..Config::new() }, @@ -1173,7 +1173,7 @@ mod tests { #[test] pub fn gvs_default_is_off_and_paths_derive_cleanly() { let tmp = tempdir().unwrap(); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1201,7 +1201,7 @@ mod tests { let tmp = tempdir().unwrap(); fs::write(tmp.path().join("pnpm-workspace.yaml"), "enableGlobalVirtualStore: false\n") .expect("write to pnpm-workspace.yaml"); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1226,7 +1226,7 @@ mod tests { format!("enableGlobalVirtualStore: true\nvirtualStoreDir: {}\n", user_path.display()), ) .expect("write to pnpm-workspace.yaml"); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1264,7 +1264,7 @@ mod tests { ), ) .expect("write to pnpm-workspace.yaml"); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1284,7 +1284,7 @@ mod tests { /// fallback through `Config::current`. The injected-`EnvVar` tests /// in `npmrc_auth/tests.rs` cover the cascade branches /// exhaustively; this one only proves the wiring through - /// `RealApi::var` reaches `std::env::var` and that the cascade + /// `Host::var` reaches `std::env::var` and that the cascade /// fires even with no `.npmrc` present. #[test] pub fn proxy_env_fallback_applies_through_current() { @@ -1320,7 +1320,7 @@ mod tests { env::remove_var("npm_config_workspace_dir"); env::set_var("HTTPS_PROXY", "http://env.example:8080"); } - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1344,7 +1344,7 @@ mod tests { // `: : :` is rejected by saphyr. fs::write(tmp.path().join("pnpm-workspace.yaml"), ": : :\n") .expect("write to pnpm-workspace.yaml"); - let result = Config::current::( + let result = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1373,7 +1373,7 @@ mod tests { fs::write(workspace_root.join("pnpm-workspace.yaml"), "packages:\n - packages/*\n") .expect("write to pnpm-workspace.yaml"); - let config = Config::current::( + let config = Config::current::( || subdir.clone().pipe(Ok::<_, ()>), || None, Config::new, @@ -1410,7 +1410,7 @@ mod tests { env::remove_var("npm_config_workspace_dir"); } let tmp = tempdir().unwrap(); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1451,7 +1451,7 @@ mod tests { env::set_var("NPM_CONFIG_WORKSPACE_DIR", env_workspace.path()); } - let config = Config::current::( + let config = Config::current::( || cwd_dir.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, @@ -1487,7 +1487,7 @@ mod tests { env::set_var("npm_config_workspace_dir", ""); } let tmp = tempdir().unwrap(); - let config = Config::current::( + let config = Config::current::( || tmp.path().to_path_buf().pipe(Ok::<_, ()>), || None, Config::new, diff --git a/pacquet/crates/config/src/npmrc_auth.rs b/pacquet/crates/config/src/npmrc_auth.rs index 262dbc3f60..9bd141e591 100644 --- a/pacquet/crates/config/src/npmrc_auth.rs +++ b/pacquet/crates/config/src/npmrc_auth.rs @@ -143,7 +143,7 @@ impl NpmrcAuth { /// plus comments starting with `;` or `#`. We hand-parse rather than /// use a strongly-typed deserializer so unknown / malformed keys don't /// blow up parsing. - pub fn from_ini(text: &str) -> Self { + pub fn from_ini(text: &str) -> Self { let mut auth = NpmrcAuth::default(); for line in text.lines() { let line = line.trim(); @@ -159,8 +159,8 @@ impl NpmrcAuth { // Apply ${VAR} substitution to both the key and the value, // matching `readAndFilterNpmrc` in pnpm's `loadNpmrcFiles.ts`. // Unresolved placeholders become "" and are recorded as warnings. - let (key, key_unresolved) = env_replace_lossy::(raw_key); - let (value, value_unresolved) = env_replace_lossy::(raw_value); + let (key, key_unresolved) = env_replace_lossy::(raw_key); + let (value, value_unresolved) = env_replace_lossy::(raw_value); for placeholder in key_unresolved.into_iter().chain(value_unresolved) { auth.warnings.push(format!("Failed to replace env in config: {placeholder}")); } @@ -308,31 +308,31 @@ impl NpmrcAuth { /// Generic over [`EnvVar`] so cascade tests can drive every branch /// without mutating the process environment (no `EnvGuard` global /// lock). - pub fn apply_proxy_cascade(&mut self, config: &mut Config) { + pub fn apply_proxy_cascade(&mut self, config: &mut Config) { // Upstream's `getProcessEnv` tries literal-, upper-, and // lower-case in order (config/reader/src/index.ts:689-693). For // the proxy var names below the literal form is already either // fully upper or fully lower, so the triple collapses to two // real attempts. - fn env_pair(upper: &str, lower: &str) -> Option { - Api::var(upper).or_else(|| Api::var(lower)) + fn env_pair(upper: &str, lower: &str) -> Option { + Sys::var(upper).or_else(|| Sys::var(lower)) } config.proxy.https_proxy = self .https_proxy .take() .or_else(|| self.legacy_proxy.clone()) - .or_else(|| env_pair::("HTTPS_PROXY", "https_proxy")); + .or_else(|| env_pair::("HTTPS_PROXY", "https_proxy")); config.proxy.http_proxy = self .http_proxy .take() .or_else(|| config.proxy.https_proxy.clone()) - .or_else(|| env_pair::("HTTP_PROXY", "http_proxy")) - .or_else(|| env_pair::("PROXY", "proxy")); + .or_else(|| env_pair::("HTTP_PROXY", "http_proxy")) + .or_else(|| env_pair::("PROXY", "proxy")); config.proxy.no_proxy = self .no_proxy .take() - .or_else(|| env_pair::("NO_PROXY", "no_proxy")) + .or_else(|| env_pair::("NO_PROXY", "no_proxy")) .map(|raw| parse_no_proxy(&raw)); } @@ -391,9 +391,9 @@ impl NpmrcAuth { /// [`apply_proxy_cascade`]: NpmrcAuth::apply_proxy_cascade /// [`build_auth_headers`]: NpmrcAuth::build_auth_headers #[cfg(test)] - pub fn apply_to(mut self, config: &mut Config) { + pub fn apply_to(mut self, config: &mut Config) { self.apply_registry_and_warn(config); - self.apply_proxy_cascade::(config); + self.apply_proxy_cascade::(config); self.apply_tls_and_local_address(config); self.build_auth_headers(config); } diff --git a/pacquet/crates/modules-yaml/src/lib.rs b/pacquet/crates/modules-yaml/src/lib.rs index f67a737b86..66ccd6cec3 100644 --- a/pacquet/crates/modules-yaml/src/lib.rs +++ b/pacquet/crates/modules-yaml/src/lib.rs @@ -65,30 +65,30 @@ pub trait Clock { } /// Production implementation, backed by [`std::fs`] and [`SystemTime::now`]. -pub struct RealApi; +pub struct Host; -impl FsReadToString for RealApi { +impl FsReadToString for Host { #[inline] fn read_to_string(path: &Path) -> io::Result { fs::read_to_string(path) } } -impl FsCreateDirAll for RealApi { +impl FsCreateDirAll for Host { #[inline] fn create_dir_all(path: &Path) -> io::Result<()> { fs::create_dir_all(path) } } -impl FsWrite for RealApi { +impl FsWrite for Host { #[inline] fn write(path: &Path, contents: &[u8]) -> io::Result<()> { fs::write(path, contents) } } -impl Clock for RealApi { +impl Clock for Host { #[inline] fn now() -> SystemTime { SystemTime::now() @@ -362,16 +362,16 @@ pub enum WriteModulesError { /// `null` document, matching upstream `readModules` at /// . /// -/// Production callers turbofish [`RealApi`]: `read_modules_manifest::(dir)`. +/// Production callers turbofish [`Host`]: `read_modules_manifest::(dir)`. /// The bounds list the minimal capabilities ([`FsReadToString`] + /// [`Clock`]) so test fakes only need to implement the methods that are /// actually called. -pub fn read_modules_manifest(modules_dir: &Path) -> Result, ReadModulesError> +pub fn read_modules_manifest(modules_dir: &Path) -> Result, ReadModulesError> where - Api: FsReadToString + Clock, + Sys: FsReadToString + Clock, { let manifest_path = modules_dir.join(MODULES_FILENAME); - let content = match Api::read_to_string(&manifest_path) { + let content = match Sys::read_to_string(&manifest_path) { Ok(content) => content, Err(source) if source.kind() == io::ErrorKind::NotFound => return Ok(None), Err(source) => { @@ -386,7 +386,7 @@ where apply_legacy_shamefully_hoist(&mut manifest); resolve_virtual_store_dir(&mut manifest, modules_dir); if manifest.pruned_at.is_empty() { - manifest.pruned_at = httpdate::fmt_http_date(Api::now()); + manifest.pruned_at = httpdate::fmt_http_date(Sys::now()); } if manifest.virtual_store_dir_max_length == 0 { manifest.virtual_store_dir_max_length = DEFAULT_VIRTUAL_STORE_DIR_MAX_LENGTH; @@ -407,14 +407,14 @@ where /// `clone()` inside the function. Per the CODE_STYLE_GUIDE rule that /// owned-vs-borrowed parameter choice should minimize copies. /// -/// Production callers turbofish [`RealApi`]: `write_modules_manifest::(dir, m)`. +/// Production callers turbofish [`Host`]: `write_modules_manifest::(dir, m)`. /// Bounds are minimal: only [`FsCreateDirAll`] and [`FsWrite`] are required. -pub fn write_modules_manifest( +pub fn write_modules_manifest( modules_dir: &Path, mut manifest: Modules, ) -> Result<(), WriteModulesError> where - Api: FsCreateDirAll + FsWrite, + Sys: FsCreateDirAll + FsWrite, { manifest.skipped.sort(); drop_legacy_hoisted_aliases_when_unreferenced(&mut manifest); @@ -426,12 +426,12 @@ where } let serialized = serde_json::to_string_pretty(&manifest).map_err(WriteModulesError::SerializeJson)?; - Api::create_dir_all(modules_dir).map_err(|source| WriteModulesError::CreateDir { + Sys::create_dir_all(modules_dir).map_err(|source| WriteModulesError::CreateDir { path: modules_dir.to_path_buf(), source, })?; let manifest_path = modules_dir.join(MODULES_FILENAME); - Api::write(&manifest_path, serialized.as_bytes()) + Sys::write(&manifest_path, serialized.as_bytes()) .map_err(|source| WriteModulesError::WriteFile { path: manifest_path, source }) } diff --git a/pacquet/crates/modules-yaml/tests/index.rs b/pacquet/crates/modules-yaml/tests/index.rs index a65b647f9d..4a7a49365f 100644 --- a/pacquet/crates/modules-yaml/tests/index.rs +++ b/pacquet/crates/modules-yaml/tests/index.rs @@ -5,7 +5,7 @@ //! sibling files (`real_fs.rs`, `fakes.rs`). use pacquet_modules_yaml::{ - HoistKind, Modules, RealApi, read_modules_manifest, write_modules_manifest, + HoistKind, Host, Modules, read_modules_manifest, write_modules_manifest, }; use pipe_trait::Pipe; use pretty_assertions::assert_eq; @@ -44,8 +44,8 @@ fn write_modules_manifest_and_read_modules_manifest() { "virtualStoreDirMaxLength": 120, })); - write_modules_manifest::(modules_dir, modules_yaml.clone()).expect("write manifest"); - let actual = read_modules_manifest::(modules_dir).expect("read manifest"); + write_modules_manifest::(modules_dir, modules_yaml.clone()).expect("write manifest"); + let actual = read_modules_manifest::(modules_dir).expect("read manifest"); assert_eq!(actual, Some(modules_yaml)); let raw: Value = modules_dir @@ -69,7 +69,7 @@ fn read_legacy_shamefully_hoist_true_manifest() { let manifest = env!("CARGO_MANIFEST_DIR") .pipe(Path::new) .join("tests/fixtures/old-shamefully-hoist") - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read manifest") .expect("modules manifest exists"); @@ -99,7 +99,7 @@ fn read_legacy_shamefully_hoist_false_manifest() { let manifest = env!("CARGO_MANIFEST_DIR") .pipe(Path::new) .join("tests/fixtures/old-no-shamefully-hoist") - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read manifest") .expect("modules manifest exists"); @@ -151,8 +151,8 @@ fn write_modules_manifest_creates_node_modules_directory() { "virtualStoreDirMaxLength": 120, })); - write_modules_manifest::(&modules_dir, modules_yaml.clone()).expect("write manifest"); - let actual = read_modules_manifest::(&modules_dir).expect("read manifest"); + write_modules_manifest::(&modules_dir, modules_yaml.clone()).expect("write manifest"); + let actual = read_modules_manifest::(&modules_dir).expect("read manifest"); assert_eq!(actual, Some(modules_yaml)); } @@ -162,7 +162,7 @@ fn read_empty_modules_manifest_returns_none() { let modules_yaml = env!("CARGO_MANIFEST_DIR") .pipe(Path::new) .join("tests/fixtures/empty-modules-yaml") - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read manifest"); assert_eq!(modules_yaml, None); } diff --git a/pacquet/crates/modules-yaml/tests/real_fs.rs b/pacquet/crates/modules-yaml/tests/real_fs.rs index 1b2b8ccb93..64a63021ac 100644 --- a/pacquet/crates/modules-yaml/tests/real_fs.rs +++ b/pacquet/crates/modules-yaml/tests/real_fs.rs @@ -8,9 +8,7 @@ //! ported, so these direct unit tests guard the behavior in the meantime. use indexmap::IndexSet; -use pacquet_modules_yaml::{ - DepPath, Modules, RealApi, read_modules_manifest, write_modules_manifest, -}; +use pacquet_modules_yaml::{DepPath, Host, Modules, read_modules_manifest, write_modules_manifest}; use pipe_trait::Pipe; use pretty_assertions::assert_eq; use serde_json::{Value, json}; @@ -33,7 +31,7 @@ fn read_preserves_absolute_virtual_store_dir() { fs::write(modules_dir.join(".modules.yaml"), raw).expect("write fixture"); let manifest = modules_dir - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read manifest") .expect("manifest exists"); assert_eq!(Path::new(&manifest.virtual_store_dir), custom_store); @@ -55,7 +53,7 @@ fn write_relativizes_non_descendant_virtual_store_dir() { "virtualStoreDir": &sibling_store, })); - write_modules_manifest::(&modules_dir, manifest).expect("write manifest"); + write_modules_manifest::(&modules_dir, manifest).expect("write manifest"); let raw: Value = modules_dir .join(".modules.yaml") .pipe(fs::read_to_string) @@ -77,7 +75,7 @@ fn write_sorts_skipped_array() { "skipped": ["zeta", "alpha", "mu"], })); - write_modules_manifest::(modules_dir, manifest).expect("write manifest"); + write_modules_manifest::(modules_dir, manifest).expect("write manifest"); let raw: Value = modules_dir .join(".modules.yaml") .pipe(fs::read_to_string) @@ -99,7 +97,7 @@ fn write_removes_null_public_hoist_pattern() { "publicHoistPattern": null, })); - write_modules_manifest::(modules_dir, manifest).expect("write manifest"); + write_modules_manifest::(modules_dir, manifest).expect("write manifest"); let raw: Value = modules_dir .join(".modules.yaml") .pipe(fs::read_to_string) @@ -136,7 +134,7 @@ fn dep_path_serializes_transparently() { [DepPath::from("/sharp/0.32.0".to_string())].into_iter().collect(); assert_eq!(manifest.ignored_builds.as_ref(), Some(&expected_ignored)); - write_modules_manifest::(modules_dir, manifest).expect("write manifest"); + write_modules_manifest::(modules_dir, manifest).expect("write manifest"); let raw: Value = modules_dir .join(".modules.yaml") .pipe(fs::read_to_string) @@ -181,8 +179,8 @@ fn hoisted_locations_round_trips() { Some(&vec!["node_modules/accepts".to_string()]), ); - write_modules_manifest::(modules_dir, manifest.clone()).expect("write manifest"); - let actual = read_modules_manifest::(modules_dir) + write_modules_manifest::(modules_dir, manifest.clone()).expect("write manifest"); + let actual = read_modules_manifest::(modules_dir) .expect("read manifest") .expect("manifest exists"); assert_eq!(actual.hoisted_locations, manifest.hoisted_locations); @@ -211,7 +209,7 @@ fn absent_hoisted_locations_is_omitted_on_write() { let manifest = manifest_from_json(json!({ "layoutVersion": 5 })); assert!(manifest.hoisted_locations.is_none(), "fixture seed"); - write_modules_manifest::(modules_dir, manifest).expect("write manifest"); + write_modules_manifest::(modules_dir, manifest).expect("write manifest"); let raw: Value = modules_dir .join(".modules.yaml") .pipe(fs::read_to_string) diff --git a/pacquet/crates/package-manager/src/install.rs b/pacquet/crates/package-manager/src/install.rs index 8ba068247c..b076b72628 100644 --- a/pacquet/crates/package-manager/src/install.rs +++ b/pacquet/crates/package-manager/src/install.rs @@ -11,8 +11,8 @@ use pacquet_lockfile::{ LoadLockfileError, Lockfile, SaveLockfileError, StalenessReason, satisfies_package_manifest, }; use pacquet_modules_yaml::{ - DEFAULT_VIRTUAL_STORE_DIR_MAX_LENGTH, IncludedDependencies, LayoutVersion, Modules, - NodeLinker as ModulesNodeLinker, RealApi, WriteModulesError, write_modules_manifest, + DEFAULT_VIRTUAL_STORE_DIR_MAX_LENGTH, Host, IncludedDependencies, LayoutVersion, Modules, + NodeLinker as ModulesNodeLinker, WriteModulesError, write_modules_manifest, }; use pacquet_network::ThrottledClient; use pacquet_package_manifest::{DependencyGroup, PackageManifest}; @@ -470,7 +470,7 @@ where // directory layout, hoist patterns, included dependency groups, // store dir, and registries so a later install (or another // tool) can detect a layout change and prune accordingly. - write_modules_manifest::( + write_modules_manifest::( &config.modules_dir, build_modules_manifest( config, diff --git a/pacquet/crates/package-manager/src/install/tests.rs b/pacquet/crates/package-manager/src/install/tests.rs index 5fc2acd008..999aa194c4 100644 --- a/pacquet/crates/package-manager/src/install/tests.rs +++ b/pacquet/crates/package-manager/src/install/tests.rs @@ -2,7 +2,7 @@ use super::{Install, InstallError}; use pacquet_config::Config; use pacquet_lockfile::Lockfile; use pacquet_modules_yaml::{ - DEFAULT_VIRTUAL_STORE_DIR_MAX_LENGTH, LayoutVersion, Modules, NodeLinker, RealApi, + DEFAULT_VIRTUAL_STORE_DIR_MAX_LENGTH, Host, LayoutVersion, Modules, NodeLinker, read_modules_manifest, write_modules_manifest, }; use pacquet_package_manifest::{DependencyGroup, PackageManifest}; @@ -656,7 +656,7 @@ async fn install_writes_modules_yaml() { package_manager, .. } = modules_dir - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read .modules.yaml") .expect("modules manifest exists"); @@ -1828,7 +1828,7 @@ async fn frozen_install_preserves_seeded_skipped_across_reinstall() { skipped: seeded_keys.iter().map(|s| (*s).to_string()).collect(), ..Default::default() }; - write_modules_manifest::(&modules_dir, seed_modules).expect("seed .modules.yaml"); + write_modules_manifest::(&modules_dir, seed_modules).expect("seed .modules.yaml"); // Empty lockfile drives the constraint-free fast path. The // seed must survive verbatim. @@ -1860,7 +1860,7 @@ async fn frozen_install_preserves_seeded_skipped_across_reinstall() { .expect("frozen-lockfile install should succeed"); let written = modules_dir - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read .modules.yaml") .expect("modules manifest exists"); @@ -1983,7 +1983,7 @@ async fn frozen_install_silently_swallows_unreachable_optional_tarball() { // that never updates `opts.skipped`, so a future install retries // the fetch (in case the URL becomes reachable again). let written = modules_dir - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read .modules.yaml") .expect("modules manifest exists"); assert!( @@ -2174,7 +2174,7 @@ async fn frozen_install_no_optional_drops_optional_only_snapshots() { // Transient — must not bleed into the persistent // `.modules.yaml.skipped` set. let written = modules_dir - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read .modules.yaml") .expect("modules manifest exists"); assert!( @@ -2423,7 +2423,7 @@ async fn hoisted_node_linker_empty_lockfile_writes_modules_yaml() { .expect("hoisted-linker install with empty lockfile should succeed"); let written = modules_dir - .pipe_as_ref(read_modules_manifest::) + .pipe_as_ref(read_modules_manifest::) .expect("read .modules.yaml") .expect("modules manifest exists"); diff --git a/pacquet/crates/package-manager/src/install_frozen_lockfile.rs b/pacquet/crates/package-manager/src/install_frozen_lockfile.rs index 2fe7bdf418..6a1de02201 100644 --- a/pacquet/crates/package-manager/src/install_frozen_lockfile.rs +++ b/pacquet/crates/package-manager/src/install_frozen_lockfile.rs @@ -18,7 +18,7 @@ use pacquet_executor::ScriptsPrependNodePath as ExecScriptsPrependNodePath; use pacquet_lockfile::{ Lockfile, PackageKey, PackageMetadata, Prefix, ProjectSnapshot, SnapshotEntry, }; -use pacquet_modules_yaml::{RealApi, read_modules_manifest}; +use pacquet_modules_yaml::{Host, read_modules_manifest}; use pacquet_network::ThrottledClient; use pacquet_package_manifest::DependencyGroup; use pacquet_patching::{ @@ -338,7 +338,7 @@ where // A read error (corrupt yaml, permissions) is degraded to // an empty seed — `.modules.yaml` is a cache artifact, not // an authoritative source. Missing file → empty seed. - let seed = match read_modules_manifest::(&config.modules_dir) { + let seed = match read_modules_manifest::(&config.modules_dir) { Ok(Some(manifest)) => SkippedSnapshots::from_strings(&manifest.skipped), Ok(None) => SkippedSnapshots::new(), Err(error) => { diff --git a/pacquet/crates/package-manager/src/install_without_lockfile.rs b/pacquet/crates/package-manager/src/install_without_lockfile.rs index 992abf13d4..fad569a6fb 100644 --- a/pacquet/crates/package-manager/src/install_without_lockfile.rs +++ b/pacquet/crates/package-manager/src/install_without_lockfile.rs @@ -7,7 +7,7 @@ use dashmap::DashSet; use derive_more::{Display, Error}; use futures_util::future; use miette::Diagnostic; -use pacquet_cmd_shim::{LinkBinsError, RealApi, link_bins}; +use pacquet_cmd_shim::{Host, LinkBinsError, link_bins}; use pacquet_config::Config; use pacquet_network::ThrottledClient; use pacquet_package_manifest::{DependencyGroup, PackageManifest}; @@ -207,7 +207,7 @@ impl<'a, DependencyGroupList> InstallWithoutLockfile<'a, DependencyGroupList> { // iterator was already consumed by the install loop above; pnpm's // own `linkBins(modulesDir, binsDir)` overload uses the same // strategy. - link_bins::(&config.modules_dir, &config.modules_dir.join(".bin")) + link_bins::(&config.modules_dir, &config.modules_dir.join(".bin")) .map_err(InstallWithoutLockfileError::LinkBins)?; // No lockfile here, so no prefetched manifests are available — diff --git a/pacquet/crates/package-manager/src/link_bins.rs b/pacquet/crates/package-manager/src/link_bins.rs index 4d591049b8..2aa6fbd86c 100644 --- a/pacquet/crates/package-manager/src/link_bins.rs +++ b/pacquet/crates/package-manager/src/link_bins.rs @@ -3,7 +3,7 @@ use derive_more::{Display, Error}; use miette::Diagnostic; use pacquet_cmd_shim::{ BinOrigin, FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, - FsReadString, FsSetExecutable, FsWalkFiles, FsWrite, LinkBinsError, PackageBinSource, RealApi, + FsReadToString, FsSetExecutable, FsWalkFiles, FsWrite, Host, LinkBinsError, PackageBinSource, link_bins_of_packages, }; use pacquet_lockfile::{LockfileResolution, PackageKey, PackageMetadata, PkgName, SnapshotEntry}; @@ -61,7 +61,7 @@ pub fn link_direct_dep_bins(modules_dir: &Path, dep_names: &[String]) -> Result< if bin_sources.is_empty() { return Ok(()); } - link_bins_of_packages::(&bin_sources, &modules_dir.join(".bin")) + link_bins_of_packages::(&bin_sources, &modules_dir.join(".bin")) } /// Top-level bin link that mixes direct-dep candidates and hoisted @@ -120,7 +120,7 @@ pub fn link_top_level_bins( if bin_sources.is_empty() { return Ok(()); } - link_bins_of_packages::(&bin_sources, &modules_dir.join(".bin")) + link_bins_of_packages::(&bin_sources, &modules_dir.join(".bin")) } /// Read each `//package.json` and assemble the @@ -245,19 +245,19 @@ pub struct LinkVirtualStoreBins<'a> { impl<'a> LinkVirtualStoreBins<'a> { pub fn run(self) -> Result<(), LinkVirtualStoreBinsError> { - self.run_with::() + self.run_with::() } /// DI-driven entry. Production callers go through [`Self::run`] which - /// turbofishes [`RealApi`]; tests inject fakes that fail specific fs + /// turbofishes [`Host`]; tests inject fakes that fail specific fs /// operations to cover error paths the real fs can't trigger /// portably. See the per-capability DI pattern at /// . - pub fn run_with(self) -> Result<(), LinkVirtualStoreBinsError> + pub fn run_with(self) -> Result<(), LinkVirtualStoreBinsError> where - Api: FsReadDir + Sys: FsReadDir + FsReadFile - + FsReadString + + FsReadToString + FsReadHead + FsCreateDirAll + FsWalkFiles @@ -268,7 +268,7 @@ impl<'a> LinkVirtualStoreBins<'a> { let LinkVirtualStoreBins { layout, snapshots, packages, package_manifests, skipped } = self; if let Some(snapshots) = snapshots { let has_bin_set = build_has_bin_set(packages); - run_lockfile_driven::( + run_lockfile_driven::( layout, snapshots, has_bin_set.as_ref(), @@ -281,7 +281,7 @@ impl<'a> LinkVirtualStoreBins<'a> { // frozen installs, which #432 doesn't activate GVS for, so // reading from `layout.package_store_dir()` reproduces // today's behaviour exactly when GVS is off. - run_with_readdir::(layout.package_store_dir()) + run_with_readdir::(layout.package_store_dir()) } } } @@ -346,7 +346,7 @@ fn build_has_bin_set( /// cold-batch packages that prefetch missed, a fallback /// `package.json` read through the existing symlink at /// `/node_modules/`. -fn run_lockfile_driven( +fn run_lockfile_driven( layout: &crate::VirtualStoreLayout, snapshots: &HashMap, has_bin_set: Option<&HashSet>, @@ -354,8 +354,8 @@ fn run_lockfile_driven( skipped: &SkippedSnapshots, ) -> Result<(), LinkVirtualStoreBinsError> where - Api: FsReadFile - + FsReadString + Sys: FsReadFile + + FsReadToString + FsReadHead + FsCreateDirAll + FsWalkFiles @@ -467,7 +467,7 @@ where // prefetched manifest map yet. Reading from disk // here is the same code path as the non-lockfile // install — see [`run_with_readdir`]. - match read_package::(&child_location) { + match read_package::(&child_location) { Ok(Some(pkg)) => bin_sources.push(pkg), Ok(None) => {} Err(error) => return Err(LinkVirtualStoreBinsError::LinkBins(error)), @@ -484,7 +484,7 @@ where if let Some(manifest) = package_manifests.get(&self_metadata_key) { bin_sources.push(PackageBinSource::new(self_pkg_dir.clone(), Arc::clone(manifest))); } else { - match read_package::(&self_pkg_dir) { + match read_package::(&self_pkg_dir) { Ok(Some(pkg)) => bin_sources.push(pkg), Ok(None) => {} Err(error) => return Err(LinkVirtualStoreBinsError::LinkBins(error)), @@ -495,7 +495,7 @@ where if bin_sources.is_empty() { return Ok(()); } - link_bins_of_packages::(&bin_sources, &bins_dir) + link_bins_of_packages::(&bin_sources, &bins_dir) .map_err(LinkVirtualStoreBinsError::LinkBins) }) } @@ -526,11 +526,11 @@ fn pkg_dir_under(modules_dir: &Path, name: &PkgName) -> PathBuf { /// then walk each slot's `node_modules` to discover children. Used /// only by [`crate::InstallWithoutLockfile`] today; the lockfile /// path bypasses every directory enumeration in here. -fn run_with_readdir(virtual_store_dir: &Path) -> Result<(), LinkVirtualStoreBinsError> +fn run_with_readdir(virtual_store_dir: &Path) -> Result<(), LinkVirtualStoreBinsError> where - Api: FsReadDir + Sys: FsReadDir + FsReadFile - + FsReadString + + FsReadToString + FsReadHead + FsCreateDirAll + FsWalkFiles @@ -538,7 +538,7 @@ where + FsSetExecutable + FsEnsureExecutableBits, { - let slots = match Api::read_dir(virtual_store_dir) { + let slots = match Sys::read_dir(virtual_store_dir) { Ok(slots) => slots, Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), Err(error) => { @@ -565,11 +565,11 @@ where // is trivial; the lockfile-driven path bypasses this by // treating the slot's own pkg dir as an invariant of // [`crate::create_virtual_dir_by_snapshot`]. - if Api::read_dir(&self_pkg_dir).is_err() { + if Sys::read_dir(&self_pkg_dir).is_err() { return Ok(()); } let bins_dir = self_pkg_dir.join("node_modules/.bin"); - link_bins_excluding::(&modules_dir, &bins_dir, &self_pkg_dir) + link_bins_excluding::(&modules_dir, &bins_dir, &self_pkg_dir) .map_err(LinkVirtualStoreBinsError::LinkBins) }) } @@ -589,7 +589,7 @@ where /// /// Returns `None` only when the slot name fails to parse — there's no /// filesystem probe for the resolved candidate. The previous version -/// stat-equivalent-ed the path with `Api::read_dir` to short-circuit +/// stat-equivalent-ed the path with `Sys::read_dir` to short-circuit /// missing slots, but on a 1267-package fixture that was 1267 /// wasted `open(O_DIRECTORY) + close` round-trips on the hot path of /// every warm install. The slot's own package directory is an @@ -627,15 +627,15 @@ fn find_slot_own_package_dir(slot_dir: &Path, modules_dir: &Path) -> Option( +fn link_bins_excluding( modules_dir: &Path, bins_dir: &Path, exclude: &Path, ) -> Result<(), LinkBinsError> where - Api: FsReadDir + Sys: FsReadDir + FsReadFile - + FsReadString + + FsReadToString + FsReadHead + FsCreateDirAll + FsWalkFiles @@ -645,7 +645,7 @@ where { let mut packages: Vec = Vec::new(); - let entries = match Api::read_dir(modules_dir) { + let entries = match Sys::read_dir(modules_dir) { Ok(entries) => entries, Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), Err(error) => { @@ -669,7 +669,7 @@ where // the bins for every package under this scope silently // disappear, so surface them instead of letting them // hide. - let scope_entries = match Api::read_dir(&path) { + let scope_entries = match Sys::read_dir(&path) { Ok(entries) => entries, Err(error) if error.kind() == io::ErrorKind::NotFound => continue, Err(error) => { @@ -680,7 +680,7 @@ where if paths_eq(&sub_path, exclude) { continue; } - if let Some(pkg) = read_package::(&sub_path)? { + if let Some(pkg) = read_package::(&sub_path)? { packages.push(pkg); } } @@ -690,7 +690,7 @@ where if paths_eq(&path, exclude) { continue; } - if let Some(pkg) = read_package::(&path)? { + if let Some(pkg) = read_package::(&path)? { packages.push(pkg); } } @@ -699,14 +699,14 @@ where return Ok(()); } - link_bins_of_packages::(&packages, bins_dir) + link_bins_of_packages::(&packages, bins_dir) } -fn read_package( +fn read_package( location: &Path, ) -> Result, LinkBinsError> { let manifest_path = location.join("package.json"); - let bytes = match Api::read_file(&manifest_path) { + let bytes = match Sys::read_file(&manifest_path) { Ok(bytes) => bytes, Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None), Err(error) => return Err(LinkBinsError::ReadManifest { path: manifest_path, error }), diff --git a/pacquet/crates/package-manager/src/link_bins/tests.rs b/pacquet/crates/package-manager/src/link_bins/tests.rs index 2022cca7e1..5023929055 100644 --- a/pacquet/crates/package-manager/src/link_bins/tests.rs +++ b/pacquet/crates/package-manager/src/link_bins/tests.rs @@ -426,7 +426,7 @@ fn link_direct_dep_bins_skips_dep_with_missing_manifest() { #[test] fn link_virtual_store_bins_propagates_read_error_via_di() { use pacquet_cmd_shim::{ - FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadString, + FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadToString, FsSetExecutable, FsWalkFiles, FsWrite, }; use std::io; @@ -442,7 +442,7 @@ fn link_virtual_store_bins_propagates_read_error_via_di() { unreachable!() } } - impl FsReadString for DenyVirtualStore { + impl FsReadToString for DenyVirtualStore { fn read_to_string(_: &Path) -> io::Result { unreachable!() } diff --git a/pacquet/crates/reporter/src/lib.rs b/pacquet/crates/reporter/src/lib.rs index 89b62092f0..8586fcc2c0 100644 --- a/pacquet/crates/reporter/src/lib.rs +++ b/pacquet/crates/reporter/src/lib.rs @@ -741,8 +741,17 @@ fn now_millis() -> u128 { /// Capability for obtaining the host name written into the [bunyan]-shaped /// envelope. /// -/// Backed by a real syscall in production via [`RealApi`]. Tests can supply -/// their own implementation when behavior depends on the value. +/// Backed by a real syscall in production via [`Host`]. The envelope itself +/// reads from a process-cached `HOSTNAME` `static` initialized by +/// [`Host::get_host_name`], so the value is fixed for the lifetime of the +/// process and the envelope path is **not** currently generic over this +/// trait. The trait therefore exists for two narrow reasons: to keep the +/// `gethostname` syscall behind a named seam (so the production call site +/// is consistent with the rest of `Host`'s capability surface), and so the +/// capability can be exercised in isolation by unit tests. Substituting a +/// hostname per-test in the rendered envelope would require plumbing a +/// `Sys: GetHostName` generic through the emission site, which has not +/// been done. /// /// [bunyan]: https://github.com/trentm/node-bunyan pub trait GetHostName { @@ -753,9 +762,9 @@ pub trait GetHostName { /// /// Each trait method calls into the real underlying system facility (for /// [`GetHostName`], the `gethostname` syscall via the [`gethostname`] crate). -pub struct RealApi; +pub struct Host; -impl GetHostName for RealApi { +impl GetHostName for Host { fn get_host_name() -> String { gethostname::gethostname().to_string_lossy().into_owned() } @@ -763,9 +772,9 @@ impl GetHostName for RealApi { // Process-wide cache of the host name. The value cannot change at runtime, // and `gethostname` is one syscall we'd otherwise repeat on every emit. -// Initialized lazily through `RealApi::get_host_name` so tests that exercise +// Initialized lazily through `Host::get_host_name` so tests that exercise // the capability trait directly can do so without paying for the syscall. -static HOSTNAME: LazyLock = LazyLock::new(RealApi::get_host_name); +static HOSTNAME: LazyLock = LazyLock::new(Host::get_host_name); #[cfg(test)] mod tests; diff --git a/pacquet/crates/reporter/src/tests.rs b/pacquet/crates/reporter/src/tests.rs index c6ee2c023f..cdc1ca70e8 100644 --- a/pacquet/crates/reporter/src/tests.rs +++ b/pacquet/crates/reporter/src/tests.rs @@ -6,9 +6,9 @@ use serde_json::Value; use crate::{ AddedRoot, BrokenModulesLog, ContextLog, DependencyType, Envelope, FetchingProgressLog, - FetchingProgressMessage, GetHostName, IgnoredScriptsLog, LifecycleLog, LifecycleMessage, + FetchingProgressMessage, GetHostName, Host, IgnoredScriptsLog, LifecycleLog, LifecycleMessage, LifecycleStdio, LogEvent, LogLevel, PackageImportMethod, PackageImportMethodLog, - PackageManifestLog, PackageManifestMessage, ProgressLog, ProgressMessage, RealApi, RemovedRoot, + PackageManifestLog, PackageManifestMessage, ProgressLog, ProgressMessage, RemovedRoot, Reporter, RequestRetryError, RequestRetryLog, RootLog, RootMessage, SilentReporter, SkippedOptionalDependencyLog, SkippedOptionalPackage, SkippedOptionalReason, Stage, StageLog, StatsLog, StatsMessage, SummaryLog, @@ -805,20 +805,20 @@ fn recording_fake_captures_emitted_events() { /// test, which is what consumers of the trait need to know. #[test] fn get_host_name_capability_is_mockable() { - struct FakeApi; - impl GetHostName for FakeApi { + struct FakeHostName; + impl GetHostName for FakeHostName { fn get_host_name() -> String { "fixture-host".to_owned() } } - assert_eq!(FakeApi::get_host_name(), "fixture-host"); + assert_eq!(FakeHostName::get_host_name(), "fixture-host"); } -/// [`RealApi::get_host_name`] returns the value of `gethostname(2)`, +/// [`Host::get_host_name`] returns the value of `gethostname(2)`, /// which any real environment populates with at least one byte. #[test] -fn real_api_returns_a_non_empty_host_name() { - let host = RealApi::get_host_name(); - eprintln!("RealApi::get_host_name() = {host:?}"); +fn host_returns_a_non_empty_host_name() { + let host = Host::get_host_name(); + eprintln!("Host::get_host_name() = {host:?}"); assert!(!host.is_empty()); } From 4195766f10057e6b07a69705f8f226666a1730ef Mon Sep 17 00:00:00 2001 From: Zoltan Kochan Date: Mon, 18 May 2026 09:51:11 +0200 Subject: [PATCH 011/169] =?UTF-8?q?feat:=20tighten=20minimumReleaseAge=20?= =?UTF-8?q?=E2=80=94=20auto-exclude,=20lockfile=20verification,=20and=20in?= =?UTF-8?q?teractive=20prompt=20(#11705)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three coordinated changes that close the silent-bypass gap in loose `minimumReleaseAge` mode AND the discover-by-loop UX problem in strict mode (#10488), plus a parallel hardening of the lockfile verifier: 1. **Auto-collect into `minimumReleaseAgeExclude` (loose mode)** — fresh resolutions that fall back to a version newer than the cutoff are auto-recorded into the workspace manifest's `minimumReleaseAgeExclude`. A single info message lists what was persisted. The workspace manifest writer dedupes against existing entries. 2. **Lockfile verifier runs in loose mode too** — `createNpmResolutionVerifier` no longer gates on `minimumReleaseAgeStrict`. With auto-collect keeping the exclude list explicit, every accepted-immature pin must be on the list — same contract strict mode enforces. Lockfiles produced under a weaker (or absent) policy that still hold immature entries are rejected the same way strict mode would. 3. **Strict mode prompts on the aggregate set instead of throwing on the first** — the resolver always collects every immature direct and transitive in one pass; the install command's `handleResolutionPolicyViolations` checkpoint decides what to do with the set. Interactive (TTY) prompts the user once with the full list (default = No) and asks whether to add them all to `minimumReleaseAgeExclude` and proceed. Approve → install continues, persisted at the end. Decline → resolution aborts before the lockfile, package.json, or modules dir is touched. Non-interactive (CI) keeps `ERR_PNPM_NO_MATURE_MATCHING_VERSION` as the exit code but lists every offending entry instead of just the first one the resolver happened to hit. 4. **The lockfile verifier now also covers `trustPolicy: 'no-downgrade'`.** The same post-resolution gate that re-checks `minimumReleaseAge` on lockfile entries now re-runs `failIfTrustDowngraded` for every npm-registry entry whose name isn't on `trustPolicyExclude`. The two checks share a single full-metadata fetch per package, so the extra coverage doesn't cost an extra round trip when both policies are active. Resolver-time trust checks still run as before — this just closes the gap when an entry bypasses resolution (peek path, `--frozen-lockfile`, restored CI cache). The steady-state flows: - **Loose mode, `pnpm add foo@immature`**: lockfile clean, verifier no-op, resolver picks via lowest-version fallback, `foo@immature` lands in `minimumReleaseAgeExclude`, install succeeds. Subsequent `pnpm install --frozen-lockfile` in CI verifies against the populated list and succeeds. - **Strict mode (interactive), security bump to `next@15.5.9`**: resolver collects `next@15.5.9` AND every immature `@next/swc-*@15.5.9` shim. pnpm prompts once with the full list. User approves → install completes, all entries persisted in `pnpm-workspace.yaml`. CI then runs the populated config cleanly. - **Strict mode (non-interactive / CI)**: aborts with `ERR_PNPM_NO_MATURE_MATCHING_VERSION` listing every immature entry's `name@version` and publish time — no more discover-by-loop dance. - **Teammate commits a poisoned lockfile**: single-policy batches reject with `ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION` (or `ERR_PNPM_TRUST_DOWNGRADE`); a batch that trips both policies escalates to the generic `ERR_PNPM_LOCKFILE_RESOLUTION_VERIFICATION` and lists each entry's per-policy code in the breakdown. ### Implementation - The npm resolver always falls back to the lowest matching version when no mature version satisfies the range, and flags the result with `ResolveResult.policyViolation` instead of throwing `NO_MATURE_MATCHING_VERSION`. `deferImmatureDecision` and `strictPublishedByCheck` are gone — every caller (install, dlx, outdated, self-update) inspects the violation and decides what to do. - `policyViolation` flows from `ResolveResult` → `PackageResponse.body.policyViolation` → a shared accumulator in `ResolutionContext` → the `resolutionPolicyViolations` field on `resolveDependencyTree`'s return → out through `mutateModules` / `addDependenciesToPackage` to the install command. - The violation type lives in `@pnpm/resolving.resolver-base` as `ResolutionPolicyViolation`; the npm resolver exports the two built-in codes (`MINIMUM_RELEASE_AGE_VIOLATION_CODE`, `TRUST_DOWNGRADE_VIOLATION_CODE`) as constants so consumers reference one source of truth. - `handleResolutionPolicyViolations` runs between `resolveDependencyTree` and `resolvePeers` — the resolver-agnostic checkpoint where the install command's plan prompts (TTY) or aborts (no-TTY) with the full violation list. - `setupPolicyHandlers` (in `installing/commands/src/policyHandlers.ts`) composes per-policy handlers behind a uniform plan interface: each handler has its own `handleResolutionPolicyViolations` (filter by code, decide what to do) and `pickManifestUpdates` (return a typed `WorkspaceManifestPolicyUpdates` patch the install command spreads into `updateWorkspaceManifest`). Today the only registered handler is `createMinimumReleaseAgeHandler` — strict + TTY prompts via `enquirer`, strict no-TTY throws `ERR_PNPM_NO_MATURE_MATCHING_VERSION` with every entry listed, loose mode auto-persists at the tail. Strict + `--no-save` is rejected up-front via `ERR_PNPM_STRICT_MIN_RELEASE_AGE_REQUIRES_SAVE`. Future policies plug in via a sibling factory + push into the handlers list, with no changes to `installDeps.ts` / `recursive.ts`. - `installDeps` / `recursive` drain `pickManifestUpdates` after install and spread the patch into `updateWorkspaceManifest`. Plain `pnpm install` (no `--update`, no params) now still updates the workspace manifest when any handler contributes a patch. The `install` command's CLI schema gained `save: Boolean` so `--no-save` actually flows through to `opts.save = false` instead of being silently dropped by nopt. - `makeResolutionStrict` (in `installing/client`) wraps a `ResolveFunction` and rethrows any `policyViolation` as a `PnpmError`. Used by `dlx` and `self-update` under strict `minimumReleaseAge` OR `trustPolicy: 'no-downgrade'`, since one-shot callers have nowhere to defer a violation to. Violation-code → error-code mapping lives in one place so future violation kinds get consistent UX. - `createNpmResolutionVerifier` extends its check to `trustPolicy: 'no-downgrade'` — same per-entry fan-out, same cache key, sharing the full-metadata fetch with the maturity check. Trust-fetch errors now propagate up so the violation reason carries the underlying message (network code, 404 detail) instead of a generic "metadata is unavailable". - `verifyLockfileResolutions`'s aggregate throw uses the per-policy code when every violation in the batch shares it, and escalates to a generic `LOCKFILE_RESOLUTION_VERIFICATION` (with per-entry codes in the breakdown) for mixed batches. - The pnpm agent path refuses installs under `trustPolicy: 'no-downgrade'` (`ERR_PNPM_TRUST_POLICY_INCOMPATIBLE_WITH_AGENT`) — the agent has no server-side counterpart to that check yet, so silently allowing it would land a lockfile the local verifier would later reject. `minimumReleaseAge` is forwarded to the agent and enforced server-side, so that combination is fine. ### Pacquet parity Pacquet only carries a stub reference to `minimumReleaseAgeExclude` (see `pacquet/crates/package-manager/src/version_policy.rs`); the broader `minimumReleaseAge` and `trustPolicy` policies aren't ported yet, so this feature is outside pacquet's current surface area. It'll come along when pacquet ports the policies. ### Closes - Closes #10488 (resolves the discover-by-loop dance for security bumps without needing `withTransitives`). --- ...uto-collect-minimum-release-age-exclude.md | 27 ++ __fixtures__/pnpm-workspace.yaml | 9 + cli/default-reporter/src/reportError.ts | 26 +- .../outdated/src/createManifestGetter.ts | 23 +- .../outdated/test/getManifest.spec.ts | 26 +- .../commands/src/self-updater/selfUpdate.ts | 15 +- exec/commands/src/dlx.ts | 16 +- exec/commands/test/dlx.e2e.ts | 2 +- installing/client/package.json | 2 + installing/client/src/index.ts | 48 ++- installing/client/tsconfig.json | 6 + installing/commands/package.json | 3 +- installing/commands/src/install.ts | 5 + installing/commands/src/installDeps.ts | 54 ++- installing/commands/src/policyHandlers.ts | 260 ++++++++++++ installing/commands/src/recursive.ts | 45 ++- installing/commands/test/add.ts | 2 +- installing/commands/test/policyHandlers.ts | 160 ++++++++ installing/commands/tsconfig.json | 3 + .../src/install/extendInstallOptions.ts | 19 +- .../deps-installer/src/install/index.ts | 48 ++- .../src/install/verifyLockfileResolutions.ts | 154 ++++--- .../test/install/minimumReleaseAge.ts | 156 +++++++- .../test/install/verifyLockfileResolutions.ts | 32 +- installing/deps-resolver/src/index.ts | 53 +++ .../deps-resolver/src/resolveDependencies.ts | 21 +- .../src/resolveDependencyTree.ts | 12 +- .../package-requester/src/packageRequester.ts | 3 + pnpm-lock.yaml | 15 +- pnpm/test/dlx.ts | 4 +- pnpm/test/install/minimumReleaseAge.ts | 223 ++++++++++- pnpm/test/install/misc.ts | 43 ++ resolving/default-resolver/src/index.ts | 8 + .../src/createNpmResolutionVerifier.ts | 375 ++++++++++++++---- resolving/npm-resolver/src/index.ts | 123 ++++-- resolving/npm-resolver/src/pickPackage.ts | 35 +- resolving/npm-resolver/src/violationCodes.ts | 12 + .../test/createNpmResolutionVerifier.test.ts | 241 +++++++++++ .../npm-resolver/test/publishedBy.test.ts | 43 +- .../npm-resolver/test/resolveJsr.test.ts | 36 ++ resolving/resolver-base/src/index.ts | 36 ++ .../src/store/cleanLockfileVerifiedCache.ts | 30 ++ store/commands/src/store/storePrune.ts | 3 + .../src/createNewStoreController.ts | 6 +- store/controller-types/src/index.ts | 8 + workspace/state/src/types.ts | 12 + 46 files changed, 2183 insertions(+), 300 deletions(-) create mode 100644 .changeset/auto-collect-minimum-release-age-exclude.md create mode 100644 installing/commands/src/policyHandlers.ts create mode 100644 installing/commands/test/policyHandlers.ts create mode 100644 resolving/npm-resolver/src/violationCodes.ts create mode 100644 resolving/npm-resolver/test/createNpmResolutionVerifier.test.ts create mode 100644 store/commands/src/store/cleanLockfileVerifiedCache.ts diff --git a/.changeset/auto-collect-minimum-release-age-exclude.md b/.changeset/auto-collect-minimum-release-age-exclude.md new file mode 100644 index 0000000000..614d9b362d --- /dev/null +++ b/.changeset/auto-collect-minimum-release-age-exclude.md @@ -0,0 +1,27 @@ +--- +"@pnpm/resolving.resolver-base": minor +"@pnpm/store.controller-types": minor +"@pnpm/resolving.npm-resolver": minor +"@pnpm/resolving.default-resolver": minor +"@pnpm/installing.client": minor +"@pnpm/installing.deps-resolver": minor +"@pnpm/installing.deps-installer": minor +"@pnpm/installing.commands": minor +"@pnpm/store.connection-manager": minor +"@pnpm/deps.inspection.outdated": patch +"@pnpm/engine.pm.commands": patch +"@pnpm/exec.commands": patch +"@pnpm/cli.default-reporter": patch +"pnpm": minor +--- + +Tightened the `minimumReleaseAge` story so the bypass becomes explicit on disk instead of silent, and removed the discover-by-loop dance for strict-mode users: + +1. Fresh resolutions in loose mode (`minimumReleaseAgeStrict: false`) that fall back to a version newer than the cutoff auto-collect the picked `name@version` into the workspace manifest's `minimumReleaseAgeExclude`. A single info message lists the additions; entries already on the list are left alone. +2. The post-resolution lockfile verifier introduced in #11583 now runs in loose mode too — every accepted-immature pin must be on `minimumReleaseAgeExclude`, just like strict mode requires. A lockfile produced under a weaker (or absent) policy that still has immature entries is rejected the same way strict mode would reject it. +3. **Strict mode (interactive)** no longer aborts on the first immature pick. The resolver gathers every immature direct *and* transitive in one pass; before peer-dependency resolution runs, pnpm prompts the user with the full list and asks whether to add them all to `minimumReleaseAgeExclude` and proceed. Approve → install continues and the workspace manifest is written at the end. Decline → resolution aborts before the lockfile or package.json is touched (tarballs already in the store stay, since the store is idempotent). This closes the [#10488](https://github.com/pnpm/pnpm/issues/10488) loop where security bumps to packages with platform-specific transitives (e.g. `next` + the `@next/swc-*` shims) made users re-run `pnpm add` once per transitive. +4. **Strict mode (non-interactive / CI)** now aborts with the full immature set in the error message instead of the first pick. The resolver always collects every immature direct + transitive; the install command then throws `ERR_PNPM_NO_MATURE_MATCHING_VERSION` listing each entry's `name@version` and publish time. Deterministic CI behavior is preserved (same exit code, same error code), but the error pinpoints every offending entry instead of forcing the discover-by-loop dance. The expected workflow is interactive approval locally → the lockfile + workspace manifest get committed → CI runs cleanly against the populated exclude list. + +5. **The lockfile verifier now also covers `trustPolicy: 'no-downgrade'`.** The same post-resolution gate that re-checks `minimumReleaseAge` on lockfile entries now re-runs `failIfTrustDowngraded` for every npm-registry entry whose name isn't on `trustPolicyExclude`. The two checks share a single full-metadata fetch per package, so the extra coverage doesn't cost an extra round trip when both policies are active. Resolver-time trust checks still run as before — this just closes the gap when an entry bypasses resolution (peek path, `--frozen-lockfile`, restored CI cache). + +Pacquet parity: not ported — pacquet's `minimumReleaseAge` policy is itself only stubbed today (see `pacquet/crates/package-manager/src/version_policy.rs`). The auto-exclude, loose-mode verifier, prompt, and the new trust-policy verifier check will travel with the broader policy port whenever that happens. diff --git a/__fixtures__/pnpm-workspace.yaml b/__fixtures__/pnpm-workspace.yaml index 21b565263d..fea4dd5fb4 100644 --- a/__fixtures__/pnpm-workspace.yaml +++ b/__fixtures__/pnpm-workspace.yaml @@ -19,6 +19,15 @@ packages: - '!workspace-has-shared-npm-shrinkwrap-json' sharedWorkspaceLockfile: false +# The fixture lockfiles are pinned to packages from the local registry-mock +# (e.g. `@pnpm.e2e/*`). The v11 default of `minimumReleaseAge: 1440` would +# spin up the lockfile verifier here, which can't reach the mock from this +# install context and rejects the entries as un-checkable. Disable the +# policy for fixture installs — they're test scaffolding, not a real +# project. The actual minimumReleaseAge code paths are covered by the +# unit and e2e tests in their own packages. +minimumReleaseAge: 0 + catalog: # Used in has-outdated-deps-using-catalog-protocol fixture. is-negative: ^1.0.0 diff --git a/cli/default-reporter/src/reportError.ts b/cli/default-reporter/src/reportError.ts index 53a791faa2..424dacd47d 100644 --- a/cli/default-reporter/src/reportError.ts +++ b/cli/default-reporter/src/reportError.ts @@ -72,7 +72,12 @@ function getErrorInfo (logObj: Log, config?: Config): ErrorInfo | null { return { title: err.message, body: 'If you cannot fix this registry issue, then set "resolution-mode" to "highest".' } case 'ERR_PNPM_NO_MATCHING_VERSION': case 'ERR_PNPM_NO_MATURE_MATCHING_VERSION': - return formatNoMatchingVersion(err, logObj as unknown as { packageMeta: PackageMeta, immatureVersion?: string }) + // ERR_PNPM_NO_MATURE_MATCHING_VERSION used to come from the resolver + // with `packageMeta` attached; it now comes from the install / dlx / + // self-update callers as a plain PnpmError once the resolver has + // surfaced the violations. `packageMeta` may be undefined, in which + // case the formatter falls back to the bare title+message. + return formatNoMatchingVersion(err, logObj as unknown as { packageMeta?: PackageMeta }) case 'ERR_PNPM_RECURSIVE_FAIL': return formatRecursiveCommandSummary(logObj as any) // eslint-disable-line @typescript-eslint/no-explicit-any case 'ERR_PNPM_BAD_TARBALL_SIZE': @@ -134,11 +139,18 @@ interface PackageMeta { time?: Record } -function formatNoMatchingVersion (err: Error, msg: { packageMeta: PackageMeta, immatureVersion?: string }) { - const meta: PackageMeta = msg.packageMeta +function formatNoMatchingVersion (err: Error, msg: { packageMeta?: PackageMeta }) { + // Errors raised by the install/dlx/self-update layer after the resolver + // surfaces violations may not carry the original packageMeta. In that + // case the error message alone already names every offending entry, + // so we just echo it through without the registry-metadata appendix. + const meta = msg.packageMeta + if (!meta) { + return { title: err.message } + } const latestVersion = meta['dist-tags'].latest let output = `The latest release of ${meta.name} is "${latestVersion}".` - const latestTime = msg.packageMeta.time?.[latestVersion] + const latestTime = meta.time?.[latestVersion] if (latestTime) { output += ` Published at ${stringifyDate(latestTime)}` } @@ -150,7 +162,7 @@ function formatNoMatchingVersion (err: Error, msg: { packageMeta: PackageMeta, i if (tag !== 'latest') { const version = meta['dist-tags'][tag] output += ` * ${tag}: ${version}` - const time = msg.packageMeta.time?.[version] + const time = meta.time?.[version] if (time) { output += ` published at ${stringifyDate(time)}` } @@ -161,10 +173,6 @@ function formatNoMatchingVersion (err: Error, msg: { packageMeta: PackageMeta, i output += `${EOL}If you need the full list of all ${Object.keys(meta.versions).length} published versions run "pnpm view ${meta.name} versions".` - if (msg.immatureVersion) { - output += `${EOL}${EOL}If you want to install the matched version ignoring the time it was published, you can add the package name to the minimumReleaseAgeExclude setting. Read more about it: https://pnpm.io/settings#minimumreleaseageexclude` - } - return { title: err.message, body: output, diff --git a/deps/inspection/outdated/src/createManifestGetter.ts b/deps/inspection/outdated/src/createManifestGetter.ts index 11ce02d677..9f0327dd24 100644 --- a/deps/inspection/outdated/src/createManifestGetter.ts +++ b/deps/inspection/outdated/src/createManifestGetter.ts @@ -29,7 +29,6 @@ export function createManifestGetter ( ...opts, configByUri: opts.configByUri, filterMetadata: false, // We need all the data from metadata for "outdated --long" to work. - strictPublishedByCheck: Boolean(opts.minimumReleaseAge) && opts.minimumReleaseAgeStrict === true, ignoreMissingTimeField: opts.minimumReleaseAgeIgnoreMissingTime, }) @@ -58,18 +57,22 @@ export async function getManifest ( publishedBy: opts.publishedBy, publishedByExclude: opts.publishedByExclude, }) + // No mature version found within range: the resolver fell back to the + // lowest immature pick and flagged it inline. `outdated` shouldn't + // present an immature version as "available", so treat it as no match + // — matching the pre-violation-collection behavior when the resolver + // threw `NO_MATURE_MATCHING_VERSION`. + if (resolution?.policyViolation?.code === 'MINIMUM_RELEASE_AGE_VIOLATION') { + return null + } return resolution?.manifest ?? null } catch (err) { const code = (err as { code?: string }).code - if (opts.publishedBy && ( - code === 'ERR_PNPM_NO_MATURE_MATCHING_VERSION' || - code === 'ERR_PNPM_NO_MATCHING_VERSION' - )) { - // No versions found that meet the minimumReleaseAge requirement. - // This can happen when all published versions (including the one the - // "latest" dist-tag points to) are newer than the minimumReleaseAge - // threshold, causing the resolver to throw NO_MATCHING_VERSION instead - // of NO_MATURE_MATCHING_VERSION. + if (opts.publishedBy && code === 'ERR_PNPM_NO_MATCHING_VERSION') { + // No version satisfies the range at all (not a maturity issue). + // Pre-violation-collection this branch also covered the maturity + // case via `NO_MATURE_MATCHING_VERSION`; with always-defer, that + // case is handled above as a `policyViolation`. return null } throw err diff --git a/deps/inspection/outdated/test/getManifest.spec.ts b/deps/inspection/outdated/test/getManifest.spec.ts index 6f58c54b87..89a694cf8b 100644 --- a/deps/inspection/outdated/test/getManifest.spec.ts +++ b/deps/inspection/outdated/test/getManifest.spec.ts @@ -58,14 +58,30 @@ test('getManifest() with minimumReleaseAge filters latest when too new', async ( const publishedBy = new Date(Date.now() - 10080 * 60 * 1000) + // The resolver no longer throws on immature picks — it falls back to + // the lowest matching version and flags the result with `policyViolation`. + // outdated treats that as "no version available within the policy" and + // returns null, same as the pre-refactor throw path. const resolve = jest.fn(async (wantedPackage, resolveOpts) => { expect(wantedPackage.bareSpecifier).toBe('latest') expect(resolveOpts.publishedBy).toBeInstanceOf(Date) - - // Simulate latest version being too new - const error = new Error('No matching version found') as Error & { code?: string } - error.code = 'ERR_PNPM_NO_MATURE_MATCHING_VERSION' - throw error + return { + id: 'foo/2.0.0' as PkgResolutionId, + latest: '2.0.0', + manifest: { + name: 'foo', + version: '2.0.0', + }, + resolution: {} as TarballResolution, + resolvedVia: 'npm-registry', + policyViolation: { + name: 'foo', + version: '2.0.0', + resolution: {} as TarballResolution, + code: 'MINIMUM_RELEASE_AGE_VIOLATION', + reason: 'was published within the minimumReleaseAge cutoff', + }, + } }) const result = await getManifest({ ...opts, resolve, publishedBy }, 'foo', 'latest') diff --git a/engine/pm/commands/src/self-updater/selfUpdate.ts b/engine/pm/commands/src/self-updater/selfUpdate.ts index d2130b5d52..bc0c435afb 100644 --- a/engine/pm/commands/src/self-updater/selfUpdate.ts +++ b/engine/pm/commands/src/self-updater/selfUpdate.ts @@ -7,7 +7,7 @@ import { docsUrl } from '@pnpm/cli.utils' import { type Config, type ConfigContext, parsePackageManager, types as allTypes } from '@pnpm/config.reader' import { getPublishedByPolicy } from '@pnpm/config.version-policy' import { PnpmError } from '@pnpm/error' -import { createResolver } from '@pnpm/installing.client' +import { createResolver, makeResolutionStrict } from '@pnpm/installing.client' import { resolvePackageManagerIntegrities } from '@pnpm/installing.env-installer' import { readEnvLockfile } from '@pnpm/lockfile.fs' import { globalInfo, globalWarn } from '@pnpm/logger' @@ -80,12 +80,21 @@ export async function handler ( throw new PnpmError('CANT_SELF_UPDATE_IN_COREPACK', 'You should update pnpm with corepack') } globalInfo('Checking for updates...') - const { resolve } = createResolver({ + const { resolve: baseResolve } = createResolver({ ...opts, configByUri: opts.configByUri, - strictPublishedByCheck: Boolean(opts.minimumReleaseAge) && opts.minimumReleaseAgeStrict === true, ignoreMissingTimeField: opts.minimumReleaseAgeIgnoreMissingTime, }) + // self-update has nowhere to "defer to" either — wrap the resolver + // under any policy that wants to reject violations up-front. Strict + // minimumReleaseAge keeps self-update from switching to an immature + // pnpm; `trustPolicy: 'no-downgrade'` keeps it from switching to a + // pnpm whose trust evidence weakened relative to the installed + // version. + const strictResolution = + (Boolean(opts.minimumReleaseAge) && opts.minimumReleaseAgeStrict === true) || + opts.trustPolicy === 'no-downgrade' + const resolve = strictResolution ? makeResolutionStrict(baseResolve) : baseResolve const pkgName = 'pnpm' const { publishedBy, publishedByExclude } = getPublishedByPolicy(opts) // `pnpm self-update` (no args) defaults to the `latest` dist-tag, but we diff --git a/exec/commands/src/dlx.ts b/exec/commands/src/dlx.ts index 5fac706b52..1293f5730a 100644 --- a/exec/commands/src/dlx.ts +++ b/exec/commands/src/dlx.ts @@ -15,7 +15,7 @@ import { type Config, types } from '@pnpm/config.reader' import { getPublishedByPolicy } from '@pnpm/config.version-policy' import { createHexHash } from '@pnpm/crypto.hash' import { PnpmError } from '@pnpm/error' -import { createResolver } from '@pnpm/installing.client' +import { createResolver, makeResolutionStrict } from '@pnpm/installing.client' import { add } from '@pnpm/installing.commands' import { readPackageJsonFromDir } from '@pnpm/pkg-manifest.reader' import { parseWantedDependency } from '@pnpm/resolving.parse-wanted-dependency' @@ -113,12 +113,11 @@ export async function handler ( ) && !opts.registrySupportsTimeField ) const catalogResolver = resolveFromCatalog.bind(null, opts.catalogs ?? {}) - const { resolve } = createResolver({ + const { resolve: baseResolve } = createResolver({ ...opts, configByUri: opts.configByUri, fullMetadata, filterMetadata: fullMetadata, - strictPublishedByCheck: Boolean(opts.minimumReleaseAge) && opts.minimumReleaseAgeStrict === true, ignoreMissingTimeField: opts.minimumReleaseAgeIgnoreMissingTime, retry: { factor: opts.fetchRetryFactor, @@ -128,6 +127,17 @@ export async function handler ( }, timeout: opts.fetchTimeout, }) + // dlx has nowhere to "defer to" — it runs the resolved package directly. + // Wrap the resolver under any policy that wants to reject violations + // up-front: strict minimumReleaseAge (refuse immature picks) and + // `trustPolicy: 'no-downgrade'` (refuse versions whose trust evidence + // weakened). Without the trust-policy arm, a downgraded version would + // resolve to a `policyViolation` that dlx silently ignored and then + // executed. + const strictResolution = + (Boolean(opts.minimumReleaseAge) && opts.minimumReleaseAgeStrict === true) || + opts.trustPolicy === 'no-downgrade' + const resolve = strictResolution ? makeResolutionStrict(baseResolve) : baseResolve const resolvedPkgAliases: string[] = [] const { publishedBy, publishedByExclude } = getPublishedByPolicy(opts) const resolvedPkgs = await Promise.all(pkgs.map(async (pkg) => { diff --git a/exec/commands/test/dlx.e2e.ts b/exec/commands/test/dlx.e2e.ts index a27f692947..520b6a0c36 100644 --- a/exec/commands/test/dlx.e2e.ts +++ b/exec/commands/test/dlx.e2e.ts @@ -482,7 +482,7 @@ test('dlx should fail when the requested package does not meet the minimum age r default: 'https://registry.npmjs.org/', }, }, ['shx@0.3.4']) - ).rejects.toThrow(/Version 0\.3\.4 \(released .+\) of shx does not meet the minimumReleaseAge constraint/) + ).rejects.toThrow(/shx@0\.3\.4 was published.+minimumReleaseAge cutoff/) }) test('dlx should respect minimumReleaseAgeExclude', async () => { diff --git a/installing/client/package.json b/installing/client/package.json index 96b16bec8f..0e35f53dca 100644 --- a/installing/client/package.json +++ b/installing/client/package.json @@ -34,6 +34,7 @@ }, "dependencies": { "@pnpm/engine.runtime.node-resolver": "workspace:*", + "@pnpm/error": "workspace:*", "@pnpm/fetching.binary-fetcher": "workspace:*", "@pnpm/fetching.directory-fetcher": "workspace:*", "@pnpm/fetching.git-fetcher": "workspace:*", @@ -43,6 +44,7 @@ "@pnpm/network.auth-header": "workspace:*", "@pnpm/network.fetch": "workspace:*", "@pnpm/resolving.default-resolver": "workspace:*", + "@pnpm/resolving.npm-resolver": "workspace:*", "@pnpm/resolving.resolver-base": "workspace:*", "@pnpm/store.index": "workspace:*", "@pnpm/types": "workspace:*", diff --git a/installing/client/src/index.ts b/installing/client/src/index.ts index 2673f51d02..f1db78a6e2 100644 --- a/installing/client/src/index.ts +++ b/installing/client/src/index.ts @@ -1,4 +1,5 @@ import { NODE_EXTRAS_IGNORE_PATTERN } from '@pnpm/engine.runtime.node-resolver' +import { PnpmError } from '@pnpm/error' import { createBinaryFetcher } from '@pnpm/fetching.binary-fetcher' import { createDirectoryFetcher } from '@pnpm/fetching.directory-fetcher' import type { BinaryFetcher, DirectoryFetcher, GitFetcher } from '@pnpm/fetching.fetcher-base' @@ -15,7 +16,8 @@ import { type ResolveFunction, type ResolverFactoryOptions, } from '@pnpm/resolving.default-resolver' -import type { ResolutionVerifier } from '@pnpm/resolving.resolver-base' +import { MINIMUM_RELEASE_AGE_VIOLATION_CODE } from '@pnpm/resolving.npm-resolver' +import type { ResolutionPolicyViolation, ResolutionVerifier } from '@pnpm/resolving.resolver-base' import type { StoreIndex } from '@pnpm/store.index' import type { RegistryConfig } from '@pnpm/types' @@ -38,7 +40,15 @@ export type ClientOptions = { preserveAbsolutePaths?: boolean fetchMinSpeedKiBps?: number } & ResolverFactoryOptions & DispatcherOptions - & Pick + & Pick export interface Client { fetchers: Fetchers @@ -74,6 +84,40 @@ export function createResolver (opts: Omit): { reso return _createResolver(fetchFromRegistry, getAuthHeader, { ...opts, customResolvers: opts.customResolvers }) } +/** + * Wraps a `ResolveFunction` so any inline policy violation surfaced by + * the resolver is rethrown as a `PnpmError` instead of being returned on + * the result. Use this from one-shot callers (dlx, self-update) that + * have nowhere to defer a violation to — the install command leaves + * resolution unwrapped because it aggregates violations across the + * whole tree before deciding what to do. + * + * The error mapping is centralized here so future violation codes + * (today: `MINIMUM_RELEASE_AGE_VIOLATION`) get a consistent error code + * across every strict-mode caller without each call site re-translating. + */ +export function makeResolutionStrict (resolve: ResolveFunction): ResolveFunction { + return (async (wantedDependency, opts) => { + const result = await resolve(wantedDependency, opts) + if (result?.policyViolation) { + throw policyViolationToError(result.policyViolation) + } + return result + }) as ResolveFunction +} + +function policyViolationToError (violation: ResolutionPolicyViolation): PnpmError { + const message = `${violation.name}@${violation.version} ${violation.reason}` + // Map the per-violation `code` to the user-facing PnpmError code that + // pre-refactor callers (and `default-reporter`) already recognize. + // Future violation codes get their mapping added here so call sites + // don't have to re-translate. + const errorCode = violation.code === MINIMUM_RELEASE_AGE_VIOLATION_CODE + ? 'NO_MATURE_MATCHING_VERSION' + : violation.code + return new PnpmError(errorCode, message) +} + type Fetchers = { git: GitFetcher directory: DirectoryFetcher diff --git a/installing/client/tsconfig.json b/installing/client/tsconfig.json index 1f246c1ebe..b09f94f5d1 100644 --- a/installing/client/tsconfig.json +++ b/installing/client/tsconfig.json @@ -9,6 +9,9 @@ "../../__typings__/**/*.d.ts" ], "references": [ + { + "path": "../../core/error" + }, { "path": "../../core/types" }, @@ -45,6 +48,9 @@ { "path": "../../resolving/default-resolver" }, + { + "path": "../../resolving/npm-resolver" + }, { "path": "../../resolving/resolver-base" }, diff --git a/installing/commands/package.json b/installing/commands/package.json index f9d418f746..72230fb0a9 100644 --- a/installing/commands/package.json +++ b/installing/commands/package.json @@ -58,6 +58,7 @@ "@pnpm/lockfile.types": "workspace:*", "@pnpm/pkg-manifest.reader": "workspace:*", "@pnpm/pkg-manifest.utils": "workspace:*", + "@pnpm/resolving.npm-resolver": "workspace:*", "@pnpm/resolving.parse-wanted-dependency": "workspace:*", "@pnpm/resolving.resolver-base": "workspace:*", "@pnpm/semver-diff": "catalog:", @@ -80,6 +81,7 @@ "@zkochan/rimraf": "catalog:", "@zkochan/table": "catalog:", "chalk": "catalog:", + "ci-info": "catalog:", "enquirer": "catalog:", "get-npm-tarball-url": "catalog:", "is-subdir": "catalog:", @@ -114,7 +116,6 @@ "@types/ramda": "catalog:", "@types/yarnpkg__lockfile": "catalog:", "@types/zkochan__table": "catalog:", - "ci-info": "catalog:", "delay": "catalog:", "jest-diff": "catalog:", "path-name": "catalog:", diff --git a/installing/commands/src/install.ts b/installing/commands/src/install.ts index 7e7ceea71d..702c6d3c4d 100644 --- a/installing/commands/src/install.ts +++ b/installing/commands/src/install.ts @@ -84,6 +84,11 @@ export const cliOptionsTypes = (): Record => ({ 'fix-lockfile': Boolean, 'resolution-only': Boolean, recursive: Boolean, + // `--no-save` lets `pnpm install` skip writing to package.json / + // pnpm-workspace.yaml. Without registering it here, nopt drops the + // flag, `opts.save` stays undefined, and the auto-add path treats + // it as "save enabled". + save: Boolean, }) export const shorthands: Record = { diff --git a/installing/commands/src/installDeps.ts b/installing/commands/src/installDeps.ts index 4155da7ed9..f837dd0efc 100644 --- a/installing/commands/src/installDeps.ts +++ b/installing/commands/src/installDeps.ts @@ -40,6 +40,7 @@ import { updateWorkspaceManifest } from '@pnpm/workspace.workspace-manifest-writ import { getPinnedVersion } from './getPinnedVersion.js' import { getSaveType } from './getSaveType.js' import { handleIgnoredBuilds } from './handleIgnoredBuilds.js' +import { setupPolicyHandlers } from './policyHandlers.js' import { type CommandFullName, createMatcher, @@ -122,7 +123,8 @@ export type InstallDepsOptions = Pick & CreateStoreControllerOptions & { +> & Partial> +& CreateStoreControllerOptions & { argv: { original: string[] } @@ -267,6 +269,13 @@ export async function installDeps ( applyRuntimeOnFailOverride(manifest, opts.runtimeOnFail) } + // `setupPolicyHandlers` composes the per-policy handlers the install + // needs for the current opts (today: minimumReleaseAge; future: + // trustPolicy UX, license policy, etc.). Returns `undefined` when no + // handler is active so the install skips the empty no-op call at + // every checkpoint when no policies are configured. + const policyHandlers = setupPolicyHandlers(opts) + const installOpts: Omit = { ...opts, // In case installation is done in a multi-package repository @@ -282,6 +291,7 @@ export async function installDeps ( resolutionVerifiers: store.resolutionVerifiers, workspacePackages, preferredVersions: opts.packageVulnerabilityAudit ? preferNonvulnerablePackageVersions(opts.packageVulnerabilityAudit) : undefined, + handleResolutionPolicyViolations: policyHandlers?.handleResolutionPolicyViolations, } let updateMatch: UpdateDepsMatcher | null @@ -340,14 +350,20 @@ export async function installDeps ( rootDir: opts.dir as ProjectRootDir, targetDependenciesField: getSaveType(opts), } - const { updatedCatalogs, updatedProject, ignoredBuilds } = await mutateModulesInSingleProject(mutatedProject, installOpts) + const { updatedCatalogs, updatedProject, ignoredBuilds, resolutionPolicyViolations } = await mutateModulesInSingleProject(mutatedProject, installOpts) if (opts.save !== false) { + // Only pick entries when we'll actually persist. Otherwise the + // info log would claim we added entries the workspace manifest + // never saw, and the next install would re-prompt or fail + // verification. + const policyUpdates = policyHandlers?.pickManifestUpdates(resolutionPolicyViolations) await Promise.all([ writeProjectManifest(updatedProject.manifest), updateWorkspaceManifest(opts.workspaceDir ?? opts.dir, { updatedCatalogs, cleanupUnusedCatalogs: opts.cleanupUnusedCatalogs, allProjects: opts.allProjects, + ...policyUpdates, }), ]) } @@ -365,20 +381,34 @@ export async function installDeps ( return } - const { updatedCatalogs, updatedManifest, ignoredBuilds } = await install(manifest, { + const { updatedCatalogs, updatedManifest, ignoredBuilds, resolutionPolicyViolations } = await install(manifest, { ...installOpts, updatePackageManifest, updateMatching, }) - if (opts.update === true && opts.save !== false) { - await Promise.all([ - writeProjectManifest(updatedManifest), - updateWorkspaceManifest(opts.workspaceDir ?? opts.dir, { - updatedCatalogs, - cleanupUnusedCatalogs: opts.cleanupUnusedCatalogs, - allProjects, - }), - ]) + // `opts.save === false` (e.g. `--no-save`) means "don't persist anything + // from this install" — both package.json and the workspace manifest. + // Skip the pick so the info log doesn't claim entries were added that + // were never written; the next install will resurface them. + if (opts.save !== false) { + const policyUpdates = policyHandlers?.pickManifestUpdates(resolutionPolicyViolations) + if (opts.update === true) { + await Promise.all([ + writeProjectManifest(updatedManifest), + updateWorkspaceManifest(opts.workspaceDir ?? opts.dir, { + updatedCatalogs, + cleanupUnusedCatalogs: opts.cleanupUnusedCatalogs, + allProjects, + ...policyUpdates, + }), + ]) + } else if (policyUpdates != null) { + // Plain `pnpm install` (no --update, no params) wouldn't otherwise touch + // the workspace manifest. Persist the auto-policy patches anyway so any + // loose bypass (today: minimumReleaseAgeExclude) remains explicit on + // subsequent installs. + await updateWorkspaceManifest(opts.workspaceDir ?? opts.dir, policyUpdates) + } } await handleIgnoredBuilds(opts, ignoredBuilds) diff --git a/installing/commands/src/policyHandlers.ts b/installing/commands/src/policyHandlers.ts new file mode 100644 index 0000000000..cb43081eb1 --- /dev/null +++ b/installing/commands/src/policyHandlers.ts @@ -0,0 +1,260 @@ +import { PnpmError } from '@pnpm/error' +import { globalInfo } from '@pnpm/logger' +import { MINIMUM_RELEASE_AGE_VIOLATION_CODE } from '@pnpm/resolving.npm-resolver' +import { isCI } from 'ci-info' +import enquirer from 'enquirer' + +/** + * Shape returned by `installing/deps-installer`'s + * `collectResolutionPolicyViolations` and the inline accumulator on + * the resolveDependencies result. Re-declared locally so the commands + * layer can react without depending on the deps-installer's private + * install types. + * + * Verifier codes (today: `MINIMUM_RELEASE_AGE_VIOLATION` and + * `TRUST_DOWNGRADE`) are the contract surface for downstream UX. + * Each `PolicyHandler` below filters violations by code to decide + * what to do with them (prompt, persist to an exclude list, log, + * abort). + */ +export interface PolicyViolation { + name: string + version: string + code: string + reason: string +} + +/** + * Workspace-manifest patch a per-policy handler can request. Each + * field maps to a `pnpm-workspace.yaml` exclude-list array; the + * install command forwards these to `updateWorkspaceManifest` so the + * workspace writer dedupes and appends them in one pass. + * + * New policies that want auto-persistence add their field here AND + * teach `updateWorkspaceManifest` how to honor it. + */ +export interface WorkspaceManifestPolicyUpdates { + addedMinimumReleaseAgeExcludes?: string[] +} + +/** + * What the install command asks of each registered policy handler. + * Both hooks are optional — a handler that only wants to abort can + * skip `pickManifestUpdates`; a handler that only wants to persist + * can skip `handleResolutionPolicyViolations`. + */ +interface PolicyHandler { + /** + * Runs between `resolveDependencyTree` and `resolvePeers`. Throw to + * abort the install before any lockfile / package.json / + * modules-dir mutation. Receives the full violations list across + * every policy — handlers filter by `code` for their own. + */ + handleResolutionPolicyViolations?: (violations: readonly PolicyViolation[]) => Promise + /** + * Called at the install's tail to assemble the workspace-manifest + * patch. Returns `undefined` (or an empty object) when this + * handler has nothing to persist for the current batch. + */ + pickManifestUpdates?: (violations: readonly PolicyViolation[]) => WorkspaceManifestPolicyUpdates | undefined +} + +/** + * Aggregated plan the install command consumes. The `handleResolutionPolicyViolations` + * call fans out across every registered handler in registration order; + * any handler can throw to abort. `pickManifestUpdates` merges the + * per-handler patches into one bag so the workspace writer runs once. + */ +export interface PolicyHandlersPlan { + handleResolutionPolicyViolations: (violations: readonly PolicyViolation[]) => Promise + pickManifestUpdates: (violations: readonly PolicyViolation[]) => WorkspaceManifestPolicyUpdates | undefined +} + +export interface PolicyHandlersOptions { + minimumReleaseAge?: number + minimumReleaseAgeStrict?: boolean + /** + * Pass `false` for `--no-save` installs. Handlers that would + * persist to the workspace manifest refuse to enter modes where + * approval is durably required (today: strict minimumReleaseAge) + * so the prompt never offers an action it can't honor. + */ + save?: boolean + /** + * Override for CI detection. Defaults to `ci-info`'s `isCI` flag. + */ + ci?: boolean +} + +/** + * Composes the per-policy handlers the install command needs for the + * current opts. Returns `undefined` only when no handler reports + * activity — saves the install command an empty no-op call at every + * checkpoint when no policies are configured. + * + * Today only the minimumReleaseAge handler is registered. Future + * policies (trustPolicy UX, license policy, etc.) plug in by + * exporting a sibling `createPolicyHandler(opts)` and getting + * pushed into the `handlers` list below. + */ +export function setupPolicyHandlers (opts: PolicyHandlersOptions): PolicyHandlersPlan | undefined { + const handlers: PolicyHandler[] = [] + const minimumReleaseAge = createMinimumReleaseAgeHandler(opts) + if (minimumReleaseAge) handlers.push(minimumReleaseAge) + + if (handlers.length === 0) return undefined + + return { + handleResolutionPolicyViolations: async (violations) => { + // Sequential, not parallel: a TTY prompt from handler N would + // race with a different prompt from N+1, and we want a clean + // throw to short-circuit before later handlers ask for input. + for (const handler of handlers) { + if (handler.handleResolutionPolicyViolations) { + // eslint-disable-next-line no-await-in-loop + await handler.handleResolutionPolicyViolations(violations) + } + } + }, + pickManifestUpdates: (violations) => { + const merged: WorkspaceManifestPolicyUpdates = {} + let any = false + for (const handler of handlers) { + if (!handler.pickManifestUpdates) continue + const patch = handler.pickManifestUpdates(violations) + if (patch == null) continue + // Shallow merge — handlers own disjoint fields by convention, + // so there's no collision policy to encode here yet. + for (const [key, value] of Object.entries(patch)) { + if (value == null) continue + ;(merged as Record)[key] = value + any = true + } + } + return any ? merged : undefined + }, + } +} + +/** + * minimumReleaseAge policy handler. + * + * Loose mode (`minimumReleaseAgeStrict: false`) lets the resolver + * install versions newer than the cutoff and auto-persists them to + * `minimumReleaseAgeExclude`. Strict mode + an interactive TTY + * surfaces the full set of immature picks (direct AND transitive) at + * once via a confirm prompt — the install proceeds if the user + * approves, otherwise it aborts before touching the lockfile or + * package.json (#10488). Strict mode in CI or any other non-TTY + * context aborts hard with the same violation list so the failure + * pinpoints every offending entry, not just the first one the + * resolver picked. + * + * Strict mode combined with `--no-save` is rejected up-front — the + * approval prompt promises persistence the install command's + * `opts.save !== false` gate would block, leaving the lockfile + * holding approved-but-unlisted immature picks that the next install + * would reject. + * + * Returns `undefined` when minimumReleaseAge is not active. + */ +function createMinimumReleaseAgeHandler (opts: PolicyHandlersOptions): PolicyHandler | undefined { + if (!opts.minimumReleaseAge) return undefined + const strictMode = opts.minimumReleaseAgeStrict === true + const persistenceEnabled = opts.save !== false + const inCi = opts.ci ?? isCI + const canPrompt = !inCi && Boolean(process.stdin.isTTY) + + return { + handleResolutionPolicyViolations: async (violations) => { + if (!strictMode) return + const immature = filterImmatureViolations(violations) + if (immature.length === 0) return + if (!persistenceEnabled) { + throw new PnpmError( + 'STRICT_MIN_RELEASE_AGE_REQUIRES_SAVE', + 'minimumReleaseAgeStrict cannot be combined with --no-save: ' + + 'approval would require writing to minimumReleaseAgeExclude in pnpm-workspace.yaml, ' + + 'which --no-save prevents.', + { + hint: 'Drop --no-save so the exclude list can be persisted, or set ' + + 'minimumReleaseAgeStrict: false to let the install proceed without prompting ' + + '(the lockfile would still trigger the auto-collect on the next normal install).', + } + ) + } + if (canPrompt) { + await promptForApproval(immature) + } else { + throw failOnImmature(immature) + } + }, + pickManifestUpdates: (violations) => { + const entries = pickImmatureEntries(violations, strictMode) + return entries ? { addedMinimumReleaseAgeExcludes: entries } : undefined + }, + } +} + +function filterImmatureViolations (violations: readonly PolicyViolation[]): PolicyViolation[] { + return violations.filter((v) => v.code === MINIMUM_RELEASE_AGE_VIOLATION_CODE) +} + +function pickImmatureEntries ( + violations: readonly PolicyViolation[], + promptRequired: boolean +): string[] | undefined { + const immature = filterImmatureViolations(violations) + if (immature.length === 0) return undefined + const sorted = [...new Set(immature.map((v) => `${v.name}@${v.version}`))].sort() + // Strict-mode picks already passed through the approval prompt, so + // the log here only confirms what was persisted. Loose-mode picks + // haven't been announced anywhere else, so the same log doubles as + // the discovery notice. + const reason = promptRequired + ? '(approved at the prompt)' + : '(loose mode allowed these immature versions)' + globalInfo( + `Added ${sorted.length} ${sorted.length === 1 ? 'entry' : 'entries'} to minimumReleaseAgeExclude in pnpm-workspace.yaml ` + + `${reason}:\n ${sorted.join('\n ')}` + ) + return sorted +} + +function failOnImmature (immature: readonly PolicyViolation[]): PnpmError { + const sorted = [...immature].sort((a, b) => `${a.name}@${a.version}`.localeCompare(`${b.name}@${b.version}`)) + const list = sorted.map((v) => ` ${v.name}@${v.version} ${v.reason}`).join('\n') + return new PnpmError( + 'NO_MATURE_MATCHING_VERSION', + `${sorted.length} ${sorted.length === 1 ? 'version does' : 'versions do'} not meet the minimumReleaseAge constraint:\n${list}`, + { + hint: 'Run the install interactively to approve these picks, or add them to ' + + 'minimumReleaseAgeExclude in pnpm-workspace.yaml, or wait for the packages ' + + 'to mature past the configured cutoff.', + } + ) +} + +async function promptForApproval (immature: readonly PolicyViolation[]): Promise { + const sorted = [...immature].sort((a, b) => `${a.name}@${a.version}`.localeCompare(`${b.name}@${b.version}`)) + const message = + `${sorted.length} ${sorted.length === 1 ? 'version does' : 'versions do'} not meet the minimumReleaseAge constraint:\n` + + sorted.map((v) => ` ${v.name}@${v.version}`).join('\n') + '\n' + + 'Add to minimumReleaseAgeExclude in pnpm-workspace.yaml and proceed with the install?' + const answer = await enquirer.prompt<{ confirmed: boolean }>({ + type: 'confirm', + name: 'confirmed', + message, + initial: false, + }) + if (!answer.confirmed) { + throw new PnpmError( + 'MINIMUM_RELEASE_AGE_DENIED', + 'Aborted: the immature versions were not approved.', + { + hint: 'Re-run the install without `minimumReleaseAgeStrict: true` to allow these versions, ' + + 'or wait for the packages to mature past the configured cutoff.', + } + ) + } +} diff --git a/installing/commands/src/recursive.ts b/installing/commands/src/recursive.ts index 44f3934021..e7442a7190 100755 --- a/installing/commands/src/recursive.ts +++ b/installing/commands/src/recursive.ts @@ -55,6 +55,7 @@ import pLimit from 'p-limit' import { getPinnedVersion } from './getPinnedVersion.js' import { getSaveType } from './getSaveType.js' import { handleIgnoredBuilds } from './handleIgnoredBuilds.js' +import { type PolicyViolation, setupPolicyHandlers } from './policyHandlers.js' import { createWorkspaceSpecs, updateToWorkspacePackagesFromManifest } from './updateWorkspaceDependencies.js' export type RecursiveOptions = CreateStoreControllerOptions & Pick> = mutatedPkgs.map(async ({ originalManifest, manifest, rootDir }) => { return manifestsByPath[rootDir].writeProjectManifest(originalManifest ?? manifest) }) @@ -309,6 +324,7 @@ export async function recursive ( updatedCatalogs, cleanupUnusedCatalogs: opts.cleanupUnusedCatalogs, allProjects, + ...policyUpdates, })) await Promise.all(promises) } @@ -321,6 +337,10 @@ export async function recursive ( let updatedCatalogs: Catalogs | undefined const allIgnoredBuilds = new Set() + // Each per-project install returns its own slice of lockfile-resolution + // violations; accumulate them here so the post-loop persist step can + // dedup and write a single batch to the workspace manifest. + const allResolutionPolicyViolations: PolicyViolation[] = [] const limitInstallation = pLimit(getWorkspaceConcurrency(opts.workspaceConcurrency)) await Promise.all(pkgPaths.map(async (rootDir) => limitInstallation(async () => { @@ -368,6 +388,7 @@ export async function recursive ( updatedCatalogs?: Catalogs updatedManifest: ProjectManifest ignoredBuilds: IgnoredBuilds | undefined + resolutionPolicyViolations?: PolicyViolation[] } type ActionFunction = (manifest: PackageManifest | ProjectManifest, opts: ActionOpts) => Promise @@ -387,6 +408,7 @@ export async function recursive ( updatedCatalogs: undefined, // there's no reason to add new or update catalogs on `pnpm remove` updatedManifest: mutationResult.updatedProjects[0].manifest, ignoredBuilds: mutationResult.ignoredBuilds, + resolutionPolicyViolations: mutationResult.resolutionPolicyViolations, } } break @@ -402,6 +424,7 @@ export async function recursive ( updatedCatalogs: newCatalogsAddition, updatedManifest: newManifest, ignoredBuilds, + resolutionPolicyViolations, } = await action( manifest, { @@ -433,6 +456,11 @@ export async function recursive ( allIgnoredBuilds.add(depPath) } } + if (resolutionPolicyViolations?.length) { + for (const violation of resolutionPolicyViolations) { + allResolutionPolicyViolations.push(violation) + } + } result[rootDir].status = 'passed' } catch (err: any) { // eslint-disable-line logger.info(err) @@ -453,11 +481,18 @@ export async function recursive ( }) )) await handleIgnoredBuilds(opts, allIgnoredBuilds.size ? allIgnoredBuilds : undefined) - await updateWorkspaceManifest(opts.workspaceDir, { - updatedCatalogs, - cleanupUnusedCatalogs: opts.cleanupUnusedCatalogs, - allProjects, - }) + if (opts.save !== false) { + // Only pick entries when we'll actually persist. Otherwise the + // info log would claim entries were added that the workspace + // manifest never saw, mirroring the gate the shared-lockfile + // branch + installDeps already apply. + await updateWorkspaceManifest(opts.workspaceDir, { + updatedCatalogs, + cleanupUnusedCatalogs: opts.cleanupUnusedCatalogs, + allProjects, + ...policyHandlers?.pickManifestUpdates(allResolutionPolicyViolations), + }) + } if ( !opts.lockfileOnly && !opts.ignoreScripts && ( diff --git a/installing/commands/test/add.ts b/installing/commands/test/add.ts index 7ee435c4fc..b4a9eb634d 100644 --- a/installing/commands/test/add.ts +++ b/installing/commands/test/add.ts @@ -381,7 +381,7 @@ test('minimumReleaseAge with minimumReleaseAgeStrict enabled makes install fail minimumReleaseAge, minimumReleaseAgeStrict: true, linkWorkspacePackages: false, - }, ['is-odd@0.1.1'])).rejects.toThrow(/Version 0\.1\.1 \(released .+\) of is-odd does not meet the minimumReleaseAge constraint/) + }, ['is-odd@0.1.1'])).rejects.toThrow(/is-odd@0\.1\.1 was published.+minimumReleaseAge cutoff/) }) describeOnLinuxOnly('filters optional dependencies based on pnpm.supportedArchitectures.libc', () => { diff --git a/installing/commands/test/policyHandlers.ts b/installing/commands/test/policyHandlers.ts new file mode 100644 index 0000000000..72edd8720b --- /dev/null +++ b/installing/commands/test/policyHandlers.ts @@ -0,0 +1,160 @@ +import { expect, jest, test } from '@jest/globals' + +import { type PolicyViolation, setupPolicyHandlers } from '../lib/policyHandlers.js' + +function violation ( + name: string, + version: string, + code = 'MINIMUM_RELEASE_AGE_VIOLATION' +): PolicyViolation { + return { name, version, code, reason: 'stub reason' } +} + +// Swap `process.stdin.isTTY` for the duration of a test, restoring the +// original descriptor — not just the value — so the property's +// configurability/enumerability shape doesn't leak between tests when +// the host process didn't define an own `isTTY` at all. +function withStdinTTY (value: boolean | undefined, fn: () => void | Promise): void | Promise { + const originalDescriptor = Object.getOwnPropertyDescriptor(process.stdin, 'isTTY') + Object.defineProperty(process.stdin, 'isTTY', { value, configurable: true, writable: true }) + const restore = (): void => { + if (originalDescriptor) { + Object.defineProperty(process.stdin, 'isTTY', originalDescriptor) + } else { + delete (process.stdin as { isTTY?: boolean }).isTTY + } + } + let result: void | Promise + try { + result = fn() + } catch (err) { + restore() + throw err + } + if (result && typeof (result as Promise).then === 'function') { + return (result as Promise).then( + (v) => { + restore(); return v + }, + (err) => { + restore(); throw err + } + ) + } + restore() + return result +} + +test('setupPolicyHandlers returns undefined when no policy is active', () => { + expect(setupPolicyHandlers({})).toBeUndefined() +}) + +test('setupPolicyHandlers returns a plan even when strict mode is on without a TTY', () => { + // Pre-refactor this returned undefined and the resolver did the fail-fast + // throw. Now the plan is always returned: the strict-no-TTY case throws + // from the handler with the full violation list, not just the first + // immature pick the resolver happened to hit. + withStdinTTY(false, () => { + expect(setupPolicyHandlers({ + minimumReleaseAge: 60, + minimumReleaseAgeStrict: true, + ci: false, + })).toBeDefined() + }) +}) + +test('strict no-TTY plan throws from the hook with the full violation list', async () => { + await withStdinTTY(false, async () => { + const plan = setupPolicyHandlers({ + minimumReleaseAge: 60, + minimumReleaseAgeStrict: true, + ci: false, + })! + await expect(plan.handleResolutionPolicyViolations([ + violation('foo', '1.0.0'), + violation('bar', '2.3.4'), + ])).rejects.toMatchObject({ + code: 'ERR_PNPM_NO_MATURE_MATCHING_VERSION', + }) + }) +}) + +test('setupPolicyHandlers returns a plan when ci=false and stdin is a TTY', () => { + withStdinTTY(true, () => { + const plan = setupPolicyHandlers({ + minimumReleaseAge: 60, + minimumReleaseAgeStrict: true, + ci: false, + }) + expect(plan).toBeDefined() + }) +}) + +test('strict + --no-save refuses up-front instead of prompting for approval it cannot persist', async () => { + // The prompt promises to write to minimumReleaseAgeExclude, but the + // install command's `opts.save !== false` gate blocks that under + // --no-save — accepting the prompt would leave the lockfile holding + // approved-but-unlisted picks that the next install rejects. + await withStdinTTY(true, async () => { + const plan = setupPolicyHandlers({ + minimumReleaseAge: 60, + minimumReleaseAgeStrict: true, + save: false, + ci: false, + })! + await expect(plan.handleResolutionPolicyViolations([violation('foo', '1.0.0')])) + .rejects.toMatchObject({ code: 'ERR_PNPM_STRICT_MIN_RELEASE_AGE_REQUIRES_SAVE' }) + }) +}) + +test('loose + --no-save runs the hook as a no-op (lockfile re-triggers auto-collect later)', async () => { + // Loose mode never persists from the hook anyway — `pickManifestUpdates` + // is what writes the exclude list at the install's tail, and the + // installDeps / recursive `opts.save !== false` gates already skip that + // when --no-save is set. + const plan = setupPolicyHandlers({ + minimumReleaseAge: 60, + save: false, + })! + await expect(plan.handleResolutionPolicyViolations([violation('foo', '1.0.0')])) + .resolves.toBeUndefined() +}) + +test('loose-mode plan emits a workspace patch with sorted unique entries and logs once', () => { + const plan = setupPolicyHandlers({ minimumReleaseAge: 60 })! + const violations = [ + violation('foo', '1.0.0'), + violation('foo', '1.0.0'), + violation('bar', '2.3.4'), + // Non-minimumReleaseAge code: the minimumReleaseAge handler ignores it. + // (When more handlers register, each filters its own codes.) + violation('quux', '0.0.1', 'TRUST_DOWNGRADE'), + ] + + // Avoid leaking console output in test runs. + const infoSpy = jest.spyOn(console, 'log').mockImplementation(() => {}) + try { + const updates = plan.pickManifestUpdates(violations) + expect(updates).toEqual({ addedMinimumReleaseAgeExcludes: ['bar@2.3.4', 'foo@1.0.0'] }) + } finally { + infoSpy.mockRestore() + } +}) + +test('pickManifestUpdates returns undefined when no handler contributes anything', () => { + const plan = setupPolicyHandlers({ minimumReleaseAge: 60 })! + expect(plan.pickManifestUpdates([])).toBeUndefined() + // Codes the minimumReleaseAge handler doesn't recognize don't produce a + // patch — and with no other handler registered yet, the merged result + // collapses to undefined so the install command skips the workspace + // writer entirely. + expect(plan.pickManifestUpdates([violation('foo', '1.0.0', 'TRUST_DOWNGRADE')])).toBeUndefined() +}) + +test('the hook is a no-op in loose mode regardless of violations', async () => { + const plan = setupPolicyHandlers({ minimumReleaseAge: 60 })! + // Loose mode never prompts — picks are persisted from + // `pickManifestUpdates` at the end of the install. + await expect(plan.handleResolutionPolicyViolations([violation('foo', '1.0.0')])) + .resolves.toBeUndefined() +}) diff --git a/installing/commands/tsconfig.json b/installing/commands/tsconfig.json index d627195020..9b0de112d2 100644 --- a/installing/commands/tsconfig.json +++ b/installing/commands/tsconfig.json @@ -90,6 +90,9 @@ { "path": "../../pkg-manifest/utils" }, + { + "path": "../../resolving/npm-resolver" + }, { "path": "../../resolving/parse-wanted-dependency" }, diff --git a/installing/deps-installer/src/install/extendInstallOptions.ts b/installing/deps-installer/src/install/extendInstallOptions.ts index 7cf60a611a..d35cb97231 100644 --- a/installing/deps-installer/src/install/extendInstallOptions.ts +++ b/installing/deps-installer/src/install/extendInstallOptions.ts @@ -11,7 +11,7 @@ import type { ProjectOptions } from '@pnpm/installing.context' import type { HoistingLimits } from '@pnpm/installing.deps-restorer' import type { IncludedDependencies } from '@pnpm/installing.modules-yaml' import type { LockfileObject } from '@pnpm/lockfile.fs' -import type { ResolutionVerifier, WorkspacePackages } from '@pnpm/resolving.resolver-base' +import type { ResolutionPolicyViolation, ResolutionVerifier, WorkspacePackages } from '@pnpm/resolving.resolver-base' import type { StoreController } from '@pnpm/store.controller-types' import type { AllowedDeprecatedVersions, @@ -175,6 +175,23 @@ export interface StrictInstallOptions { ci?: boolean minimumReleaseAge?: number minimumReleaseAgeExclude?: string[] + /** + * Resolver-agnostic post-tree gate, invoked between + * `resolveDependencyTree` and `resolvePeers` inside + * `resolveDependencies`. Receives the violations the verifier + * fan-out collected from the freshly-resolved tree. Throwing here + * unwinds the install before peer-dep resolution runs — nothing on + * disk has changed, and the (potentially expensive) peer pass is + * skipped on abort. + * + * Intentionally policy-neutral. Each verifier owns its violation + * codes (`MINIMUM_RELEASE_AGE_VIOLATION`, `TRUST_DOWNGRADE`, …); the + * install command filters by code to decide what to do. Future + * resolvers can plug verifiers in without touching this signature. + */ + handleResolutionPolicyViolations?: ( + violations: readonly ResolutionPolicyViolation[] + ) => Promise /** * Resolver-side verifiers that re-check each lockfile-pinned resolution * against policies configured upstream (today: at most one, diff --git a/installing/deps-installer/src/install/index.ts b/installing/deps-installer/src/install/index.ts index 59d363bd76..0ea17ae628 100644 --- a/installing/deps-installer/src/install/index.ts +++ b/installing/deps-installer/src/install/index.ts @@ -63,6 +63,7 @@ import { createVersionSpecFromResolvedVersion, getAllDependenciesFromManifest, g import { parseWantedDependency } from '@pnpm/resolving.parse-wanted-dependency' import type { PreferredVersions, + ResolutionPolicyViolation, } from '@pnpm/resolving.resolver-base' import type { AllowBuild, @@ -159,6 +160,8 @@ export interface InstallResult { updatedCatalogs: Catalogs | undefined updatedManifest: ProjectManifest ignoredBuilds: IgnoredBuilds | undefined + /** Forwarded from {@link MutateModulesResult.resolutionPolicyViolations}. */ + resolutionPolicyViolations: ResolutionPolicyViolation[] } export async function install ( @@ -173,7 +176,7 @@ export async function install ( return installFromPnpmRegistry(manifest, rootDir, opts) } - const { updatedCatalogs, updatedProjects: projects, ignoredBuilds } = await mutateModules( + const { updatedCatalogs, updatedProjects: projects, ignoredBuilds, resolutionPolicyViolations } = await mutateModules( [ { mutation: 'install', @@ -195,7 +198,7 @@ export async function install ( }], } ) - return { updatedCatalogs, updatedManifest: projects[0].manifest, ignoredBuilds } + return { updatedCatalogs, updatedManifest: projects[0].manifest, ignoredBuilds, resolutionPolicyViolations } } interface ProjectToBeInstalled { @@ -219,6 +222,8 @@ export interface MutateModulesInSingleProjectResult { updatedCatalogs: Catalogs | undefined updatedProject: UpdatedProject ignoredBuilds: IgnoredBuilds | undefined + /** Forwarded from {@link MutateModulesResult.resolutionPolicyViolations}. */ + resolutionPolicyViolations: ResolutionPolicyViolation[] } export async function mutateModulesInSingleProject ( @@ -252,6 +257,7 @@ export async function mutateModulesInSingleProject ( updatedCatalogs: result.updatedCatalogs, updatedProject: result.updatedProjects[0], ignoredBuilds: result.ignoredBuilds, + resolutionPolicyViolations: result.resolutionPolicyViolations, } } @@ -261,6 +267,15 @@ export interface MutateModulesResult { stats: InstallationResultStats depsRequiringBuild?: DepPath[] ignoredBuilds: IgnoredBuilds | undefined + /** + * Resolver-policy violations the post-resolution scan found in the + * freshly-resolved lockfile. Each violation carries a verifier code + * (e.g. `MINIMUM_RELEASE_AGE_VIOLATION`, `TRUST_DOWNGRADE`); the + * install command filters by code to decide what to do (persist to + * `minimumReleaseAgeExclude`, log, etc.). Empty array when no + * verifier reported a violation or no policy was active. + */ + resolutionPolicyViolations: ResolutionPolicyViolation[] } const pickCatalogSpecifier: CatalogResultMatcher = { @@ -452,6 +467,7 @@ export async function mutateModules ( stats: result.stats ?? { added: 0, removed: 0, linkedToRoot: 0 }, depsRequiringBuild: result.depsRequiringBuild, ignoredBuilds, + resolutionPolicyViolations: result.resolutionPolicyViolations ?? [], } interface InnerInstallResult { @@ -460,6 +476,7 @@ export async function mutateModules ( readonly stats?: InstallationResultStats readonly depsRequiringBuild?: DepPath[] readonly ignoredBuilds: IgnoredBuilds | undefined + readonly resolutionPolicyViolations?: ResolutionPolicyViolation[] } async function _install (): Promise { @@ -799,6 +816,7 @@ export async function mutateModules ( stats: result.stats, depsRequiringBuild: result.depsRequiringBuild, ignoredBuilds: result.ignoredBuilds, + resolutionPolicyViolations: result.resolutionPolicyViolations, } } @@ -1158,7 +1176,7 @@ export async function addDependenciesToPackage ( } & InstallMutationOptions ): Promise { const rootDir = (opts.dir ?? process.cwd()) as ProjectRootDir - const { updatedCatalogs, updatedProjects: projects, ignoredBuilds } = await mutateModules( + const { updatedCatalogs, updatedProjects: projects, ignoredBuilds, resolutionPolicyViolations } = await mutateModules( [ { allowNew: opts.allowNew, @@ -1186,7 +1204,7 @@ export async function addDependenciesToPackage ( }, ], }) - return { updatedCatalogs, updatedManifest: projects[0].manifest, ignoredBuilds } + return { updatedCatalogs, updatedManifest: projects[0].manifest, ignoredBuilds, resolutionPolicyViolations } } export type ImporterToUpdate = { @@ -1217,6 +1235,7 @@ interface InstallFunctionResult { stats?: InstallationResultStats depsRequiringBuild: DepPath[] ignoredBuilds?: IgnoredBuilds + resolutionPolicyViolations: ResolutionPolicyViolation[] } type InstallFunction = ( @@ -1331,6 +1350,7 @@ const _installInContext: InstallFunction = async (projects, ctx, opts) => { peerDependencyIssuesByProjects, wantedToBeSkippedPackageIds, waitTillAllFetchingsFinish, + resolutionPolicyViolations, } = await resolveDependencies( projects, { @@ -1387,6 +1407,7 @@ const _installInContext: InstallFunction = async (projects, ctx, opts) => { trustPolicyIgnoreAfter: opts.trustPolicyIgnoreAfter, blockExoticSubdeps: opts.blockExoticSubdeps, allProjectIds: Object.values(ctx.projects).map((p) => p.id), + handleResolutionPolicyViolations: opts.handleResolutionPolicyViolations, } ) if (!opts.include.optionalDependencies || !opts.include.devDependencies || !opts.include.dependencies) { @@ -1748,6 +1769,7 @@ const _installInContext: InstallFunction = async (projects, ctx, opts) => { stats, depsRequiringBuild, ignoredBuilds, + resolutionPolicyViolations, } } @@ -2173,6 +2195,18 @@ async function installFromPnpmRegistry ( opts: Opts, allInstallProjects?: Array<{ rootDir: ProjectRootDir, manifest: ProjectManifest }> ): Promise { + // The agent path skips client-side resolution, so resolver-side policies + // can't be enforced locally. `minimumReleaseAge` is forwarded to the + // agent and enforced server-side. `trustPolicy` has no server-side + // counterpart yet, so refuse to run under it instead of silently + // letting through a lockfile the local verifier would reject. + if (opts.trustPolicy === 'no-downgrade') { + throw new PnpmError( + 'TRUST_POLICY_INCOMPATIBLE_WITH_AGENT', + 'The pnpm agent does not yet enforce `trustPolicy: no-downgrade`, so running an install through the agent under this policy would produce a lockfile that the local verifier rejects.', + { hint: 'Unset `trustPolicy` for this install, or disable the agent (unset `--agent` / `agent` in pnpm-workspace.yaml) so resolution runs locally and the trust check applies.' } + ) + } const { fetchFromPnpmRegistry } = await import('@pnpm/agent.client') const { StoreIndex } = await import('@pnpm/store.index') const { setImportConcurrency } = await import('@pnpm/worker') @@ -2320,6 +2354,12 @@ async function installFromPnpmRegistry ( ignoredBuilds, stats, lockfile, + // Server-side resolution (pnpm agent) enforces `minimumReleaseAge` + // itself — the agent picks only mature versions and the lockfile + // can't contain immature entries to auto-collect. `trustPolicy` is + // guarded above (we refuse to enter this path when it's set), so + // there's nothing for the install command to react to here. + resolutionPolicyViolations: [], } } finally { // Close the storeController to flush queued StoreIndex writes — the diff --git a/installing/deps-installer/src/install/verifyLockfileResolutions.ts b/installing/deps-installer/src/install/verifyLockfileResolutions.ts index a3fe703163..39cb347022 100644 --- a/installing/deps-installer/src/install/verifyLockfileResolutions.ts +++ b/installing/deps-installer/src/install/verifyLockfileResolutions.ts @@ -2,7 +2,11 @@ import { hashObject } from '@pnpm/crypto.object-hasher' import { PnpmError } from '@pnpm/error' import type { LockfileObject } from '@pnpm/lockfile.fs' import { nameVerFromPkgSnapshot } from '@pnpm/lockfile.utils' -import type { Resolution, ResolutionVerifier } from '@pnpm/resolving.resolver-base' +import type { + Resolution, + ResolutionPolicyViolation, + ResolutionVerifier, +} from '@pnpm/resolving.resolver-base' import type { DepPath } from '@pnpm/types' import pLimit from 'p-limit' @@ -11,11 +15,10 @@ import { tryLockfileVerificationCache, } from './verifyLockfileResolutionsCache.js' -interface Violation { - pkgId: string - code: string - reason: string -} +// Re-exported for back-compat with the existing import surface. +// The interface itself lives in resolver-base so deps-resolver can +// participate in the same shape; see the doc there. +export type { ResolutionPolicyViolation } // Cap the per-entry breakdown so a verifier rejecting hundreds of entries // (e.g. a poisoned lockfile) doesn't flood the terminal / CI log; the full @@ -110,49 +113,7 @@ export async function verifyLockfileResolutions ( cachePrecomputed = result.precomputed } - // depPath can include peer-dependency and patch_hash suffixes (e.g. - // `react@18.0.0(peer)(patch_hash=…)`); the same (name, version) pair may - // therefore appear multiple times. Dedupe so we issue at most one - // verification per package version. - // - // Include a serialization of `resolution` in the key so two entries that - // share a (name, version) but differ in *what* was resolved (e.g. one - // pinned via npm, another via a git URL under the same alias) don't - // collapse: if the wrong shape wins the dedup, a protocol-scoped - // verifier short-circuits on the surviving entry and the real one is - // never checked. - const candidates = new Map() - for (const [depPath, snapshot] of Object.entries(lockfile.packages)) { - const { name, version } = nameVerFromPkgSnapshot(depPath as DepPath, snapshot) - if (!name || !version) continue - const key = `${name}@${version}@${JSON.stringify(snapshot.resolution)}` - candidates.set(key, { - name, - version, - resolution: snapshot.resolution as Resolution, - }) - } - - const violations: Violation[] = [] - const limit = pLimit(options?.concurrency ?? DEFAULT_CONCURRENCY) - await Promise.all( - Array.from(candidates.values(), ({ name, version, resolution }) => limit(async () => { - const pkgId = `${name}@${version}` - // Fan out across every active verifier; each handles its own - // protocol short-circuit (e.g. the npm verifier returns ok:true for - // git resolutions). We stop at the first failure per entry so a - // multi-verifier setup doesn't produce duplicate violations for the - // same (name, version). - for (const verifier of verifiers) { - // eslint-disable-next-line no-await-in-loop - const result = await verifier.verify(resolution, { name, version }) - if (!result.ok) { - violations.push({ pkgId, code: result.code, reason: result.reason }) - break - } - } - })) - ) + const violations = await iterateLockfileViolations(lockfile, verifiers, options?.concurrency) if (violations.length === 0) { // Persist the success so the next install can stat-only the lockfile. @@ -167,19 +128,27 @@ export async function verifyLockfileResolutions ( } // Stable order so the error output is deterministic. - violations.sort((a, b) => a.pkgId.localeCompare(b.pkgId)) + violations.sort((a, b) => `${a.name}@${a.version}`.localeCompare(`${b.name}@${b.version}`)) + // Pick the throw code: a single-code batch keeps the per-policy code + // (so existing handlers / docs / search keywords still route correctly); + // a mixed batch (e.g. minimumReleaseAge + trust-downgrade on the same + // lockfile) escalates to the generic `LOCKFILE_RESOLUTION_VERIFICATION` + // and the per-entry code goes into the breakdown so the user can see + // which policy each entry tripped. + const distinctCodes = new Set(violations.map((v) => v.code)) + const isMixed = distinctCodes.size > 1 + const errorCode = isMixed ? 'LOCKFILE_RESOLUTION_VERIFICATION' : violations[0].code const visible = violations.slice(0, MAX_VIOLATIONS_TO_PRINT) const omitted = violations.length - visible.length - const breakdown = visible.map((v) => ` ${v.pkgId} ${v.reason}`).join('\n') + const formatEntry = isMixed + ? (v: ResolutionPolicyViolation): string => ` ${v.name}@${v.version} [${v.code}] ${v.reason}` + : (v: ResolutionPolicyViolation): string => ` ${v.name}@${v.version} ${v.reason}` + const breakdown = visible.map(formatEntry).join('\n') const details = omitted > 0 ? `${breakdown}\n …and ${omitted} more` : breakdown - // Use the code of the first violation — all of today's violations are the - // same shape (one verifier, one code). If multiple verifiers fire later - // with mixed codes, switch to a generic LOCKFILE_RESOLUTION_VERIFICATION - // code and list per-entry codes in the breakdown. throw new PnpmError( - violations[0].code, + errorCode, `${violations.length} lockfile entries failed verification:\n${details}`, { hint: 'The lockfile contains entries that the active policies reject. ' + @@ -192,3 +161,76 @@ export async function verifyLockfileResolutions ( } ) } + +/** + * Collect-mode sibling of {@link verifyLockfileResolutions}: runs the + * same fan-out over every verifier and every lockfile entry, but + * returns the violations as data instead of throwing on the first batch. + * No cache lookup or write — the throw-mode `verifyLockfileResolutions` + * is what populates / honors the cache; this is for callers that need + * to inspect violations (auto-collect into `minimumReleaseAgeExclude`, + * the strict-mode interactive prompt, future resolver-specific + * policies). + * + * Returns an empty array when `verifiers` is empty or the lockfile has + * no packages, so callers don't need a separate emptiness check. + */ +export async function collectResolutionPolicyViolations ( + lockfile: LockfileObject, + verifiers: ResolutionVerifier[], + options?: Pick +): Promise { + if (verifiers.length === 0) return [] + if (!lockfile.packages) return [] + return iterateLockfileViolations(lockfile, verifiers, options?.concurrency) +} + +async function iterateLockfileViolations ( + lockfile: LockfileObject, + verifiers: readonly ResolutionVerifier[], + concurrency: number | undefined +): Promise { + // depPath can include peer-dependency and patch_hash suffixes (e.g. + // `react@18.0.0(peer)(patch_hash=…)`); the same (name, version) pair may + // therefore appear multiple times. Dedupe so we issue at most one + // verification per package version. + // + // Include a serialization of `resolution` in the key so two entries that + // share a (name, version) but differ in *what* was resolved (e.g. one + // pinned via npm, another via a git URL under the same alias) don't + // collapse: if the wrong shape wins the dedup, a protocol-scoped + // verifier short-circuits on the surviving entry and the real one is + // never checked. + const candidates = new Map() + for (const [depPath, snapshot] of Object.entries(lockfile.packages ?? {})) { + const { name, version } = nameVerFromPkgSnapshot(depPath as DepPath, snapshot) + if (!name || !version) continue + const key = `${name}@${version}@${JSON.stringify(snapshot.resolution)}` + candidates.set(key, { + name, + version, + resolution: snapshot.resolution as Resolution, + }) + } + + const violations: ResolutionPolicyViolation[] = [] + const limit = pLimit(concurrency ?? DEFAULT_CONCURRENCY) + await Promise.all( + Array.from(candidates.values(), ({ name, version, resolution }) => limit(async () => { + // Fan out across every active verifier; each handles its own + // protocol short-circuit (e.g. the npm verifier returns ok:true for + // git resolutions). We stop at the first failure per entry so a + // multi-verifier setup doesn't produce duplicate violations for the + // same (name, version). + for (const verifier of verifiers) { + // eslint-disable-next-line no-await-in-loop + const result = await verifier.verify(resolution, { name, version }) + if (!result.ok) { + violations.push({ name, version, resolution, code: result.code, reason: result.reason }) + break + } + } + })) + ) + return violations +} diff --git a/installing/deps-installer/test/install/minimumReleaseAge.ts b/installing/deps-installer/test/install/minimumReleaseAge.ts index 14c8842a6d..63310eaf5f 100644 --- a/installing/deps-installer/test/install/minimumReleaseAge.ts +++ b/installing/deps-installer/test/install/minimumReleaseAge.ts @@ -72,21 +72,33 @@ test('minimumReleaseAge falls back to immature version when no mature version sa // The fallback picks the lowest matching version (0.1.0), which differs from // normal resolution without minimumReleaseAge that would pick the highest (0.1.2). const opts = testDefaults({ minimumReleaseAge: allImmatureMinimumReleaseAge }) - const { updatedManifest: manifest } = await addDependenciesToPackage({}, ['is-odd@0.1'], opts) + const { updatedManifest: manifest } = await addDependenciesToPackage({}, ['is-odd@0.1'], { + ...opts, + // Acknowledge the policy violations without aborting — this test + // only inspects the resolved manifest. resolveDependencies refuses + // to proceed if violations fire and no handler is wired. + handleResolutionPolicyViolations: async () => {}, + }) expect(manifest.dependencies!['is-odd']).toBe('~0.1.0') }) -test('minimumReleaseAge throws when no mature version satisfies the range and strict mode is enabled', async () => { +test('strict minimumReleaseAge surfaces every immature pick via handleResolutionPolicyViolations, then aborts', async () => { + // Pre-refactor strict mode threw at the resolver on the first immature + // pick (forcing a discover-by-loop dance, #10488). With always-defer the + // resolver records every immature pick inline; the install command (here + // simulated via the hook) decides what to do once it has the full set. prepareEmpty() - - await expect(async () => { - const opts = testDefaults( - { minimumReleaseAge: allImmatureMinimumReleaseAge }, - { strictPublishedByCheck: true } - ) - await addDependenciesToPackage({}, ['is-odd@0.1'], opts) - }).rejects.toThrow(/does not meet the minimumReleaseAge constraint/) + const opts = testDefaults({ minimumReleaseAge: allImmatureMinimumReleaseAge }) + const seen: string[] = [] + await expect(addDependenciesToPackage({}, ['is-odd@0.1'], { + ...opts, + handleResolutionPolicyViolations: async (violations) => { + for (const v of violations) seen.push(`${v.name}@${v.version}`) + throw new Error('immature picks rejected') + }, + })).rejects.toThrow(/immature picks rejected/) + expect(seen).toContain('is-odd@0.1.0') }) test('time-based resolution repopulates missing lockfile time entries on re-install', async () => { @@ -180,17 +192,131 @@ test('minimumReleaseAge is enforced on pre-existing lockfile entries during pnpm ).rejects.toThrow(/minimumReleaseAge/) }) -test('the lockfile minimumReleaseAge gate is inert when strict mode is off (default-value semantics)', async () => { +test('the lockfile minimumReleaseAge gate runs in loose mode too', async () => { prepareEmpty() const { updatedManifest: manifest } = await addDependenciesToPackage({}, ['is-odd@0.1.2'], testDefaults()) expect(manifest.dependencies!['is-odd']).toBe('0.1.2') - // Without explicit strict mode — the same shape as the CLI built-in default - // (1-day release-age window applied without `minimumReleaseAge` being set in - // .npmrc) — the revalidation pass stays inert and the locked version - // installs cleanly. + // Loose mode no longer skips the verifier — once auto-collect makes every + // accepted-immature pin explicit in `minimumReleaseAgeExclude`, running + // the verifier in loose mode is what keeps the manifest in sync with the + // lockfile. A pre-existing immature lockfile entry that isn't yet on the + // exclude list is rejected here, same as strict mode. await expect( install(manifest, testDefaults({ minimumReleaseAge })) + ).rejects.toThrow(/minimumReleaseAge/) +}) + +test('the lockfile minimumReleaseAge gate accepts loose-mode entries already on the exclude list', async () => { + prepareEmpty() + + const { updatedManifest: manifest } = await addDependenciesToPackage({}, ['is-odd@0.1.2'], testDefaults()) + + // is-odd@0.1.2 pulls in is-buffer and kind-of transitively. With the exclude + // list pre-populated (as the auto-collect would have produced on a previous + // install), the loose-mode verifier accepts all three and the install + // completes — the steady-state shape this feature is built around. + await expect( + install(manifest, testDefaults({ + minimumReleaseAge, + minimumReleaseAgeStrict: false, + minimumReleaseAgeExclude: ['is-odd@0.1.2', 'is-buffer', 'kind-of'], + })) ).resolves.toBeDefined() }) + +test('loose mode surfaces immature fresh picks in the install result', async () => { + prepareEmpty() + + // Every version is younger than the cutoff. With strict mode off the + // resolver's lowest-version fallback installs the immature version, + // and the post-resolution scan in `mutateModules` reports it back via + // `resolutionPolicyViolations`. The CLI command filters by code to + // persist the entries to `minimumReleaseAgeExclude`. + const opts = testDefaults({ minimumReleaseAge: allImmatureMinimumReleaseAge }) + const result = await addDependenciesToPackage({}, ['is-odd@0.1'], { + ...opts, + // Acknowledge the violations without aborting so the install + // proceeds and the result can be inspected. + handleResolutionPolicyViolations: async () => {}, + }) + + expect(result.resolutionPolicyViolations).toContainEqual( + expect.objectContaining({ + name: 'is-odd', + version: '0.1.0', + code: 'MINIMUM_RELEASE_AGE_VIOLATION', + }) + ) +}) + +test('versions excluded via minimumReleaseAgeExclude are not surfaced as violations', async () => { + prepareEmpty() + + const opts = testDefaults({ + minimumReleaseAge: allImmatureMinimumReleaseAge, + minimumReleaseAgeExclude: ['is-odd'], + }) + // is-odd is excluded, but `is-odd@0.1.2` pulls in is-buffer / is-number / + // kind-of transitively — those still produce policy violations. Wire a + // no-op handler to acknowledge them. + const result = await addDependenciesToPackage({}, ['is-odd@0.1'], { + ...opts, + handleResolutionPolicyViolations: async () => {}, + }) + + // is-odd is excluded by policy — the install installed 0.1.2 (the highest in + // range) treating it as fully trusted. The verifier short-circuits on the + // excluded entry, so it doesn't end up in the violations array — otherwise + // every install would re-add the same exclude entry the user just dismissed. + expect(result.resolutionPolicyViolations.find((v) => v.name === 'is-odd')).toBeUndefined() +}) + +test('handleResolutionPolicyViolations throwing aborts the install before the lockfile is written', async () => { + // Simulates the strict-mode interactive prompt rejecting the immature + // picks. The hook runs after the new lockfile is built but before it's + // written to disk; throwing unwinds the install in its pre-install state. + prepareEmpty() + const opts = testDefaults({ minimumReleaseAge: allImmatureMinimumReleaseAge }) + await expect(addDependenciesToPackage({}, ['is-odd@0.1'], { + ...opts, + handleResolutionPolicyViolations: async () => { + throw new Error('user denied') + }, + })).rejects.toThrow(/user denied/) + + // The lockfile must NOT have been written — the throw fires before the + // resolver finishes, so no on-disk side effects. + await expect(readWantedLockfile('.', { ignoreIncompatible: false })).resolves.toBeNull() +}) + +test('resolveDependencies throws if violations fire but no handleResolutionPolicyViolations is wired', async () => { + // Safety net: the policy contract is "every pick that trips a check + // produces a violation that gets handled". A caller that opted into a + // policy but forgot to wire the handler would otherwise silently drop + // the violations and land policy-rejected versions in the lockfile. + prepareEmpty() + const opts = testDefaults({ minimumReleaseAge: allImmatureMinimumReleaseAge }) + await expect(addDependenciesToPackage({}, ['is-odd@0.1'], { + ...opts, + // Explicitly omit handleResolutionPolicyViolations. + handleResolutionPolicyViolations: undefined, + })).rejects.toMatchObject({ code: 'ERR_PNPM_RESOLUTION_POLICY_VIOLATIONS_UNHANDLED' }) +}) + +test('handleResolutionPolicyViolations approval lets the install proceed cleanly', async () => { + prepareEmpty() + const opts = testDefaults({ minimumReleaseAge: allImmatureMinimumReleaseAge }) + const result = await addDependenciesToPackage({}, ['is-odd@0.1'], { + ...opts, + handleResolutionPolicyViolations: async (violations) => { + // The real install command would inspect the violations and run + // an enquirer prompt here. The test just confirms the hook gets a + // full set and returns to approve. + expect(violations.some((v) => v.name === 'is-odd' && v.version === '0.1.0')).toBe(true) + }, + }) + + expect(result.updatedManifest.dependencies!['is-odd']).toBe('~0.1.0') +}) diff --git a/installing/deps-installer/test/install/verifyLockfileResolutions.ts b/installing/deps-installer/test/install/verifyLockfileResolutions.ts index a340e3261d..7b1bf0adbb 100644 --- a/installing/deps-installer/test/install/verifyLockfileResolutions.ts +++ b/installing/deps-installer/test/install/verifyLockfileResolutions.ts @@ -68,6 +68,29 @@ test('throws with the verifier-supplied code and reason on a single failure', as }) }) +test('throws a generic code with per-entry codes in the breakdown when violations span policies', async () => { + const lockfile = makeLockfile({ + 'is-odd@0.1.2': { resolution: tarballResolution('sha512-a') }, + 'untrusted@1.0.0': { resolution: tarballResolution('sha512-b') }, + }) + const verifier = wrap(async (_, { name }) => { + if (name === 'is-odd') { + return { ok: false, code: 'MINIMUM_RELEASE_AGE_VIOLATION', reason: 'too fresh' } + } + return { ok: false, code: 'TRUST_DOWNGRADE', reason: 'trust weakened' } + }) + + await expect(verifyLockfileResolutions(lockfile, [verifier])).rejects.toMatchObject({ + // Mixed-code batch escalates to the generic LOCKFILE_RESOLUTION_VERIFICATION + // code so downstream handlers don't mis-route on whichever entry happened + // to land first. + code: 'ERR_PNPM_LOCKFILE_RESOLUTION_VERIFICATION', + // Per-entry code is included in the breakdown so the user can see + // which policy each line tripped. + message: expect.stringMatching(/is-odd@0\.1\.2 \[MINIMUM_RELEASE_AGE_VIOLATION\][\s\S]*untrusted@1\.0\.0 \[TRUST_DOWNGRADE\]/), + }) +}) + test('lists violations in stable order across multiple failures', async () => { const lockfile = makeLockfile({ 'fresh-b@2.0.0': { resolution: tarballResolution('sha512-b') }, @@ -157,14 +180,17 @@ test('the verifier sees the resolution shape verbatim', async () => { expect(received).toEqual(expect.arrayContaining([npmResolution, gitResolution])) }) -test('uses the first violation\'s code when multiple verifiers fire', async () => { +test('keeps the per-policy code when every violation in the batch shares it', async () => { + // Same code across all violations → throw with that code so existing + // handlers / docs / search routes still match. Mixed-code coverage is + // in the dedicated "throws a generic code …" test above. const lockfile = makeLockfile({ 'a@1.0.0': { resolution: tarballResolution('sha512-a') }, 'b@1.0.0': { resolution: tarballResolution('sha512-b') }, }) - const verifier = wrap(async (_, { name }) => ({ + const verifier = wrap(async () => ({ ok: false, - code: name === 'a' ? 'POLICY_A' : 'POLICY_B', + code: 'POLICY_A', reason: 'failed', })) diff --git a/installing/deps-resolver/src/index.ts b/installing/deps-resolver/src/index.ts index d00287a1ba..fc6756a65e 100644 --- a/installing/deps-resolver/src/index.ts +++ b/installing/deps-resolver/src/index.ts @@ -6,6 +6,7 @@ import { } from '@pnpm/core-loggers' import { findRuntimeNodeVersion, iterateHashedGraphNodes } from '@pnpm/deps.graph-hasher' import { isRuntimeDepPath } from '@pnpm/deps.path' +import { PnpmError } from '@pnpm/error' import type { LockfileObject, ProjectSnapshot, @@ -16,6 +17,7 @@ import { getAllDependenciesFromManifest, getSpecFromPackageManifest, } from '@pnpm/pkg-manifest.utils' +import type { ResolutionPolicyViolation } from '@pnpm/resolving.resolver-base' import { type AllowBuild, DEPENDENCIES_FIELDS, @@ -110,6 +112,17 @@ export interface ResolveDependenciesResult { peerDependencyIssuesByProjects: PeerDependencyIssuesByProjects waitTillAllFetchingsFinish: () => Promise wantedToBeSkippedPackageIds: Set + /** + * Policy violations collected inline during resolution — each + * resolver pushes to the list whenever it picks a version that + * trips one of its own checks (today: `minimumReleaseAge`). The + * install command reacts via `handleResolutionPolicyViolations` + * (prompt / abort) and `mutateModules` forwards the array out so + * the auto-persist path at the install's tail can drain it into + * the workspace manifest. Empty when no policy is active or no + * pick violates. + */ + resolutionPolicyViolations: ResolutionPolicyViolation[] } export async function resolveDependencies ( @@ -127,6 +140,17 @@ export async function resolveDependencies ( allowUnusedPatches?: boolean enableGlobalVirtualStore?: boolean allProjectIds: string[] + /** + * Generic checkpoint invoked between `resolveDependencyTree` and + * `resolvePeers` once any inline-collected policy violations have + * been gathered. Callers can prompt, persist, or throw based on + * the violations. Throwing unwinds before any peer-dep work, + * lockfile write, package.json update, or modules-dir change. + * Intentionally policy-neutral: each resolver owns its violation + * codes and the hook implementer (install command) decides what + * to do with them. + */ + handleResolutionPolicyViolations?: (violations: readonly ResolutionPolicyViolation[]) => Promise } ): Promise { const _toResolveImporter = toResolveImporter.bind(null, { @@ -148,8 +172,36 @@ export async function resolveDependencies ( appliedPatches, time, allPeerDepNames, + resolutionPolicyViolations, } = await resolveDependencyTree(projectsToResolve, opts) + // Resolver-policy gate between main resolution and peer-dep + // resolution: every resolver records its own policy violations + // inline as it picks each version, and we hand the accumulated + // list to the install command's hook. The hook throws to abort + // cleanly — nothing on disk has changed yet, and we haven't paid + // the cost of peer resolution. Dispatch stays policy-neutral: each + // resolver owns its violation codes, and the hook implementer + // decides what to do with them. + // + // If violations fired but no hook was wired, throw rather than + // silently dropping them — the resolver-policy contract is "every + // pick that trips a check produces a violation that gets handled"; + // a missing handler means the caller forgot to opt in and would + // otherwise see policy-rejected versions land in the lockfile. + if (resolutionPolicyViolations.length > 0) { + if (!opts.handleResolutionPolicyViolations) { + throw new PnpmError( + 'RESOLUTION_POLICY_VIOLATIONS_UNHANDLED', + `${resolutionPolicyViolations.length} resolution-policy ${resolutionPolicyViolations.length === 1 ? 'violation was' : 'violations were'} produced but no handleResolutionPolicyViolations callback was wired to react to them.`, + { + hint: 'Internal: resolveDependencies needs a handleResolutionPolicyViolations callback whenever a policy that can produce violations (today: minimumReleaseAge) is active. Wire setupPolicyHandlers (in @pnpm/installing.commands) or supply a callback directly.', + } + ) + } + await opts.handleResolutionPolicyViolations(resolutionPolicyViolations) + } + opts.storeController.clearResolutionCache() // We only check whether patches were applied in cases when the whole lockfile was reanalyzed. @@ -358,6 +410,7 @@ export async function resolveDependencies ( peerDependencyIssuesByProjects, waitTillAllFetchingsFinish, wantedToBeSkippedPackageIds, + resolutionPolicyViolations, } } diff --git a/installing/deps-resolver/src/resolveDependencies.ts b/installing/deps-resolver/src/resolveDependencies.ts index 1142afe759..6cb91c7acb 100644 --- a/installing/deps-resolver/src/resolveDependencies.ts +++ b/installing/deps-resolver/src/resolveDependencies.ts @@ -28,6 +28,7 @@ import { type PkgResolutionId, type PreferredVersions, type Resolution, + type ResolutionPolicyViolation, type WorkspacePackages, } from '@pnpm/resolving.resolver-base' import type { @@ -187,6 +188,16 @@ export interface ResolutionContext { hoistPeers?: boolean maximumPublishedBy?: Date publishedByExclude?: PackageVersionPolicy + /** + * Shared accumulator the resolver pushes into when an inline policy + * check (today: minimumReleaseAge in `npm-resolver`) flags a pick. + * resolveDependencyTree hands the populated array back to the install + * command via its return so the post-tree gate can prompt / abort / + * persist without re-walking the resolved tree. Each verifier code + * (`MINIMUM_RELEASE_AGE_VIOLATION`, `TRUST_DOWNGRADE`, …) is the + * contract surface for downstream UX. + */ + resolutionPolicyViolations: ResolutionPolicyViolation[] trustPolicy?: TrustPolicy trustPolicyExclude?: PackageVersionPolicy trustPolicyIgnoreAfter?: number @@ -1373,7 +1384,7 @@ async function resolveDependency ( bareSpecifier: wantedDependency.bareSpecifier, version: wantedDependency.alias ? wantedDependency.bareSpecifier : undefined, } - if (wantedDependency.optional && err.code !== 'ERR_PNPM_TRUST_DOWNGRADE' && err.code !== 'ERR_PNPM_NO_MATURE_MATCHING_VERSION') { + if (wantedDependency.optional && err.code !== 'ERR_PNPM_TRUST_DOWNGRADE') { skippedOptionalDependencyLogger.debug({ details: err.toString(), package: wantedDependencyDetails, @@ -1398,6 +1409,14 @@ async function resolveDependency ( }, }) + // Resolver-inline policy violations (e.g. minimumReleaseAge) flow up + // here; collect them onto the shared context so resolveDependencyTree + // can hand the full set to the install command between + // resolveDependencyTree and resolvePeers. + if (pkgResponse.body.policyViolation) { + ctx.resolutionPolicyViolations.push(pkgResponse.body.policyViolation) + } + // Check if exotic dependencies are disallowed in subdependencies if ( ctx.blockExoticSubdeps && diff --git a/installing/deps-resolver/src/resolveDependencyTree.ts b/installing/deps-resolver/src/resolveDependencyTree.ts index 404284882d..5b522f3448 100644 --- a/installing/deps-resolver/src/resolveDependencyTree.ts +++ b/installing/deps-resolver/src/resolveDependencyTree.ts @@ -5,7 +5,7 @@ import type { LockfileObject } from '@pnpm/lockfile.types' import { globalWarn } from '@pnpm/logger' import type { PatchGroupRecord } from '@pnpm/patching.config' import { BUILTIN_NAMED_REGISTRIES } from '@pnpm/resolving.npm-resolver' -import type { PreferredVersions, Resolution, WorkspacePackages } from '@pnpm/resolving.resolver-base' +import type { PreferredVersions, Resolution, ResolutionPolicyViolation, WorkspacePackages } from '@pnpm/resolving.resolver-base' import type { StoreController } from '@pnpm/store.controller-types' import type { AllowBuild, @@ -159,6 +159,14 @@ export interface ResolveDependencyTreeResult { wantedToBeSkippedPackageIds: Set appliedPatches: Set time?: Record + /** + * Policy violations collected inline during resolution — the + * resolver pushes to this list whenever it picks a package that + * trips one of its own checks (today: `minimumReleaseAge`). The + * shape mirrors `ResolutionPolicyViolation`; downstream callers + * filter by `code` to decide what to do. + */ + resolutionPolicyViolations: ResolutionPolicyViolation[] } export async function resolveDependencyTree ( @@ -220,6 +228,7 @@ export async function resolveDependencyTree ( trustPolicyExclude: opts.trustPolicyExclude ? createPackageVersionPolicyOrThrow(opts.trustPolicyExclude, 'trustPolicyExclude') : undefined, trustPolicyIgnoreAfter: opts.trustPolicyIgnoreAfter, blockExoticSubdeps: opts.blockExoticSubdeps, + resolutionPolicyViolations: [], } const resolveArgs: ImporterToResolve[] = importers.map((importer) => { @@ -343,6 +352,7 @@ export async function resolveDependencyTree ( appliedPatches: ctx.appliedPatches, time, allPeerDepNames: ctx.allPeerDepNames, + resolutionPolicyViolations: ctx.resolutionPolicyViolations, } } diff --git a/installing/package-requester/src/packageRequester.ts b/installing/package-requester/src/packageRequester.ts index 97b0e118d2..7547a0ec56 100644 --- a/installing/package-requester/src/packageRequester.ts +++ b/installing/package-requester/src/packageRequester.ts @@ -193,6 +193,7 @@ async function resolveAndFetch ( publishedAt, normalizedBareSpecifier, alias, + policyViolation, } = resolveResult // Check if the integrity has changed between the current and newly resolved package @@ -256,6 +257,7 @@ async function resolveAndFetch ( updated, publishedAt, alias, + policyViolation, }, } } @@ -319,6 +321,7 @@ async function resolveAndFetch ( updated, publishedAt, alias, + policyViolation, }, fetching: fetchResult.fetching, filesIndexFile: fetchResult.filesIndexFile, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 43ccb39b7e..8d51207e77 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -5138,6 +5138,9 @@ importers: '@pnpm/engine.runtime.node-resolver': specifier: workspace:* version: link:../../engine/runtime/node-resolver + '@pnpm/error': + specifier: workspace:* + version: link:../../core/error '@pnpm/fetching.binary-fetcher': specifier: workspace:* version: link:../../fetching/binary-fetcher @@ -5165,6 +5168,9 @@ importers: '@pnpm/resolving.default-resolver': specifier: workspace:* version: link:../../resolving/default-resolver + '@pnpm/resolving.npm-resolver': + specifier: workspace:* + version: link:../../resolving/npm-resolver '@pnpm/resolving.resolver-base': specifier: workspace:* version: link:../../resolving/resolver-base @@ -5271,6 +5277,9 @@ importers: '@pnpm/pkg-manifest.utils': specifier: workspace:* version: link:../../pkg-manifest/utils + '@pnpm/resolving.npm-resolver': + specifier: workspace:* + version: link:../../resolving/npm-resolver '@pnpm/resolving.parse-wanted-dependency': specifier: workspace:* version: link:../../resolving/parse-wanted-dependency @@ -5337,6 +5346,9 @@ importers: chalk: specifier: 'catalog:' version: 5.6.2 + ci-info: + specifier: 'catalog:' + version: 4.4.0 enquirer: specifier: 'catalog:' version: 2.4.1 @@ -5422,9 +5434,6 @@ importers: '@types/zkochan__table': specifier: 'catalog:' version: '@types/table@6.0.0' - ci-info: - specifier: 'catalog:' - version: 4.4.0 delay: specifier: 'catalog:' version: 7.0.0 diff --git a/pnpm/test/dlx.ts b/pnpm/test/dlx.ts index 1d3f65c3da..aa2f965e01 100644 --- a/pnpm/test/dlx.ts +++ b/pnpm/test/dlx.ts @@ -108,7 +108,7 @@ describe('minimumReleaseAge from pnpm-workspace.yaml', () => { ], { omitEnvDefaults: ['pnpm_config_minimum_release_age'] }) expect(result.status).toBe(1) - expect(result.stderr.toString()).toMatch(/does not meet the minimumReleaseAge constraint/) + expect(result.stderr.toString()).toMatch(/was published.+minimumReleaseAge cutoff/) }) test('dlx succeeds when the requested version is older than minimumReleaseAge', () => { @@ -172,7 +172,7 @@ skipOnWindows('pnpm create respects minimumReleaseAge from pnpm-workspace.yaml', ], { omitEnvDefaults: ['pnpm_config_minimum_release_age'] }) expect(result.status).toBe(1) - expect(result.stderr.toString()).toMatch(/does not meet the minimumReleaseAge constraint/) + expect(result.stderr.toString()).toMatch(/was published.+minimumReleaseAge cutoff/) }) describe('catalogs inherited from pnpm-workspace.yaml', () => { diff --git a/pnpm/test/install/minimumReleaseAge.ts b/pnpm/test/install/minimumReleaseAge.ts index 077808fe37..0cf4e54dfc 100644 --- a/pnpm/test/install/minimumReleaseAge.ts +++ b/pnpm/test/install/minimumReleaseAge.ts @@ -2,7 +2,8 @@ import fs from 'node:fs' import path from 'node:path' import { describe, expect, test } from '@jest/globals' -import { prepare } from '@pnpm/prepare' +import { prepare, preparePackages } from '@pnpm/prepare' +import { readYamlFileSync } from 'read-yaml-file' import { writeYamlFileSync } from 'write-yaml-file' import { execPnpm, execPnpmSync } from '../utils/index.js' @@ -119,11 +120,12 @@ describe('lockfile minimumReleaseAge verification', () => { ) }) - test('install is unaffected by minimumReleaseAge when strict mode is explicitly off', () => { - // The config reader auto-enables strict mode the moment a user - // explicitly sets `minimumReleaseAge`, so opting out requires an - // explicit `minimumReleaseAgeStrict: false`. With that, the verifier - // doesn't construct and the lockfile passes through untouched. + test('loose mode rejects immature lockfile entries that are not on minimumReleaseAgeExclude', () => { + // The verifier now runs in loose mode too, so a lockfile produced under + // no policy that still has immature pins is rejected the same way + // strict mode would reject it. The expected workflow is: the loose-mode + // auto-collect (during fresh resolution) populates the exclude list, and + // subsequent installs run cleanly against that list. prepare({ dependencies: { 'is-odd': '0.1.2' }, }) @@ -134,9 +136,218 @@ describe('lockfile minimumReleaseAge verification', () => { minimumReleaseAgeStrict: false, }) + const result = execPnpmSync( + [PUBLIC_REGISTRY, 'install', '--frozen-lockfile'], + omitMinReleaseAgeEnv + ) + expect(result.status).toBe(1) + const output = `${result.stdout.toString()}\n${result.stderr.toString()}` + expect(output).toContain('ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION') + expect(output).toMatch(/is-odd@0\.1\.2/) + }) + + test('loose mode auto-adds fresh immature picks to minimumReleaseAgeExclude', () => { + // Fresh resolution under loose mode: the resolver's lowest-version + // fallback picks an immature version, and the install layer surfaces it + // to `minimumReleaseAgeExclude`. The verifier sees an empty lockfile at + // the start (no entries to reject) and the workspace manifest grows. + prepare({ + dependencies: { 'is-odd': '0.1.2' }, + }) + writeYamlFileSync('pnpm-workspace.yaml', { + minimumReleaseAge: IMMATURE_FOR_EVERYTHING, + minimumReleaseAgeStrict: false, + }) + + execPnpmSync( + [PUBLIC_REGISTRY, 'install'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + + const workspaceManifest = readYamlFileSync<{ minimumReleaseAgeExclude?: string[] }>('pnpm-workspace.yaml') + // is-odd@0.1.2 pulls in is-buffer, is-number, and kind-of transitively; + // every one of those is immature relative to the (deliberately extreme) + // cutoff, so all four end up on the exclude list. Match by package name + // (any version) so the test stays stable across npm-registry republishes + // that shift the transitive pins. + expect(workspaceManifest.minimumReleaseAgeExclude).toEqual(expect.arrayContaining([ + 'is-odd@0.1.2', + expect.stringMatching(/^is-buffer@/), + expect.stringMatching(/^is-number@/), + expect.stringMatching(/^kind-of@/), + ])) + }) + + test('loose-mode auto-exclude is a no-op when no immature picks occur', () => { + // is-positive@1.0.0 was published in 2014; with a 1-minute cutoff it + // stays mature relative to the policy. Auto-exclude should not touch + // the workspace manifest when there's nothing to add. + prepare({ + dependencies: { 'is-positive': '1.0.0' }, + }) + execPnpmSync([PUBLIC_REGISTRY, 'install'], { expectSuccess: true }) + + writeYamlFileSync('pnpm-workspace.yaml', { + minimumReleaseAge: 1, + minimumReleaseAgeStrict: false, + }) + + execPnpmSync( + [PUBLIC_REGISTRY, 'install'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + + const workspaceManifest = readYamlFileSync<{ minimumReleaseAgeExclude?: string[] }>('pnpm-workspace.yaml') + expect(workspaceManifest.minimumReleaseAgeExclude).toBeUndefined() + }) + + test('subsequent installs run cleanly once the exclude list is populated', () => { + // Round-trip the auto-collect: first install populates the exclude list + // from fresh resolution, the next install runs the verifier against the + // now-populated list and succeeds without re-announcing anything. The + // verifier and the auto-collect together keep the workspace manifest in + // sync with the lockfile across installs. + prepare({ + dependencies: { 'is-odd': '0.1.2' }, + }) + writeYamlFileSync('pnpm-workspace.yaml', { + minimumReleaseAge: IMMATURE_FOR_EVERYTHING, + minimumReleaseAgeStrict: false, + }) + + execPnpmSync( + [PUBLIC_REGISTRY, 'install'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + execPnpmSync( [PUBLIC_REGISTRY, 'install', '--frozen-lockfile'], { ...omitMinReleaseAgeEnv, expectSuccess: true } ) }) + + test('recursive --no-save leaves the workspace manifest untouched even when picks are collected (shared lockfile)', () => { + // The shared-lockfile recursive branch in recursive.ts: a single + // `mutateModules` call across all importers. Same drain-only-when- + // saving gate has to hold here. + preparePackages([ + { + name: 'project-a', + version: '1.0.0', + dependencies: { 'is-odd': '0.1.2' }, + }, + ]) + writeYamlFileSync('pnpm-workspace.yaml', { + packages: ['*'], + minimumReleaseAge: IMMATURE_FOR_EVERYTHING, + minimumReleaseAgeStrict: false, + }) + + execPnpmSync( + [PUBLIC_REGISTRY, '-r', 'install', '--no-save'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + + const workspaceManifest = readYamlFileSync<{ minimumReleaseAgeExclude?: string[] }>('pnpm-workspace.yaml') + expect(workspaceManifest.minimumReleaseAgeExclude).toBeUndefined() + }) + + test('recursive --no-save leaves the workspace manifest untouched even when picks are collected (per-project lockfiles)', () => { + // The other recursive branch: with sharedWorkspaceLockfile: false + // the per-project loop is taken instead of the single + // mutateModules call. The post-loop updateWorkspaceManifest at the + // tail of recursive.ts also has to honor --no-save. + preparePackages([ + { + name: 'project-a', + version: '1.0.0', + dependencies: { 'is-odd': '0.1.2' }, + }, + ]) + writeYamlFileSync('pnpm-workspace.yaml', { + packages: ['*'], + sharedWorkspaceLockfile: false, + minimumReleaseAge: IMMATURE_FOR_EVERYTHING, + minimumReleaseAgeStrict: false, + }) + + execPnpmSync( + [PUBLIC_REGISTRY, '-r', 'install', '--no-save'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + + const workspaceManifest = readYamlFileSync<{ minimumReleaseAgeExclude?: string[] }>('pnpm-workspace.yaml') + expect(workspaceManifest.minimumReleaseAgeExclude).toBeUndefined() + }) + + test('--no-save leaves the workspace manifest untouched even when picks are collected', () => { + // `--no-save` means "don't persist anything from this install" — the + // auto-add should obey that. Without the gate, the info log would + // claim entries were added that never reached pnpm-workspace.yaml, + // and the next install would either re-prompt or fail verification. + prepare({ + dependencies: { 'is-odd': '0.1.2' }, + }) + writeYamlFileSync('pnpm-workspace.yaml', { + minimumReleaseAge: IMMATURE_FOR_EVERYTHING, + minimumReleaseAgeStrict: false, + }) + + // First install resolves and populates the lockfile but not the + // workspace manifest (because --no-save). + execPnpmSync( + [PUBLIC_REGISTRY, 'install', '--no-save'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + + const workspaceManifest = readYamlFileSync<{ minimumReleaseAgeExclude?: string[] }>('pnpm-workspace.yaml') + expect(workspaceManifest.minimumReleaseAgeExclude).toBeUndefined() + }) + + test('verifier cache invalidates when minimumReleaseAgeExclude is shrunk', async () => { + // Removing an entry from the exclude list could expose a violation + // that previously passed verification. The cache record snapshots the + // exclude list and `canTrustPastCheck` rejects the cached run when + // today's list isn't a superset of the cached one — so the next + // install re-verifies and the now-uncovered immature lockfile entry + // is flagged. + prepare({ + dependencies: { 'is-odd': '0.1.2' }, + }) + await execPnpm([PUBLIC_REGISTRY, 'install']) + + const cacheDir = path.resolve('pnpm-cache') + writeYamlFileSync('pnpm-workspace.yaml', { + minimumReleaseAge: IMMATURE_FOR_EVERYTHING, + minimumReleaseAgeStrict: true, + minimumReleaseAgeExclude: ['is-odd', 'is-buffer', 'is-number', 'kind-of'], + cacheDir, + }) + // Step 1: install with the full exclude list — verifier writes a + // cache record under that policy. + execPnpmSync( + [PUBLIC_REGISTRY, 'install', '--frozen-lockfile'], + { ...omitMinReleaseAgeEnv, expectSuccess: true } + ) + const cacheFile = path.join(cacheDir, 'lockfile-verified.jsonl') + expect(fs.existsSync(cacheFile)).toBe(true) + + // Step 2: drop `is-odd` from the exclude list. The cached record + // had it; today doesn't. canTrustPastCheck must reject so the + // re-verification flags is-odd@0.1.2 as immature. + writeYamlFileSync('pnpm-workspace.yaml', { + minimumReleaseAge: IMMATURE_FOR_EVERYTHING, + minimumReleaseAgeStrict: true, + minimumReleaseAgeExclude: ['is-buffer', 'is-number', 'kind-of'], + cacheDir, + }) + const result = execPnpmSync( + [PUBLIC_REGISTRY, 'install', '--frozen-lockfile'], + omitMinReleaseAgeEnv + ) + expect(result.status).toBe(1) + const output = `${result.stdout.toString()}\n${result.stderr.toString()}` + expect(output).toContain('ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION') + expect(output).toMatch(/is-odd@0\.1\.2/) + }) }) diff --git a/pnpm/test/install/misc.ts b/pnpm/test/install/misc.ts index 712ff5fee4..2b62667d89 100644 --- a/pnpm/test/install/misc.ts +++ b/pnpm/test/install/misc.ts @@ -643,3 +643,46 @@ test('install does not fail when the trust evidence of a package is downgraded b expect(result.status).toBe(0) project.has('@pnpm/e2e.test-provenance') }) + +test('lockfile verifier rejects a trust-downgraded entry that bypassed resolution', () => { + // Step 1: install with trust policy off. The resolver picks up the + // downgraded version without complaint and writes it to the lockfile. + prepare() + execPnpmSync( + ['add', '@pnpm/e2e.test-provenance@0.0.5', '--trust-policy=off'], + { expectSuccess: true } + ) + + // Step 2: turn the policy on. The resolver wouldn't be invoked under + // --frozen-lockfile (peek-path takes over), so the trust check would + // be silently bypassed if the verifier weren't running. The lockfile + // verifier catches the same downgrade pattern the resolver-time check + // catches at fresh resolution. + const result = execPnpmSync([ + 'install', + '--frozen-lockfile', + '--trust-policy=no-downgrade', + ]) + expect(result.status).toBe(1) + const output = `${result.stdout.toString()}\n${result.stderr.toString()}` + expect(output).toContain('ERR_PNPM_TRUST_DOWNGRADE') + expect(output).toMatch(/@pnpm\/e2e\.test-provenance/) +}) + +test('lockfile verifier respects trust-policy-exclude on a downgraded lockfile entry', () => { + prepare() + execPnpmSync( + ['add', '@pnpm/e2e.test-provenance@0.0.5', '--trust-policy=off'], + { expectSuccess: true } + ) + + // With the exclude entry in place, the verifier short-circuits before + // running the trust check on this package — mirrors the resolver-time + // exclude path so users have one consistent way to allow a downgrade. + execPnpmSync([ + 'install', + '--frozen-lockfile', + '--trust-policy=no-downgrade', + '--trust-policy-exclude=@pnpm/e2e.test-provenance', + ], { expectSuccess: true }) +}) diff --git a/resolving/default-resolver/src/index.ts b/resolving/default-resolver/src/index.ts index 00849f52ab..e3b26b04ce 100644 --- a/resolving/default-resolver/src/index.ts +++ b/resolving/default-resolver/src/index.ts @@ -151,6 +151,10 @@ export type ResolutionVerifierFactoryOptions = | 'minimumReleaseAge' | 'minimumReleaseAgeStrict' | 'minimumReleaseAgeExclude' + | 'ignoreMissingTimeField' + | 'trustPolicy' + | 'trustPolicyExclude' + | 'trustPolicyIgnoreAfter' | 'now' > & { configByUri?: Record @@ -184,6 +188,10 @@ export function createResolutionVerifiers ( minimumReleaseAge: opts.minimumReleaseAge, minimumReleaseAgeStrict: opts.minimumReleaseAgeStrict, minimumReleaseAgeExclude: opts.minimumReleaseAgeExclude, + ignoreMissingTimeField: opts.ignoreMissingTimeField, + trustPolicy: opts.trustPolicy, + trustPolicyExclude: opts.trustPolicyExclude, + trustPolicyIgnoreAfter: opts.trustPolicyIgnoreAfter, registries: opts.registries, namedRegistries: opts.namedRegistries, fetchOpts, diff --git a/resolving/npm-resolver/src/createNpmResolutionVerifier.ts b/resolving/npm-resolver/src/createNpmResolutionVerifier.ts index 0d7f093aa7..edafec34c1 100644 --- a/resolving/npm-resolver/src/createNpmResolutionVerifier.ts +++ b/resolving/npm-resolver/src/createNpmResolutionVerifier.ts @@ -2,11 +2,12 @@ import { pickRegistryForPackage } from '@pnpm/config.pick-registry-for-package' import { createPackageVersionPolicy } from '@pnpm/config.version-policy' import { FULL_META_DIR } from '@pnpm/constants' import { PnpmError } from '@pnpm/error' +import type { PackageMeta } from '@pnpm/resolving.registry.types' import type { Resolution, ResolutionVerifier, } from '@pnpm/resolving.resolver-base' -import type { PackageVersionPolicy, Registries } from '@pnpm/types' +import type { PackageVersionPolicy, Registries, TrustPolicy } from '@pnpm/types' import semver from 'semver' import type { FetchMetadataFromFromRegistryOptions } from './fetch.js' @@ -17,7 +18,12 @@ import { type FetchFullMetadataCachedOptions, } from './fetchFullMetadataCached.js' import { BUILTIN_NAMED_REGISTRIES } from './parseBareSpecifier.js' -import { getPkgMirrorPath, loadMeta } from './pickPackage.js' +import { getPkgMirrorPath, loadMeta, warnMissingTimeFieldOnce } from './pickPackage.js' +import { failIfTrustDowngraded } from './trustChecks.js' +import { + MINIMUM_RELEASE_AGE_VIOLATION_CODE, + TRUST_DOWNGRADE_VIOLATION_CODE, +} from './violationCodes.js' export interface CreateNpmResolutionVerifierOptions { /** @@ -26,13 +32,37 @@ export interface CreateNpmResolutionVerifierOptions { */ minimumReleaseAge?: number /** - * Gate the age check on strict mode so the built-in default doesn't - * silently enforce for users who never opted in. The verifier factory - * returns `undefined` unless both `minimumReleaseAge > 0` and - * `minimumReleaseAgeStrict` are set. + * Retained on the options bag because the resolver path branches on it + * (the lowest-version fallback) and tests forward both fields together. + * The verifier itself no longer gates on this flag — once the loose-mode + * auto-collect makes every accepted-immature pin explicit in + * `minimumReleaseAgeExclude`, running the verifier in loose mode is the + * thing that proves the manifest stays in sync with the lockfile. */ minimumReleaseAgeStrict?: boolean minimumReleaseAgeExclude?: string[] + /** + * When the registry's metadata lacks the per-version `time` field + * (some self-hosted registries strip it), the verifier can't apply + * the maturity cutoff. Set this to `true` to mirror the resolver's + * `pickMatchingVersionFinal` warn-and-skip behavior — the verifier + * passes the entry with a one-time `globalWarn`, instead of failing + * closed. Defaults to `false` so the verifier stays stricter than + * the resolver only when the user has explicitly opted in to the + * skip on the resolver side. + */ + ignoreMissingTimeField?: boolean + /** + * `'no-downgrade'` rejects a lockfile entry whose version has weaker + * trust evidence (no attestations) than an earlier-published version + * had. This mirrors the resolver-time `failIfTrustDowngraded` check + * applied during fresh resolution — the verifier catches the same + * supply-chain signal on entries that bypassed resolution (peek-path, + * frozen lockfile, etc.). + */ + trustPolicy?: TrustPolicy + trustPolicyExclude?: string[] + trustPolicyIgnoreAfter?: number registries: Registries /** * Registries reached via the named-registry resolver chain (e.g. `gh:` → @@ -55,9 +85,10 @@ export interface CreateNpmResolutionVerifierOptions { /** * Returns a `ResolutionVerifier` that re-applies the `minimumReleaseAge` - * policy to npm-registry-resolved lockfile entries, or `undefined` when no - * policy is active. Pairs with `createNpmResolver`: each resolver factory - * may export a sibling verifier factory that the default-resolver combines. + * and/or `trustPolicy='no-downgrade'` policies to npm-registry-resolved + * lockfile entries, or `undefined` when no policy is active. Pairs with + * `createNpmResolver`: each resolver factory may export a sibling + * verifier factory that the default-resolver combines. * * Designed for fail-closed semantics: if the manifest can't be loaded or * the pinned version is missing from it, the verifier reports a violation @@ -67,11 +98,20 @@ export interface CreateNpmResolutionVerifierOptions { export function createNpmResolutionVerifier ( opts: CreateNpmResolutionVerifierOptions ): ResolutionVerifier | undefined { - if (!opts.minimumReleaseAge || !opts.minimumReleaseAgeStrict) return undefined + const ageCheckActive = Boolean(opts.minimumReleaseAge) + const trustCheckActive = opts.trustPolicy === 'no-downgrade' + // No policy → no verifier. Skipping early keeps the install-side fan-out + // empty when nothing is configured. + if (!ageCheckActive && !trustCheckActive) return undefined - const cutoff = (opts.now ?? Date.now()) - opts.minimumReleaseAge * 60 * 1000 + const cutoff = ageCheckActive + ? (opts.now ?? Date.now()) - opts.minimumReleaseAge! * 60 * 1000 + : 0 const excludePolicy = opts.minimumReleaseAgeExclude?.length - ? createExcludePolicy(opts.minimumReleaseAgeExclude) + ? createExcludePolicy(opts.minimumReleaseAgeExclude, 'minimumReleaseAgeExclude') + : undefined + const trustExcludePolicy = opts.trustPolicyExclude?.length + ? createExcludePolicy(opts.trustPolicyExclude, 'trustPolicyExclude') : undefined // Pre-normalize named-registry URLs and sort by length so two registries @@ -98,10 +138,14 @@ export function createNpmResolutionVerifier ( .filter((value): value is string => value != null) .sort((a, b) => b.length - a.length) - // Per-install dedup of every network/disk fetch the verifier issues - // (see fetchPublishedAt for the lookup order). The on-disk - // conditional-GET cache is handled inside fetch{Abbreviated,Full}MetadataCached - // via the resolver's shared mirrors at opts.cacheDir. + // Per-install dedup of every network/disk fetch the verifier issues. + // The maturity check uses the layered `fetchPublishedAt` lookup; the + // trust check uses an attestation fast-path before falling back to + // the same full-metadata mirror. All maps live here so verifying + // many versions of the same package only pays the disk/network costs + // once. The on-disk conditional-GET cache is handled inside + // fetch{Abbreviated,Full}MetadataCached via the resolver's shared + // mirrors at opts.cacheDir. const lookupContext: PublishedAtLookupContext = { fetchOpts: opts.fetchOpts, getAuthHeaderValueByURI: opts.getAuthHeaderValueByURI, @@ -111,74 +155,233 @@ export function createNpmResolutionVerifier ( publishedAtCache: new Map(), localMetaCache: new Map(), fullMetaCache: new Map(), + fullMetaForTrustCache: new Map(), } - const minimumReleaseAge = opts.minimumReleaseAge + const minimumReleaseAge = opts.minimumReleaseAge ?? 0 + const trustPolicy = opts.trustPolicy + const trustPolicyIgnoreAfter = opts.trustPolicyIgnoreAfter const verify: ResolutionVerifier['verify'] = async (resolution, { name, version }) => { if (!isNpmRegistryResolution(resolution)) return { ok: true } // Non-semver versions identify URL tarballs, file: refs, git refs, etc. - // The age policy doesn't apply and a registry lookup would 404. + // Neither the age nor the trust policy applies, and a registry lookup + // would 404. if (!semver.valid(version)) return { ok: true } - if (isExcluded(excludePolicy, name, version)) return { ok: true } + + const ageApplies = ageCheckActive && !isExcluded(excludePolicy, name, version) + const trustApplies = trustCheckActive && !isExcluded(trustExcludePolicy, name, version) + if (!ageApplies && !trustApplies) return { ok: true } const tarballUrl = (resolution as { tarball?: string }).tarball const registry = pickRegistryForVersion(opts.registries, namedRegistryPrefixes, name, tarballUrl) - let published: string | undefined - try { - published = await fetchPublishedAt(lookupContext, registry, name, version) - } catch (err) { - return { - ok: false, - code: 'MINIMUM_RELEASE_AGE_VIOLATION', - reason: uncheckable(err instanceof Error ? err.message : String(err)), - } + + if (ageApplies) { + const ageViolation = await runAgeCheck(lookupContext, registry, name, version, cutoff, opts.ignoreMissingTimeField === true) + if (ageViolation) return ageViolation } - if (!published) { - // No source — attestation, local mirror, or full metadata — - // surfaced a publish timestamp for this version. Either it's - // unpublished or the registry doesn't expose per-version - // timestamps. Report a violation rather than silently passing. - return { - ok: false, - code: 'MINIMUM_RELEASE_AGE_VIOLATION', - reason: uncheckable('version not present in registry manifest'), - } - } - const publishedAt = new Date(published) - const ts = publishedAt.getTime() - if (Number.isNaN(ts)) { - return { - ok: false, - code: 'MINIMUM_RELEASE_AGE_VIOLATION', - reason: 'publish timestamp is not a valid date', - } - } - if (ts > cutoff) { - return { - ok: false, - code: 'MINIMUM_RELEASE_AGE_VIOLATION', - reason: `was published at ${publishedAt.toISOString()}, within the minimumReleaseAge cutoff (${new Date(cutoff).toISOString()})`, - } + + if (trustApplies) { + const trustViolation = await runTrustCheck(lookupContext, registry, name, version, { + trustPolicyExclude: trustExcludePolicy, + trustPolicyIgnoreAfter, + }) + if (trustViolation) return trustViolation } + return { ok: true } } + // Snapshot the exclude lists (sorted, deduped) and require an exact + // match in `canTrustPastCheck`: cache identity == policy identity. + // Any change to either exclude list — adding, removing, or + // substituting an entry — invalidates the cached run. This is + // stricter than a pure correctness check would require (adding to + // either list is more permissive and the cached pass would still + // hold), but it makes the cache contract trivial to reason about and + // removes a class of bypasses where a previously-approved version + // stays trusted after its exclude entry has been pulled. + const sortedMinAgeExcludes = [...new Set(opts.minimumReleaseAgeExclude ?? [])].sort() + const sortedTrustExcludes = [...new Set(opts.trustPolicyExclude ?? [])].sort() return { verify, - policy: { minimumReleaseAge }, + policy: { + minimumReleaseAge, + minimumReleaseAgeExclude: sortedMinAgeExcludes, + trustPolicy: trustPolicy ?? null, + trustPolicyExclude: sortedTrustExcludes, + trustPolicyIgnoreAfter: trustPolicyIgnoreAfter ?? null, + }, canTrustPastCheck: (cached) => { - // A previously cached run under a larger cutoff (stricter window) - // is trustworthy under a smaller current one — its set of - // accepted versions is a subset of today's. The reverse — - // tightening the cutoff — invalidates the cached run: versions - // that passed before may now be in-window. Non-number cached - // values come from an older record shape and aren't trusted. + // Maturity: a previously cached run under a larger cutoff + // (stricter window) is trustworthy under a smaller current one — + // its set of accepted versions is a subset of today's. The + // reverse — tightening the cutoff — invalidates the cached run: + // versions that passed before may now be in-window. Non-number + // cached values come from an older record shape and aren't trusted. const past = cached.minimumReleaseAge - return typeof past === 'number' && past >= minimumReleaseAge + const pastNumber = typeof past === 'number' ? past : 0 + if (pastNumber < minimumReleaseAge) return false + + // Excludes: today's sorted-deduped lists must match the cached + // ones byte for byte. Older records (no field) fall back to an + // empty array, so they only trust today's empty policy. + const pastMinAgeExcludes = Array.isArray(cached.minimumReleaseAgeExclude) + ? cached.minimumReleaseAgeExclude + : [] + if (JSON.stringify(pastMinAgeExcludes) !== JSON.stringify(sortedMinAgeExcludes)) return false + + // Trust policy: any change to `trustPolicy`, the exclude list, or + // the ignore-after cutoff invalidates the cached run. Older + // records (no trust field at all) treat the trust policy as + // absent and are only trusted under an unset-today policy. + const pastTrustPolicy = cached.trustPolicy ?? null + const todayTrustPolicy = trustPolicy ?? null + if (pastTrustPolicy !== todayTrustPolicy) return false + const pastTrustExcludes = Array.isArray(cached.trustPolicyExclude) + ? cached.trustPolicyExclude + : [] + if (JSON.stringify(pastTrustExcludes) !== JSON.stringify(sortedTrustExcludes)) return false + const pastIgnoreAfter = typeof cached.trustPolicyIgnoreAfter === 'number' + ? cached.trustPolicyIgnoreAfter + : null + const todayIgnoreAfter = trustPolicyIgnoreAfter ?? null + if (pastIgnoreAfter !== todayIgnoreAfter) return false + + return true }, } } +async function runAgeCheck ( + context: PublishedAtLookupContext, + registry: string, + name: string, + version: string, + cutoff: number, + ignoreMissingTimeField: boolean +): Promise<{ ok: false, code: string, reason: string } | undefined> { + let published: string | undefined + try { + published = await fetchPublishedAt(context, registry, name, version) + } catch (err) { + return { + ok: false, + code: MINIMUM_RELEASE_AGE_VIOLATION_CODE, + reason: uncheckable('minimumReleaseAge', err instanceof Error ? err.message : String(err)), + } + } + if (!published) { + // No source — attestation, local mirror, or full metadata — + // surfaced a publish timestamp for this version. The resolver's + // pickMatchingVersionFinal honors `minimumReleaseAgeIgnoreMissingTime` + // for the same shape (some self-hosted registries strip per-version + // `time`); the verifier mirrors that so it can't be stricter than + // fresh resolution. Without the flag we still fail closed — better + // a false reject than silent bypass when the user hasn't opted in. + if (ignoreMissingTimeField) { + warnMissingTimeFieldOnce(name) + return undefined + } + return { + ok: false, + code: MINIMUM_RELEASE_AGE_VIOLATION_CODE, + reason: uncheckable('minimumReleaseAge', 'version not present in registry manifest'), + } + } + const publishedAt = new Date(published) + const ts = publishedAt.getTime() + if (Number.isNaN(ts)) { + return { + ok: false, + code: MINIMUM_RELEASE_AGE_VIOLATION_CODE, + reason: 'publish timestamp is not a valid date', + } + } + if (ts > cutoff) { + return { + ok: false, + code: MINIMUM_RELEASE_AGE_VIOLATION_CODE, + reason: `was published at ${publishedAt.toISOString()}, within the minimumReleaseAge cutoff (${new Date(cutoff).toISOString()})`, + } + } + return undefined +} + +/** + * Run the resolver-time `failIfTrustDowngraded` check against the + * pinned lockfile version. The packument is fetched through a + * per-install cache so multiple versions of the same package share + * one fetch. + * + * No attestation fast-path here even though the per-version + * attestation endpoint is cheaper than the packument: presence of + * provenance on the current version is not sufficient to clear a + * downgrade. A package could have shipped earlier versions under a + * `trustedPublisher` (the higher-rank evidence) and then dropped + * back to plain provenance for the version we're verifying — + * `failIfTrustDowngraded` correctly flags that, and a "has any + * attestation → pass" shortcut would silently miss it. + */ +async function runTrustCheck ( + context: PublishedAtLookupContext, + registry: string, + name: string, + version: string, + opts: { + trustPolicyExclude?: PackageVersionPolicy + trustPolicyIgnoreAfter?: number + } +): Promise<{ ok: false, code: string, reason: string } | undefined> { + let meta: PackageMeta + try { + meta = await fetchFullMetaForTrust(context, registry, name) + } catch (err) { + // `fetchFullMetadataCached` rejects (network error, 404, etc.); the + // verifier fails closed so a missing manifest can't be mistaken + // for a passing trust check. + return { + ok: false, + code: TRUST_DOWNGRADE_VIOLATION_CODE, + reason: uncheckable('trustPolicy', err instanceof Error ? err.message : String(err)), + } + } + + try { + failIfTrustDowngraded(meta, version, opts) + } catch (err) { + return { + ok: false, + code: TRUST_DOWNGRADE_VIOLATION_CODE, + reason: err instanceof Error ? err.message : String(err), + } + } + return undefined +} + +function fetchFullMetaForTrust ( + context: PublishedAtLookupContext, + registry: string, + name: string +): Promise { + const cacheKey = `${registry}\x00${name}` + let cachedPromise = context.fullMetaForTrustCache.get(cacheKey) + if (cachedPromise == null) { + // Don't swallow the fetch rejection here — `runTrustCheck` catches it + // and surfaces the underlying message in the violation reason, which + // is more actionable than the generic "metadata is unavailable" the + // `!meta` fallback emits. The cache still holds the rejected promise + // so repeat verifier calls for the same (registry, name) within one + // install don't refetch a known-failing endpoint. + cachedPromise = fetchFullMetadataCached(context.fetchOpts, name, { + registry, + authHeaderValue: context.getAuthHeaderValueByURI(registry), + cacheDir: context.cacheDir, + }) + context.fullMetaForTrustCache.set(cacheKey, cachedPromise) + } + return cachedPromise +} + type PublishedAtTimeMap = Record interface PublishedAtLookupContext { @@ -200,7 +403,7 @@ interface PublishedAtLookupContext { * ~zero cost. Resolves to the parsed metadata or `undefined` on * failure. */ - abbreviatedMetaCache: Map> + abbreviatedMetaCache: Map } | undefined>> /** * Per-(registry+name+version) memo of the final published-at answer * the verifier hands to the policy check. One install verifies each @@ -219,6 +422,14 @@ interface PublishedAtLookupContext { * attestation endpoint fail to yield a timestamp. */ fullMetaCache: Map> + /** + * Per-(registry+name) memo of the full packument used by the trust + * check (history walk for `failIfTrustDowngraded`). Kept separate + * from `fullMetaCache` because the trust check needs the whole + * document (`_npmUser`, `dist.attestations` per version) where the + * age check only needs `time`. + */ + fullMetaForTrustCache: Map> } /** @@ -263,7 +474,7 @@ async function resolvePublishedAt ( name: string, version: string ): Promise { - const abbreviatedShortcut = await tryAbbreviatedModifiedShortcut(context, registry, name) + const abbreviatedShortcut = await tryAbbreviatedModifiedShortcut(context, registry, name, version) if (abbreviatedShortcut != null) return abbreviatedShortcut const localTime = await readLocalMetaTime(context, registry, name) @@ -282,18 +493,25 @@ async function resolvePublishedAt ( /** * Returns the abbreviated metadata's `modified` timestamp **iff** it * proves the gate would pass — i.e. modified is strictly older than - * the policy cutoff. In that case every version this package contains - * predates the cutoff, so the caller can short-circuit with `modified` - * as a conservative upper-bound publish time. + * the policy cutoff *and* the pinned version still exists in the + * package's current versions map. + * + * The version check is the fail-closed contract: an unpublished or + * never-published version must not slip through on the package-level + * `modified` timestamp. When the version is missing here we fall + * through to the later layers so the caller eventually surfaces the + * "version not present in registry manifest" violation. * * Returns `undefined` otherwise (modified is too recent, the metadata - * lacks a parseable modified field, or the fetch failed) and the - * caller proceeds with per-version lookups. + * lacks a parseable modified field, the version isn't in the abbreviated + * form, or the fetch failed) and the caller proceeds with per-version + * lookups. */ async function tryAbbreviatedModifiedShortcut ( context: PublishedAtLookupContext, registry: string, - name: string + name: string, + version: string ): Promise { const meta = await fetchAbbreviatedMeta(context, registry, name) const modified = meta?.modified @@ -301,6 +519,11 @@ async function tryAbbreviatedModifiedShortcut ( const modifiedMs = Date.parse(modified) if (Number.isNaN(modifiedMs)) return undefined if (modifiedMs >= context.cutoffMs) return undefined + // The shortcut treats `modified` as an upper bound on every version's + // publish time — but only for versions the registry currently lists. + // An unpublished or never-published pin would otherwise pass the gate + // on a stale package-level timestamp. + if (!meta?.versions || !(version in meta.versions)) return undefined return modified } @@ -308,7 +531,7 @@ function fetchAbbreviatedMeta ( context: PublishedAtLookupContext, registry: string, name: string -): Promise<{ modified?: string } | undefined> { +): Promise<{ modified?: string, versions?: Record } | undefined> { const cacheKey = `${registry}\x00${name}` let cachedPromise = context.abbreviatedMetaCache.get(cacheKey) if (cachedPromise == null) { @@ -395,11 +618,11 @@ function tryParseUrl (url: string): URL | null { } } -function uncheckable (why: string): string { - return `could not be checked against minimumReleaseAge (${why})` +function uncheckable (policy: 'minimumReleaseAge' | 'trustPolicy', why: string): string { + return `could not be checked against ${policy} (${why})` } -function createExcludePolicy (patterns: string[]): PackageVersionPolicy { +function createExcludePolicy (patterns: string[], key: string): PackageVersionPolicy { // Mirror the wrapping done by the full-resolution path // (installing/deps-resolver/src/resolveDependencyTree.ts) so the error // code is identical regardless of which path surfaced the invalid pattern. @@ -408,8 +631,8 @@ function createExcludePolicy (patterns: string[]): PackageVersionPolicy { } catch (err) { if (!err || typeof err !== 'object' || !('message' in err)) throw err throw new PnpmError( - 'INVALID_MINIMUM_RELEASE_AGE_EXCLUDE', - `Invalid value in minimumReleaseAgeExclude: ${(err as { message: string }).message}` + `INVALID_${key.replace(/([A-Z])/g, '_$1').toUpperCase()}`, + `Invalid value in ${key}: ${(err as { message: string }).message}` ) } } diff --git a/resolving/npm-resolver/src/index.ts b/resolving/npm-resolver/src/index.ts index 31a6871682..4122f065e4 100644 --- a/resolving/npm-resolver/src/index.ts +++ b/resolving/npm-resolver/src/index.ts @@ -12,6 +12,8 @@ import type { DirectoryResolution, PkgResolutionId, PreferredVersions, + Resolution, + ResolutionPolicyViolation, ResolveResult, TarballResolution, WantedDependency, @@ -55,6 +57,7 @@ import { } from './pickPackage.js' import { pickPackageFromMeta, pickVersionByVersionRange } from './pickPackageFromMeta.js' import { failIfTrustDowngraded } from './trustChecks.js' +import { MINIMUM_RELEASE_AGE_VIOLATION_CODE } from './violationCodes.js' import { whichVersionIsPinned } from './whichVersionIsPinned.js' import { workspacePrefToNpm } from './workspacePrefToNpm.js' @@ -62,29 +65,16 @@ export interface NoMatchingVersionErrorOptions { wantedDependency: WantedDependency packageMeta: PackageMeta registry: string - immatureVersion?: string - publishedBy?: Date } export class NoMatchingVersionError extends PnpmError { public readonly packageMeta: PackageMeta - public readonly immatureVersion?: string constructor (opts: NoMatchingVersionErrorOptions) { const dep = opts.wantedDependency.alias ? `${opts.wantedDependency.alias}@${opts.wantedDependency.bareSpecifier ?? ''}` : opts.wantedDependency.bareSpecifier! - let errorMessage: string - if (opts.publishedBy && opts.immatureVersion && opts.packageMeta.time) { - const time = new Date(opts.packageMeta.time[opts.immatureVersion]) - const releaseAgeText = formatTimeAgo(time) ?? 'just now' - const pkgName = opts.wantedDependency.alias ?? opts.packageMeta.name - errorMessage = `Version ${opts.immatureVersion} (released ${releaseAgeText}) of ${pkgName} does not meet the minimumReleaseAge constraint` - } else { - errorMessage = `No matching version found for ${dep} while fetching it from ${opts.registry}` - } - super(opts.publishedBy ? 'NO_MATURE_MATCHING_VERSION' : 'NO_MATCHING_VERSION', errorMessage) + super('NO_MATCHING_VERSION', `No matching version found for ${dep} while fetching it from ${opts.registry}`) this.packageMeta = opts.packageMeta - this.immatureVersion = opts.immatureVersion } } @@ -130,6 +120,10 @@ export { workspacePrefToNpm, } export { createNpmResolutionVerifier, type CreateNpmResolutionVerifierOptions } from './createNpmResolutionVerifier.js' +export { + MINIMUM_RELEASE_AGE_VIOLATION_CODE, + TRUST_DOWNGRADE_VIOLATION_CODE, +} from './violationCodes.js' export { whichVersionIsPinned } from './whichVersionIsPinned.js' export interface ResolverFactoryOptions { @@ -145,7 +139,6 @@ export interface ResolverFactoryOptions { namedRegistries?: Record saveWorkspaceProtocol?: boolean | 'rolling' preserveAbsolutePaths?: boolean - strictPublishedByCheck?: boolean ignoreMissingTimeField?: boolean fetchWarnTimeoutMs?: number /** Pre-populated metadata cache. When provided, the resolver uses this @@ -249,7 +242,6 @@ export function createNpmResolver ( offline: opts.offline, preferOffline: opts.preferOffline, cacheDir: opts.cacheDir, - strictPublishedByCheck: opts.strictPublishedByCheck, ignoreMissingTimeField: opts.ignoreMissingTimeField, }), registries: opts.registries, @@ -384,6 +376,19 @@ async function resolveNpm ( resolution: currentResolution as TarballResolution, resolvedVia: 'npm-registry', publishedAt: opts.currentPkg.publishedAt, + // Loose-mode bypass: a lockfile entry whose publishedAt sits + // after the maturity cutoff would have been rejected at + // resolver time, but the peek path skips the maturity check. + // Report inline so the deps-resolver aggregator surfaces it + // to the install command. + policyViolation: detectMinReleaseAgeViolation({ + name: manifest.name, + version: manifest.version, + publishedAt: opts.currentPkg.publishedAt, + resolution: currentResolution, + publishedBy: opts.publishedBy, + publishedByExclude: opts.publishedByExclude, + }), } } } @@ -443,22 +448,6 @@ async function resolveNpm ( } } - if (opts.publishedBy) { - const immatureVersion = pickVersionByVersionRange({ - meta, - versionRange: spec.fetchSpec, - preferredVersionSelectors: opts.preferredVersions?.[spec.name], - }) - if (immatureVersion) { - throw new NoMatchingVersionError({ - wantedDependency, - packageMeta: meta, - registry, - immatureVersion, - publishedBy: opts.publishedBy, - }) - } - } throw new NoMatchingVersionError({ wantedDependency, packageMeta: meta, registry }) } else if (opts.trustPolicy === 'no-downgrade') { failIfTrustDowngraded(meta, pickedPackage.version, opts) @@ -512,14 +501,23 @@ async function resolveNpm ( defaultPinnedVersion: opts.pinnedVersion, }) } + const publishedAt = meta.time?.[pickedPackage.version] return { id, latest: meta['dist-tags'].latest, manifest: pickedPackage, resolution, resolvedVia: 'npm-registry', - publishedAt: meta.time?.[pickedPackage.version], + publishedAt, normalizedBareSpecifier, + policyViolation: detectMinReleaseAgeViolation({ + name: pickedPackage.name, + version: pickedPackage.version, + publishedAt, + resolution, + publishedBy: opts.publishedBy, + publishedByExclude: opts.publishedByExclude, + }), } } @@ -632,6 +630,7 @@ async function pickFromSimpleRegistry ( manifest: DependencyManifest resolution: TarballResolution publishedAt?: string + policyViolation?: ResolutionPolicyViolation }> { const authHeaderValue = ctx.getAuthHeaderValueByURI(registry) const { meta, pickedPackage } = await ctx.pickPackage(spec, { @@ -648,15 +647,25 @@ async function pickFromSimpleRegistry ( if (pickedPackage == null) { throw new NoMatchingVersionError({ wantedDependency, packageMeta: meta, registry }) } + const resolution = { + integrity: getIntegrity(pickedPackage.dist), + tarball: normalizeRegistryUrl(pickedPackage.dist.tarball), + } + const publishedAt = meta.time?.[pickedPackage.version] return { id: `${pickedPackage.name}@${pickedPackage.version}` as PkgResolutionId, latest: meta['dist-tags'].latest, manifest: pickedPackage, - resolution: { - integrity: getIntegrity(pickedPackage.dist), - tarball: normalizeRegistryUrl(pickedPackage.dist.tarball), - }, - publishedAt: meta.time?.[pickedPackage.version], + resolution, + publishedAt, + policyViolation: detectMinReleaseAgeViolation({ + name: pickedPackage.name, + version: pickedPackage.version, + publishedAt, + resolution, + publishedBy: opts.publishedBy, + publishedByExclude: opts.publishedByExclude, + }), } } @@ -907,6 +916,44 @@ function defaultTagForAlias (alias: string, defaultTag: string): RegistryPackage } } +/** + * Inline minimumReleaseAge detection: returns a violation entry when the + * picked version's publish timestamp is past the policy cutoff (and + * isn't covered by `publishedByExclude`). The resolver already has the + * timestamp in hand, so reporting inline saves the install layer from + * re-walking the resolved tree and re-fetching the same metadata. The + * deps-resolver aggregates the per-resolve `policyViolation` fields into + * a single set the install command reacts to. + * + * Returns `undefined` for resolutions outside the policy — no policy + * active, version excluded by pattern, timestamp missing or malformed, + * or version mature. Specific-version exclusions (`pkg@1.0.0`) and + * full-name exclusions (`pkg`) are both honored so an entry already on + * the user's exclude list isn't re-announced every install. + */ +function detectMinReleaseAgeViolation (args: { + name: string + version: string + publishedAt: string | undefined + resolution: Resolution + publishedBy: Date | undefined + publishedByExclude: PackageVersionPolicy | undefined +}): ResolutionPolicyViolation | undefined { + if (!args.publishedBy || !args.publishedAt) return undefined + const excludeResult = args.publishedByExclude?.(args.name) + if (excludeResult === true) return undefined + if (Array.isArray(excludeResult) && excludeResult.includes(args.version)) return undefined + const ts = new Date(args.publishedAt).getTime() + if (Number.isNaN(ts) || ts <= args.publishedBy.getTime()) return undefined + return { + name: args.name, + version: args.version, + resolution: args.resolution, + code: MINIMUM_RELEASE_AGE_VIOLATION_CODE, + reason: `was published at ${new Date(ts).toISOString()}, within the minimumReleaseAge cutoff (${args.publishedBy.toISOString()})`, + } +} + function getIntegrity (dist: { integrity?: string shasum: string diff --git a/resolving/npm-resolver/src/pickPackage.ts b/resolving/npm-resolver/src/pickPackage.ts index 71555cf4e3..4b8cfbfb7c 100644 --- a/resolving/npm-resolver/src/pickPackage.ts +++ b/resolving/npm-resolver/src/pickPackage.ts @@ -75,7 +75,6 @@ export interface PickPackageOptions extends PickPackageFromMetaOptions { interface PickerOptions extends PickPackageFromMetaOptions { pickLowestVersion?: boolean includeLatestTag?: boolean - strictPublishedByCheck?: boolean ignoreMissingTimeField?: boolean } @@ -106,8 +105,9 @@ const pickHighest = pickPackageFromMeta.bind(null, pickVersionByVersionRange) const pickLowest = pickPackageFromMeta.bind(null, pickLowestVersionByVersionRange) // When minimumReleaseAge is active: try the highest mature version; if none -// and strictPublishedByCheck is off, fall back to the lowest version in range -// without applying the maturity filter. +// satisfies the range, fall back to the lowest version regardless of maturity +// so the resolver can report the violation inline and let the install layer +// (or other caller) decide what to do — never throw at this layer. function pickRespectingMinReleaseAge ( pickerOpts: PickerOptions, spec: RegistryPackageSpec, @@ -115,7 +115,7 @@ function pickRespectingMinReleaseAge ( ): PackageInRegistry | null { return runPicker(pickerOpts, spec, (targetSpec) => { const highest = pickHighest(pickerOpts, meta, targetSpec) - if (highest || pickerOpts.strictPublishedByCheck) return highest + if (highest) return highest return pickLowest({ preferredVersionSelectors: pickerOpts.preferredVersionSelectors, }, meta, targetSpec) @@ -178,7 +178,6 @@ export async function pickPackage ( offline?: boolean preferOffline?: boolean filterMetadata?: boolean - strictPublishedByCheck?: boolean ignoreMissingTimeField?: boolean }, spec: RegistryPackageSpec, @@ -192,7 +191,6 @@ export async function pickPackage ( publishedByExclude: opts.publishedByExclude, pickLowestVersion: opts.pickLowestVersion, includeLatestTag: opts.includeLatestTag, - strictPublishedByCheck: ctx.strictPublishedByCheck, ignoreMissingTimeField: ctx.ignoreMissingTimeField, } @@ -279,10 +277,11 @@ export async function pickPackage ( pickedPackage, } } - } catch (err: unknown) { - if (shouldRethrowFromFastPathCache(err, ctx.strictPublishedByCheck)) { - throw err - } + } catch { + // Swallow fast-path errors (e.g. ERR_PNPM_MISSING_TIME from + // abbreviated meta) and fall through to the network fetch, which + // can upgrade to full metadata and run the maturity check on + // real `time` data. } } } @@ -299,10 +298,8 @@ export async function pickPackage ( pickedPackage, } } - } catch (err: unknown) { - if (shouldRethrowFromFastPathCache(err, ctx.strictPublishedByCheck)) { - throw err - } + } catch { + // Same as above — fall through to the network fetch. } } } @@ -495,14 +492,6 @@ async function maybeUpgradeAbbreviatedMetaForReleaseAge ( return { meta: fullFetchResult.meta, upgradedFrom: fullFetchResult } } -// Returns true when a fast-path cache catch should rethrow under -// strictPublishedByCheck. ERR_PNPM_MISSING_TIME is excluded so callers fall -// through to the network fetch path, which can upgrade abbreviated cached -// metadata to full and run the maturity check on real `time` data. -function shouldRethrowFromFastPathCache (err: unknown, strictPublishedByCheck: boolean | undefined): boolean { - return strictPublishedByCheck === true && !isMissingTimeError(err) -} - // Persists upgraded full metadata to the on-disk cache mirror and returns // the meta to store in the in-memory cache. When `filterMetadata` is on, the // in-memory and on-disk forms are both stripped via `clearMeta`; otherwise @@ -604,7 +593,7 @@ function isMissingTimeError (err: unknown): boolean { const MAX_WARNED_MISSING_TIME = 1024 const warnedMissingTimeFor = new Set() -function warnMissingTimeFieldOnce (pkgName: string): void { +export function warnMissingTimeFieldOnce (pkgName: string): void { if (warnedMissingTimeFor.has(pkgName)) return if (warnedMissingTimeFor.size >= MAX_WARNED_MISSING_TIME) { // Set preserves insertion order, so the first entry is the oldest. diff --git a/resolving/npm-resolver/src/violationCodes.ts b/resolving/npm-resolver/src/violationCodes.ts new file mode 100644 index 0000000000..f86b6d9c41 --- /dev/null +++ b/resolving/npm-resolver/src/violationCodes.ts @@ -0,0 +1,12 @@ +/** + * Violation codes the npm resolver attaches to + * `ResolutionPolicyViolation.code` when an inline policy check rejects + * a pick. Exported so downstream code (the install command, the strict + * resolver wrapper, tests) references one source of truth instead of + * re-typing the string. + * + * Lives in its own module — both `index.ts` and `createNpmResolutionVerifier.ts` + * import it, so keeping the constants here avoids a cycle. + */ +export const MINIMUM_RELEASE_AGE_VIOLATION_CODE = 'MINIMUM_RELEASE_AGE_VIOLATION' +export const TRUST_DOWNGRADE_VIOLATION_CODE = 'TRUST_DOWNGRADE' diff --git a/resolving/npm-resolver/test/createNpmResolutionVerifier.test.ts b/resolving/npm-resolver/test/createNpmResolutionVerifier.test.ts new file mode 100644 index 0000000000..a778a186b3 --- /dev/null +++ b/resolving/npm-resolver/test/createNpmResolutionVerifier.test.ts @@ -0,0 +1,241 @@ +import { afterEach, beforeEach, expect, test } from '@jest/globals' +import { createFetchFromRegistry } from '@pnpm/network.fetch' +import { createNpmResolutionVerifier } from '@pnpm/resolving.npm-resolver' +import type { Resolution } from '@pnpm/resolving.resolver-base' +import type { Registries } from '@pnpm/types' +import { temporaryDirectory } from 'tempy' + +import { getMockAgent, setupMockAgent, teardownMockAgent } from './utils/index.js' + +const registries: Registries = { + default: 'https://registry.npmjs.org/', +} + +const fetchFromRegistry = createFetchFromRegistry({}) +const getAuthHeaderValueByURI = (): undefined => undefined + +function makeVerifierOpts (overrides: Partial[0]> = {}): Parameters[0] { + return { + registries, + fetchOpts: { + fetch: fetchFromRegistry, + retry: { retries: 0 }, + timeout: 60_000, + fetchWarnTimeoutMs: 10_000, + }, + getAuthHeaderValueByURI, + cacheDir: temporaryDirectory(), + now: Date.UTC(2026, 0, 1), + ...overrides, + } +} + +function makeTarballResolution (name: string, version: string): Resolution { + return { + integrity: 'sha512-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==', + tarball: `https://registry.npmjs.org/${name}/-/${name}-${version}.tgz`, + } as unknown as Resolution +} + +afterEach(async () => { + await teardownMockAgent() +}) + +beforeEach(async () => { + await setupMockAgent() +}) + +test('createNpmResolutionVerifier() returns undefined when no policy is active', () => { + expect(createNpmResolutionVerifier(makeVerifierOpts())).toBeUndefined() +}) + +test('createNpmResolutionVerifier() flags a trustedPublisher → provenance downgrade', async () => { + // 0.0.1 was published by a trustedPublisher → rank 2. + // 0.0.2 is provenance-only (rank 1, weaker) → downgrade vs 0.0.1. + // This is exactly the case the resolver-time trustChecks unit tests + // cover, but routed through the lockfile verifier. The verifier must + // not pass simply because 0.0.2 has *some* attestation. + const meta = { + name: 'demo', + 'dist-tags': { latest: '0.0.2' }, + versions: { + '0.0.1': { + name: 'demo', + version: '0.0.1', + dist: { tarball: 'https://registry.npmjs.org/demo/-/demo-0.0.1.tgz', shasum: 'aa' }, + _npmUser: { trustedPublisher: { id: 'gha', oidcConfigId: 'cfg' } }, + }, + '0.0.2': { + name: 'demo', + version: '0.0.2', + dist: { + tarball: 'https://registry.npmjs.org/demo/-/demo-0.0.2.tgz', + shasum: 'bb', + attestations: { provenance: { url: 'https://example.org/p' } }, + }, + }, + }, + time: { + '0.0.1': '2025-01-01T00:00:00.000Z', + '0.0.2': '2025-06-01T00:00:00.000Z', + }, + modified: '2025-06-01T00:00:00.000Z', + } + const pool = getMockAgent().get('https://registry.npmjs.org') + pool.intercept({ path: '/demo', method: 'GET' }).reply(200, meta).persist() + + const verifier = createNpmResolutionVerifier(makeVerifierOpts({ + trustPolicy: 'no-downgrade', + }))! + expect(verifier).toBeDefined() + + const result = await verifier.verify(makeTarballResolution('demo', '0.0.2'), { name: 'demo', version: '0.0.2' }) + expect(result).toMatchObject({ + ok: false, + code: 'TRUST_DOWNGRADE', + }) +}) + +test('createNpmResolutionVerifier() passes a same-evidence-level version', async () => { + // 0.0.1 had provenance, 0.0.2 still has provenance → no downgrade. + // Verifies the trust check isn't over-aggressive for stable evidence. + const meta = { + name: 'demo', + 'dist-tags': { latest: '0.0.2' }, + versions: { + '0.0.1': { + name: 'demo', + version: '0.0.1', + dist: { + tarball: 'https://registry.npmjs.org/demo/-/demo-0.0.1.tgz', + shasum: 'aa', + attestations: { provenance: { url: 'https://example.org/p1' } }, + }, + }, + '0.0.2': { + name: 'demo', + version: '0.0.2', + dist: { + tarball: 'https://registry.npmjs.org/demo/-/demo-0.0.2.tgz', + shasum: 'bb', + attestations: { provenance: { url: 'https://example.org/p2' } }, + }, + }, + }, + time: { + '0.0.1': '2025-01-01T00:00:00.000Z', + '0.0.2': '2025-06-01T00:00:00.000Z', + }, + modified: '2025-06-01T00:00:00.000Z', + } + const pool = getMockAgent().get('https://registry.npmjs.org') + pool.intercept({ path: '/demo', method: 'GET' }).reply(200, meta).persist() + + const verifier = createNpmResolutionVerifier(makeVerifierOpts({ + trustPolicy: 'no-downgrade', + }))! + const result = await verifier.verify(makeTarballResolution('demo', '0.0.2'), { name: 'demo', version: '0.0.2' }) + expect(result).toEqual({ ok: true }) +}) + +test('createNpmResolutionVerifier() abbreviated shortcut requires the pinned version to be in metadata', async () => { + // Package's `modified` is well before the cutoff (default 1-day window + // means modified=2010 is fine), but `0.0.2` was unpublished and is no + // longer in `versions`. The shortcut must NOT return the package-level + // `modified` for that version — that would be a fail-open on a + // missing pin. The verifier should fall through to the deeper layers + // and end up reporting a violation (no source could surface the time). + const abbreviatedMeta = { + name: 'unpublished-pkg', + 'dist-tags': {}, + versions: { + '0.0.1': { + name: 'unpublished-pkg', + version: '0.0.1', + dist: { tarball: 'https://registry.npmjs.org/unpublished-pkg/-/unpublished-pkg-0.0.1.tgz', shasum: 'aa' }, + }, + }, + modified: '2010-01-01T00:00:00.000Z', + } + const fullMeta = { + ...abbreviatedMeta, + time: { '0.0.1': '2010-01-01T00:00:00.000Z' }, + } + const pool = getMockAgent().get('https://registry.npmjs.org') + pool.intercept({ path: '/unpublished-pkg', method: 'GET' }).reply(200, abbreviatedMeta).persist() + pool.intercept({ path: '/-/npm/v1/attestations/unpublished-pkg@0.0.2', method: 'GET' }).reply(404, {}).persist() + + const verifier = createNpmResolutionVerifier(makeVerifierOpts({ + minimumReleaseAge: 1440, // 1 day + }))! + const result = await verifier.verify( + makeTarballResolution('unpublished-pkg', '0.0.2'), + { name: 'unpublished-pkg', version: '0.0.2' } + ) + expect(result).toMatchObject({ + ok: false, + code: 'MINIMUM_RELEASE_AGE_VIOLATION', + }) + + // Sanity check: the unrelated full meta isn't used here because the + // abbreviated shortcut won't fire (version missing), and the deeper + // layers also have no entry for 0.0.2. Keep `fullMeta` in scope so + // future test additions can wire it in without redefining. + expect(fullMeta.versions['0.0.1'].version).toBe('0.0.1') +}) + +test('createNpmResolutionVerifier() ignoreMissingTimeField passes the entry when no source surfaces a timestamp', async () => { + // Mirrors the resolver-side `pickMatchingVersionFinal` warn-and-skip + // behavior: when the registry strips the per-version `time` field and + // the user has opted into `minimumReleaseAgeIgnoreMissingTime`, the + // verifier shouldn't be stricter than fresh resolution. + const abbreviatedMeta = { + name: 'time-free-pkg', + 'dist-tags': {}, + versions: { + '1.0.0': { + name: 'time-free-pkg', + version: '1.0.0', + dist: { tarball: 'https://registry.npmjs.org/time-free-pkg/-/time-free-pkg-1.0.0.tgz', shasum: 'aa' }, + }, + }, + modified: '2010-01-01T00:00:00.000Z', + } + const pool = getMockAgent().get('https://registry.npmjs.org') + // Full meta also lacks `time`, so no layer surfaces a publish timestamp. + pool.intercept({ path: '/time-free-pkg', method: 'GET' }).reply(200, abbreviatedMeta).persist() + pool.intercept({ path: '/-/npm/v1/attestations/time-free-pkg@1.0.0', method: 'GET' }).reply(404, {}).persist() + + const verifier = createNpmResolutionVerifier(makeVerifierOpts({ + minimumReleaseAge: 1440, + ignoreMissingTimeField: true, + }))! + const result = await verifier.verify( + makeTarballResolution('time-free-pkg', '1.0.0'), + { name: 'time-free-pkg', version: '1.0.0' } + ) + expect(result).toEqual({ ok: true }) +}) + +test('createNpmResolutionVerifier() canTrustPastCheck rejects when the trust-exclude list shrinks', () => { + const verifier = createNpmResolutionVerifier(makeVerifierOpts({ + trustPolicy: 'no-downgrade', + trustPolicyExclude: ['foo'], + }))! + // Same policy → trust. + expect(verifier.canTrustPastCheck({ + minimumReleaseAge: 0, + minimumReleaseAgeExclude: [], + trustPolicy: 'no-downgrade', + trustPolicyExclude: ['foo'], + trustPolicyIgnoreAfter: null, + })).toBe(true) + // Cached run had a wider exclude list (today's is stricter) → invalidate. + expect(verifier.canTrustPastCheck({ + minimumReleaseAge: 0, + minimumReleaseAgeExclude: [], + trustPolicy: 'no-downgrade', + trustPolicyExclude: ['foo', 'bar'], + trustPolicyIgnoreAfter: null, + })).toBe(false) +}) diff --git a/resolving/npm-resolver/test/publishedBy.test.ts b/resolving/npm-resolver/test/publishedBy.test.ts index 2f84c776d2..f36b1dcb7f 100644 --- a/resolving/npm-resolver/test/publishedBy.test.ts +++ b/resolving/npm-resolver/test/publishedBy.test.ts @@ -90,12 +90,14 @@ test('request metadata when the one in cache does not have a version satisfying expect(resolveResult!.id).toBe('bad-dates@1.0.0') }) -test('do not pick version that does not satisfy the date requirement even if it is loaded from cache and requested by exact version', async () => { +test('reports an immature pick via policyViolation even when loaded from cache and requested by exact version', async () => { const cacheDir = temporaryDirectory() const fooMeta = { 'dist-tags': {}, versions: { '1.0.0': { + name: 'foo', + version: '1.0.0', dist: { integrity: 'sha512-9Qa5b+9n69IEuxk4FiNcavXqkixb9lD03BLtdTeu2bbORnLZQrw+pR/exiSg7SoODeu08yxS47mdZa9ddodNwQ==', shasum: '857db584a1ba5d1cb2980527fc3b6c435d37b0fd', @@ -118,17 +120,25 @@ test('do not pick version that does not satisfy the date requirement even if it .intercept({ path: '/foo', method: 'GET' }) .reply(200, fooMeta) + // The resolver no longer throws on immature picks — it falls back to the + // lowest match in range and flags the result with `policyViolation`. The + // outer caller (install / dlx / self-update) decides what to do with it. const { resolveFromNpm } = createResolveFromNpm({ storeDir: temporaryDirectory(), cacheDir, filterMetadata: true, fullMetadata: true, registries, - strictPublishedByCheck: true, }) - await expect(resolveFromNpm({ alias: 'foo', bareSpecifier: '1.0.0' }, { + const result = await resolveFromNpm({ alias: 'foo', bareSpecifier: '1.0.0' }, { publishedBy: new Date('2015-08-17T19:26:00.508Z'), - })).rejects.toThrow(/Version 1\.0\.0 \(released .+\) of foo does not meet the minimumReleaseAge constraint/) + }) + expect(result!.id).toBe('foo@1.0.0') + expect(result!.policyViolation).toMatchObject({ + name: 'foo', + version: '1.0.0', + code: 'MINIMUM_RELEASE_AGE_VIOLATION', + }) }) test('should skip time field validation for excluded packages', async () => { @@ -320,20 +330,20 @@ test('ignoreMissingTimeField=true skips maturity check from disk-cached metadata expect(resolveResult!.id).toBe('is-positive@3.1.0') }) -test('strictPublishedByCheck=true does not rethrow ERR_PNPM_MISSING_TIME from the version-spec cache path', async () => { +test('falls through to the registry fetch when cached abbreviated meta lacks time on the version-spec cache path', async () => { // Regression test for the bug where the version-spec cache fast path // (`!opts.includeLatestTag && spec.type === 'version'`) in pickPackage - // would rethrow ERR_PNPM_MISSING_TIME under strictPublishedByCheck, instead - // of falling through to the registry-fetch path like the adjacent mtime-gated - // cache block does. The fix makes the two catch blocks consistent. + // would rethrow ERR_PNPM_MISSING_TIME under what used to be + // strictPublishedByCheck, instead of falling through to the registry-fetch + // path like the adjacent mtime-gated cache block does. The fix makes the + // two catch blocks consistent — both now always swallow and fall through. // // Setup: cache abbreviated metadata (no per-version `time` field) for the // package, then request an exact-version pin that IS present in the cached // meta.versions. The version-spec fast path will try pickMatchingVersionFast // against the cached meta, which throws MISSING_TIME because the abbreviated - // form lacks `time`. Before the fix this would rethrow. After the fix it - // falls through to the registry fetch, which returns full metadata with time, - // and resolution succeeds. + // form lacks `time`. The catch falls through to the registry fetch, which + // returns full metadata with time, and resolution succeeds. const cacheDir = temporaryDirectory() const abbrevCacheDir = path.join(cacheDir, `${ABBREVIATED_META_DIR}/registry.npmjs.org`) fs.mkdirSync(abbrevCacheDir, { recursive: true }) @@ -363,7 +373,6 @@ test('strictPublishedByCheck=true does not rethrow ERR_PNPM_MISSING_TIME from th storeDir: temporaryDirectory(), cacheDir, registries, - strictPublishedByCheck: true, ignoreMissingTimeField: true, }) @@ -377,12 +386,11 @@ test('strictPublishedByCheck=true does not rethrow ERR_PNPM_MISSING_TIME from th expect(resolveResult!.id).toBe('is-positive@3.0.0') }) -test('strictPublishedByCheck=true with default ignoreMissingTimeField does not rethrow ERR_PNPM_MISSING_TIME from the version-spec cache path', async () => { +test('falls through to the registry fetch even with default ignoreMissingTimeField on the version-spec cache path', async () => { // Companion to the test above: same scenario but with the default - // ignoreMissingTimeField (false). The catch-block fix must hold regardless - // of the ignore flag — MISSING_TIME from cached abbreviated meta should - // never escape the catch under strict mode, so resolution falls through to - // the registry fetch and succeeds with full (time-bearing) metadata. + // ignoreMissingTimeField (false). MISSING_TIME from cached abbreviated + // meta should never escape the catch — resolution falls through to the + // registry fetch and succeeds with full (time-bearing) metadata. const cacheDir = temporaryDirectory() const abbrevCacheDir = path.join(cacheDir, `${ABBREVIATED_META_DIR}/registry.npmjs.org`) fs.mkdirSync(abbrevCacheDir, { recursive: true }) @@ -406,7 +414,6 @@ test('strictPublishedByCheck=true with default ignoreMissingTimeField does not r storeDir: temporaryDirectory(), cacheDir, registries, - strictPublishedByCheck: true, }) const resolveResult = await resolveFromNpm({ alias: 'is-positive', bareSpecifier: '3.0.0' }, { diff --git a/resolving/npm-resolver/test/resolveJsr.test.ts b/resolving/npm-resolver/test/resolveJsr.test.ts index f33e781d72..db680acea2 100644 --- a/resolving/npm-resolver/test/resolveJsr.test.ts +++ b/resolving/npm-resolver/test/resolveJsr.test.ts @@ -128,3 +128,39 @@ test('resolveFromJsr() on jsr with packages without scope', async () => { code: 'ERR_PNPM_MISSING_JSR_PACKAGE_SCOPE', }) }) + +test('resolveFromJsr() returns the immature pick with policyViolation when publishedBy excludes it', async () => { + // jsr-rus-greet's 0.0.3 was published 2024-11-16; passing a `publishedBy` + // before that makes the version immature relative to the cutoff. The + // resolver always falls back to the requested version and flags the + // result with `policyViolation`; the install command (or other caller) + // decides what to do with it. This is the named-registry / jsr path's + // coverage for inline violation reporting. + const slash = '%2F' + const defaultPool = getMockAgent().get(registries.default.replace(/\/$/, '')) + defaultPool.intercept({ path: `/@jsr${slash}rus__greet`, method: 'GET' }).reply(404, {}) + const jsrPool = getMockAgent().get(registries['@jsr'].replace(/\/$/, '')) + jsrPool.intercept({ path: `/@jsr${slash}rus__greet`, method: 'GET' }).reply(200, jsrRusGreetMeta) + + const cacheDir = temporaryDirectory() + const { resolveFromJsr } = createResolveFromNpm({ + storeDir: temporaryDirectory(), + cacheDir, + registries, + }) + const result = await resolveFromJsr( + { alias: '@rus/greet', bareSpecifier: 'jsr:0.0.3' }, + { + publishedBy: new Date('2020-01-01T00:00:00Z'), + } + ) + + expect(result).toMatchObject({ + id: '@jsr/rus__greet@0.0.3', + policyViolation: { + name: '@jsr/rus__greet', + version: '0.0.3', + code: 'MINIMUM_RELEASE_AGE_VIOLATION', + }, + }) +}) diff --git a/resolving/resolver-base/src/index.ts b/resolving/resolver-base/src/index.ts index 3546948904..513c5b8a58 100644 --- a/resolving/resolver-base/src/index.ts +++ b/resolving/resolver-base/src/index.ts @@ -129,6 +129,26 @@ export interface ResolutionVerifier { canTrustPastCheck: (cachedPolicy: Record) => boolean } +/** + * A `ResolutionVerifier`'s rejection materialized for one (name, + * version, resolution) entry. The install side aggregates these across + * every active verifier on the freshly-resolved tree and either prompts + * the user, persists them (e.g. into `minimumReleaseAgeExclude`), or + * aborts. Code is the verifier-defined error code + * (`MINIMUM_RELEASE_AGE_VIOLATION`, `TRUST_DOWNGRADE`, etc.) — the + * install command filters by code to decide downstream UX. Lifted here + * (rather than in deps-installer) so both deps-resolver and + * deps-installer can share one shape; future resolver packages plug in + * without needing the deps-installer dependency. + */ +export interface ResolutionPolicyViolation { + name: string + version: string + resolution: Resolution + code: string + reason: string +} + /** Concrete platform selector used when picking a variant from a VariationsResolution. */ export interface PlatformSelector { os: string @@ -197,6 +217,22 @@ export interface ResolveResult { resolvedVia: string normalizedBareSpecifier?: string alias?: string + /** + * Set when the resolver picked this version despite a policy + * violation (e.g. immature relative to `publishedBy`, trust + * downgrade detected by `failIfTrustDowngraded`). The resolver + * already has the metadata it needs to decide, so reporting inline + * here avoids the install layer having to re-scan the tree and + * re-fetch the same metadata. The deps-resolver aggregates these + * across every resolve call into a single set the install command + * can react to. + * + * `resolution` on the violation is the same `resolution` field + * above — supplied for symmetry with `ResolutionPolicyViolation` + * entries that flow out of `verifyLockfileResolutions` for + * lockfile-only paths. + */ + policyViolation?: ResolutionPolicyViolation } export interface WorkspacePackage { diff --git a/store/commands/src/store/cleanLockfileVerifiedCache.ts b/store/commands/src/store/cleanLockfileVerifiedCache.ts new file mode 100644 index 0000000000..02455733fb --- /dev/null +++ b/store/commands/src/store/cleanLockfileVerifiedCache.ts @@ -0,0 +1,30 @@ +import fs from 'node:fs' +import path from 'node:path' +import util from 'node:util' + +// Mirrors the constant in +// `installing/deps-installer/src/install/verifyLockfileResolutionsCache.ts`. +// Kept in sync by hand (both sides own a small string; introducing a +// shared package just for this would outweigh the cost of the duplicate). +const LOCKFILE_VERIFIED_CACHE_FILE = 'lockfile-verified.jsonl' + +/** + * Remove the lockfile-verification cache JSONL written by the install + * command's resolution-policy verifier. Pruning the store invalidates + * derived state; a stale verification record under a different + * policy/lockfile-content key would otherwise survive into the next + * install (still correct because of the cache's identity comparator, + * but visually leaks "alien" files into `cacheDir`). + * + * Silent on a missing file — prune is idempotent and the cache may + * never have been written in the first place. + */ +export function cleanLockfileVerifiedCache (cacheDir: string): void { + const cacheFilePath = path.join(cacheDir, LOCKFILE_VERIFIED_CACHE_FILE) + try { + fs.unlinkSync(cacheFilePath) + } catch (err: unknown) { + if (util.types.isNativeError(err) && 'code' in err && err.code === 'ENOENT') return + throw err + } +} diff --git a/store/commands/src/store/storePrune.ts b/store/commands/src/store/storePrune.ts index 557d7d5acf..19698507b8 100644 --- a/store/commands/src/store/storePrune.ts +++ b/store/commands/src/store/storePrune.ts @@ -3,6 +3,7 @@ import { streamParser } from '@pnpm/logger' import type { StoreController } from '@pnpm/store.controller-types' import { cleanExpiredDlxCache } from './cleanExpiredDlxCache.js' +import { cleanLockfileVerifiedCache } from './cleanLockfileVerifiedCache.js' import type { ReporterFunction } from './types.js' export async function storePrune ( @@ -29,6 +30,8 @@ export async function storePrune ( now: new Date(), }) + cleanLockfileVerifiedCache(opts.cacheDir) + if (opts.globalPkgDir) { cleanOrphanedInstallDirs(opts.globalPkgDir) } diff --git a/store/connection-manager/src/createNewStoreController.ts b/store/connection-manager/src/createNewStoreController.ts index 18a610f459..5e09089eb8 100644 --- a/store/connection-manager/src/createNewStoreController.ts +++ b/store/connection-manager/src/createNewStoreController.ts @@ -51,6 +51,8 @@ export type CreateNewStoreControllerOptions = CreateResolverOptions & Pick From e74a6b7e4879da0134e93bf078c02dd0109c418d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kh=E1=BA=A3i?= Date: Mon, 18 May 2026 15:21:42 +0700 Subject: [PATCH 012/169] test(pacquet): coverage (#11710) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test: fill coverage holes across lockfile, package-manifest, fs, and friends Adds ~44 unit and integration tests to close the easier targets in the coverage analysis at #339-style boundaries. Touches: - `lockfile::resolved_dependency`: cover `as_alias` / `ver_peer` for non-matching variants, alias / parse error variants, `TryFrom`, serialize-alias / serialize-link, `From / From`, and a Display vs. `String` round-trip. - `lockfile::freshness`: cover the plural arms of `SpecDiff::Display` and the comma separator inside the removed/modified loops. - `lockfile::pkg_name`: cover `TryFrom` and `TryFrom` happy and empty-input paths. - `lockfile::snapshot_dep_ref`: cover `ver_peer` and `From`. - `lockfile::save_lockfile`: cover the `CreateDir` / `RemoveFile` / `RenameFile` error classifications by planting a regular file or directory where the writer expects a file or a writable path. - `registry::package`: cover `PartialEq`, `latest`, and `pinned_version` happy and no-match paths. - `graph-hasher::engine_name`: cover `detect_node_version` / `detect_node_major` when `node` is on PATH; skip cleanly otherwise. - `graph-hasher::object_hasher`: cover the null / bool / array arms of the bytestream serializer. - `patching::apply`: cover non-NotFound read-patch error, partial-delete non-empty result, unsupported rename/copy operation, and `Create` with an unwritable parent. - `workspace-state`: cover the `CreateDir` / `ReadFile` / `ParseJson` error variants. - `package-manifest`: cover `from_path` ENOENT, `add_dependency` on a non-object field, `safe_read_package_json_from_dir` non-NotFound IO error, and `convert_engines_runtime_to_dependencies` for unsupported shapes. - `fs::file_mode`: cover `EXEC_MASK` / `EXEC_MODE` constants, `is_executable` for all positions, and `make_file_executable` on Unix. - `executor`: cover `execute_shell` happy and non-zero-exit paths. - `cli/tests/run.rs`: new integration tests for `pacquet run`, including argument forwarding and `--if-present`. Also bumps `hex_decode` in `graph-hasher::tests` to use `is_multiple_of` so test-time clippy stays clean under Rust 1.95. https://claude.ai/code/session_01D8WBTfQzTpsZsRknrzwNKL * style(package-manifest): drop trailing comma in single-line `assert!` Dylint's perfectionist::macro-trailing-comma rule rejected the single- line `assert!` formatting I'd accidentally inherited from the multi- line shape — remove the trailing comma so the lint clears. https://claude.ai/code/session_01D8WBTfQzTpsZsRknrzwNKL * test(graph-hasher): make `node` a hard prerequisite for detect tests The two `detect_node_*` tests previously skipped silently when `node` was missing from `PATH`. `node` is a documented prerequisite of the test suite (see `pacquet/CONTRIBUTING.md`'s setup section), so a missing binary is a test-env bug to surface, not a condition to paper over. Switch the `let Some(...) = ... else { return }` skips to `expect` so a missing `node` fails loudly with a clear message. https://claude.ai/code/session_01D8WBTfQzTpsZsRknrzwNKL * test(cli,patching): switch fixtures to `json!` and `text_block_fnl!` Addresses six review comments on #11710: - `cli/tests/run.rs`: the three `package.json` fixtures used `format!("{{ ... }}", marker = …)` with literal-brace escapes, which is awkward and brittle when path strings contain JSON-special characters. Switched to `json!({...}).to_string()` so serde handles escaping and the shape reads as JSON, not as a brace-escaped template. - `patching/src/apply/tests.rs`: the three patch fixtures I added used `"\...\n"` raw strings. Converted them to `text_block_fnl!`, matching the convention the rest of the workspace (lockfile, package-manager, testing-utils) uses for multi-line fixture text. The `_fnl` variant keeps the trailing newline that git-diff parsers expect. Pre-existing fixtures in the same file (`IS_POSITIVE_PATCH`, etc.) were not touched — keeping the scope to the lines reviewers flagged. https://claude.ai/code/session_01D8WBTfQzTpsZsRknrzwNKL * style(registry): use `assert_ne!` over `assert!(lhs != rhs)` Idiomatic Rust and produces a better failure message when the assertion trips. Caught during a self-review pass over #11710. https://claude.ai/code/session_01D8WBTfQzTpsZsRknrzwNKL * test: address zkochan + Copilot review feedback on #11710 - Move executor + fs::file_mode unit tests from inline modules to the project's standard `src//tests.rs` layout. The convention is documented in `CODE_STYLE_GUIDE.md` under "Unit test file layout"; zkochan flagged the violation in the executor file. - Gate `pacquet-executor::tests` on `cfg(all(test, unix))` at the declaration site so the `use super::execute_shell` import isn't dead on Windows. Copilot flagged that `clippy --tests --deny warnings` would fail there. - Drop `execute_shell_propagates_nonzero_exit_as_ok`. Copilot flagged that it regression-pinned a behavior that *should* change: pnpm/npm `run` exits with the script's status, but pacquet's wrapper currently returns `Ok(())` regardless. A fix belongs in its own PR; until then we should not add tests that block that fix. - Quote the temp-path redirect targets in the `cli/tests/run.rs` shell fixtures so a tempdir path containing a space (`/var/folders/...` on macOS) doesn't split the `touch` / redirect argument. Copilot flagged the unquoted paths. The `fs::file_mode::make_file_executable_sets_exec_bits` test keeps `#[cfg(unix)]` at the test site (and moves its `make_file_executable` import inside) rather than gating the whole `tests` module — the other two tests (`exec_constants_pin_pnpm_layout`, `is_executable_matches_any_exec_bit`) are platform-neutral and should still run on Windows. Copilot's concern about `detect_node_version`'s `expect` (in `engine_name.rs`) is intentionally not addressed: KSXGitHub asked for the hard-fail in 5d0987c on grounds that `node` is a documented prerequisite of the test suite, and the human reviewer's call wins over the bot's. https://claude.ai/code/session_01D8WBTfQzTpsZsRknrzwNKL --------- Co-authored-by: Claude --- Cargo.lock | 1 + pacquet/crates/cli/tests/run.rs | 117 +++++++++++++++ pacquet/crates/executor/src/lib.rs | 3 + pacquet/crates/executor/src/tests.rs | 11 ++ pacquet/crates/fs/src/file_mode.rs | 3 + pacquet/crates/fs/src/file_mode/tests.rs | 38 +++++ .../crates/graph-hasher/src/engine_name.rs | 26 +++- pacquet/crates/graph-hasher/src/tests.rs | 40 +++++- .../crates/lockfile/src/freshness/tests.rs | 24 ++++ pacquet/crates/lockfile/src/pkg_name/tests.rs | 26 ++++ .../lockfile/src/resolved_dependency/tests.rs | 134 +++++++++++++++++- .../lockfile/src/save_lockfile/tests.rs | 92 ++++++++++++ .../lockfile/src/snapshot_dep_ref/tests.rs | 23 +++ pacquet/crates/package-manifest/src/tests.rs | 86 ++++++++++- pacquet/crates/patching/Cargo.toml | 1 + pacquet/crates/patching/src/apply/tests.rs | 120 ++++++++++++++++ pacquet/crates/registry/src/package/tests.rs | 64 +++++++++ pacquet/crates/workspace-state/src/tests.rs | 75 +++++++++- 18 files changed, 877 insertions(+), 7 deletions(-) create mode 100644 pacquet/crates/cli/tests/run.rs create mode 100644 pacquet/crates/executor/src/tests.rs create mode 100644 pacquet/crates/fs/src/file_mode/tests.rs diff --git a/Cargo.lock b/Cargo.lock index 805f7dd60b..89b2eb1f74 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2305,6 +2305,7 @@ dependencies = [ "pretty_assertions", "sha2", "tempfile", + "text-block-macros", ] [[package]] diff --git a/pacquet/crates/cli/tests/run.rs b/pacquet/crates/cli/tests/run.rs new file mode 100644 index 0000000000..fad051ec20 --- /dev/null +++ b/pacquet/crates/cli/tests/run.rs @@ -0,0 +1,117 @@ +use assert_cmd::prelude::*; +use command_extra::CommandExtra; +use pacquet_testing_utils::bin::CommandTempCwd; +use serde_json::json; +use std::fs; + +/// `pacquet run