mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-28 09:55:39 -04:00
d2b42c2dfc87bf2e822e83d7fcbd813698d1fd9b
11825 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Kochan
|
d2b42c2dfc |
fix(pacquet): per-level preferred-version fold + all-importers hoist rounds (#12357)
## Summary Two parity changes for pacquet's resolver, continuing https://github.com/pnpm/pnpm/issues/12266. Measured on the monorepo (fresh state, `install --lockfile-only`, back-to-back vs **pnpm 11.6.0**), the real-lockfile document diff drops from **128 to 5 changed lines** (re-measured after rebasing over #12361/#12362: **132 → 11**, where 8 of the 11 are a divergence the pacquet side of #12362 itself introduced — see the analysis on pnpm/pnpm#12266 — and 3 are the known cycle-closing-edge gap). ### 1. Per-level preferred-version fold pnpm extends the preferred-versions map per resolution level: after a package's direct dependencies settle, their `(name, version)` pairs join the map the *children's* subtree resolutions pick against ([resolveDependencies.ts#L717-L746](https://github.com/pnpm/pnpm/blob/ce9c096e8e/installing/deps-resolver/src/resolveDependencies.ts#L717-L746)). So `signed-varint`'s `varint@~5.0.0` dedupes to the `varint@5.0.0` its parent pinned as a sibling instead of drifting to `5.0.2`. pacquet picked against a static seed only; besides `varint`/`es-abstract`, this turned out to drive the remaining `jest`/`@types/node` duplicate variants too. - The walk resolves a whole sibling level before any child subtree starts (upstream's postponed-resolution barrier): `resolve_node` splits into `resolve_node_seed` + `walk_node_children`. - Each level layers its versions onto a new `PreferredVersionsOverlay` (O(1) `Arc`-chained layers in `resolver-base`); the npm picker folds the per-name view in as plain `version` selectors at both registry seams. - The overlay's per-name view joins the per-wanted dedup cache key; lockfile-reuse subtrees keep the no-overlay path (exact pins). ### 2. Hoist rounds across all importers (deterministic barrier, same logic as pnpm) pnpm resolves **every importer's initial wave before any peer hoist**, then repeats global hoist rounds (per round: each importer's required-peer loop to a fixpoint, then one optional-peer hoist) until no importer hoists ([resolveDependencies.ts#L335-L445](https://github.com/pnpm/pnpm/blob/ce9c096e8e/installing/deps-resolver/src/resolveDependencies.ts#L335-L445)). pacquet ran each importer's whole hoist loop before the next importer's initial wave, so an early importer's optional-peer pick couldn't see versions a later importer resolves — `@cyclonedx`'s `spdx-expression-parse` hoisted `3.0.1` where pnpm's barrier-visible map picks `4.0.0`. `resolve_importer_with_workspace` is now an `ImporterHoistState` (`init` / `run_required_round` / `hoist_optional_round`) driven by `resolve_workspace` in upstream's exact phase order. Both implementations are deterministic here; the rule is identical. ## Verification - New regression test `child_resolution_prefers_parent_level_sibling_versions` (fails with the fold disabled) + full `resolving-*`, `package-manager`, `cli` suites: 1,242 tests pass; clippy `--deny warnings`, rustfmt, typos clean. - Whole-monorepo diff vs fresh pnpm 11.6.0: 128 → 5 changed lines; consecutive pacquet runs byte-identical. |
||
|
Zoltan Kochan
|
9b35a6004e |
fix(deps-resolver): make shared children resolution deterministic (#12362)
## Summary - Make shared package child resolution deterministic by choosing the owner by depth, importer order, and parent path instead of async completion timing. - Keep non-owner and stale occurrences lazy while reusing the current owner children and missing-peer context. - Port the same behavior to pacquet and add TypeScript plus Rust regression coverage. ## Verification - `pnpm --filter @pnpm/installing.deps-resolver test test/resolveDependencyTree.test.ts` - `cargo test -p pacquet-resolving-deps-resolver` - Pre-push hook: TypeScript typecheck, pnpm bundle, lint, spellcheck, meta lint, cargo fmt, cargo doc, cargo dylint, taplo format check Fixes pnpm/pnpm#12358 |
||
|
Zoltan Kochan
|
43b5bf7520 |
perf(pacquet): cache build metadata during installs (#12360)
* perf(pacquet): cache build metadata during installs * fix(pacquet): satisfy clippy for build helpers * fix(pacquet): reduce prefetch row type complexity |
||
|
Zoltan Kochan
|
2aa2eaa6ff |
feat(pacquet): hand custom resolvers the currentPkg of a re-resolving edge (#12361)
* feat(pacquet): hand custom resolvers the currentPkg of a re-resolving edge
Custom resolvers received currentPkg: null on every call because nothing
populated ResolveOptions.current_pkg. Mirror pnpm's hand-off
(currentPkg: extendedWantedDep.infoFromLockfile in resolveDependencies):
- the tree walker now derives each edge's prior lockfile snapshot key
(importer refs satisfies-gated like pnpm's referenceSatisfiesWantedSpec;
transitive keys from the parent's snapshot) and, when the edge
re-resolves anyway, threads the recorded entry into the per-call
ResolveOptions as currentPkg
- a Registry ({integrity}-only) entry regains its derived tarball URL,
like pnpm's pkgSnapshotToResolution; pick_registry_for_package moved
from pacquet-resolving-npm-resolver to pacquet-lockfile (next to
npm_tarball_url) so the deps-resolver can route scoped packages, with
a re-export keeping existing callers unchanged
- children of a freshly resolved parent that landed back on its
recorded version keep their prior refs for currentPkg purposes
(pnpm's non-updated parentPkg arm) via the new ReuseSource::PriorOnly;
subtree reuse semantics are unchanged
- the per-wanted memo key includes the prior key so two edges sharing a
specifier but recording different versions never share a
currentPkg-dependent result
The e2e test ports upstream's 'custom resolver receives currentPkg on
subsequent installs': a forced re-resolve receives the prior entry with
the re-derived tarball URL, and echoing it back keeps the pinned
version.
* fix(pacquet): gate prior child refs on the canonical dep-path id
The 'parent landed back on its recorded entry' check compared the
recorded snapshot key against the raw resolver id, which is not the
canonical dep-path form: build_pkg_id_with_patch_hash may append a
(patch_hash=...) suffix and name@-prefix file:/git/tarball ids. Extract
the comparison into landed_on_prior_entry, which strips suffixes from
both sides (the recorded key's peers/patch hash via without_peer, the
resolved id via remove_suffix) so the gate keys on which package
version the parent is, like pnpm's parentPkg.updated flag.
|
||
|
dependabot[bot]
|
52148e6916 |
chore(cargo): bump tabled from 0.20.0 to 0.21.0 (#12352)
Bumps [tabled](https://github.com/zhiburt/tabled) from 0.20.0 to 0.21.0. - [Changelog](https://github.com/zhiburt/tabled/blob/master/CHANGELOG.md) - [Commits](https://github.com/zhiburt/tabled/commits) --- updated-dependencies: - dependency-name: tabled dependency-version: 0.21.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Alessio Attilio
|
a9d2ec8817 |
feat(pacquet): support pnpmfile custom resolver hooks (adapter, IPC, chain integration, force-refresh) (#12153)
Port pnpm's custom resolver hooks to the Rust pacquet engine: a pnpmfile can export a top-level `resolvers` array whose entries override built-in dependency resolution and force re-resolution when needed. See pnpm/pnpm#10389 for the TypeScript-side feature request that motivated this port. ## What's included - **Hook contract** — `CustomResolver` trait (`canResolve` / `resolve` / `shouldRefreshResolution`) mirroring `hooks/types/src/index.ts`. All three methods are optional upstream, so the Node worker reports per-resolver capability flags in one IPC round trip and pacquet skips calls a resolver doesn't implement (mirrors pnpm's `if (!customResolver.canResolve || !customResolver.resolve) continue` and `checkCustomResolverForceResolve`'s hook filter). - **Node IPC** — the long-lived pnpmfile worker gained `resolvers` (capabilities) and `resolver` (method invocation) requests. Methods are invoked with `this` bound to the resolver object, like pnpm. Pending-request cleanup is cancellation-safe via an RAII guard. - **Adapter & chain integration** — `CustomResolverAdapter` bridges the JSON hook contract to the typed `Resolver` trait. Custom resolvers are built into the inner resolver chain ahead of the built-ins (upstream chain priority), inside the prefetching/observing wrappers so their tarball results get resolve-time prefetch and pnpr streaming. `canResolve` results are memoized keyed `alias@bareSpecifier`, exactly like pnpm's `getCustomResolverCacheKey`. A resolver-returned `manifest` passes through (pnpm spreads the whole hook result). Payloads match upstream: `prevSpecifier`, and resolve opts carry `lockfileDir` / `projectDir` / `preferredVersions` / `currentPkg`. - **`shouldRefreshResolution` semantics** — port of `checkCustomResolverForceResolve`: the hook receives the merged packages+snapshots entry (pnpm's in-memory `PackageSnapshot`), checks run concurrently with first-true/first-error short-circuit, and a throwing hook aborts the install (`PNPMFILE_FAIL`). A `true` verdict defeats both up-to-date optimizations, as documented in the hook's contract: - the prefer-frozen dispatch consults the hook (pnpm: `forceResolutionFromHook` → `needsFullResolution` blocks `isFrozenInstallPossible`) and routes to the fresh-resolve path with lockfile reuse disabled (`UpdateReuseScope::None`); - the optimistic repeat-install fast path now ports the pnpmfile branch of `patchesOrHooksAreModified`: the workspace state records the loaded pnpmfile list, and an added/removed/edited pnpmfile invalidates the mtime check. - **`CurrentPkg`** — added to `ResolveOptions`, matching upstream's `currentPkg` shape `{id, name?, version?, resolution, publishedAt?}` (camelCase). ## Tests - Adapter unit tests: missing `id`/`resolution`, invalid shapes, `canResolve` memoization, payload shapes, manifest passthrough. - `check_custom_resolver_force_resolve` unit tests: port of upstream's `checkCustomResolverForceResolve.ts` suite (capability filter, true/false/error propagation, merged snapshot payload). - Node IPC integration tests against a real pnpmfile: capabilities, `this` binding, round trips, error propagation, cancellation cleanup. - CLI e2e tests against the mock registry: custom resolver precedence over the npm resolver, `shouldRefreshResolution` re-resolving past an up-to-date lockfile, and a throwing hook failing the install. |
||
|
Zoltan Kochan
|
ce9c096e8e |
fix(pacquet): share a subtree's peer context across importers + record array-form engines (#12349)
* fix(lockfile): record array-form engines as index-keyed entries
A package.json with `"engines": ["node >= 0.2.0"]` (e.g.
jsonparse@1.3.1) records `engines: {'0': node >= 0.2.0}` in pnpm's
lockfile — the shape Object.entries() yields for an array. pacquet
read engines via as_object only and dropped the field.
Part of https://github.com/pnpm/pnpm/issues/12266.
* fix(deps-resolver): keep a shared subtree's peer context stable across importers
A package first resolved under one importer keeps that walk's
missing-peer report in pnpm (missingPeersOfChildrenByPkgId,
https://github.com/pnpm/pnpm/blob/a751c7f27d/installing/deps-resolver/src/resolveDependencies.ts#L193):
an importer revisiting the package neither recomputes its subtree's
missing peers nor re-hoists optional peers the first context already
satisfied — its occurrence then shares the first context's
peer-suffixed variant. pacquet re-walked every importer's tree from
scratch, so an importer without e.g. @types/node hoisted the highest
satisfying version (25.x) for a subtree the root importer had already
resolved against its own @types/node (22.x), forking the snapshot into
duplicate suffix variants (verified against pnpm 11.6.0 with a
three-importer repro on @pnpm/meta-updater).
The walk now records, per package, the importer whose tree first
resolved it and the missing-peer names of that first peer walk. The
per-importer hoist input drops a missing peer declared under a
foreign-claimed ancestor when the claiming walk did not report it
missing — misses the first context could not satisfy either (e.g.
clipanion's typanion under a root that only hoists it later) stay
visible to every importer, which then hoists its own copy exactly like
pnpm. The final workspace-wide pass stays unscoped so peer warnings
still cover every importer.
Whole-monorepo lockfile diff vs fresh pnpm 11.6.0: 445 → 128 changed
lines; the remaining divergence is two importer entries (jest's
@types/node under __utils__/eslint-config and @cyclonedx's
spdx-expression-parse) plus per-level preferred-version picks (varint,
es-abstract), tracked in the issue.
Part of https://github.com/pnpm/pnpm/issues/12266.
* perf(deps-resolver): snapshot the hoist-missing scope once per importer
The hoist loop cloned the claim and first-walk-missing maps on every
iteration; suppression only consults entries recorded by earlier
importers (an importer's own loop additions are self-claimed and
exempt), so one Arc-shared snapshot per importer is equivalent.
Addresses a review comment on the PR.
|
||
|
Zoltan Kochan
|
a751c7f27d |
fix(pacquet): close four lockfile-parity gaps (importer peer+dev merge, peer-vs-own-dep, peer overrides, optional-peer hoist) (#12345)
## Summary Four lockfile-parity fixes for pacquet, continuing https://github.com/pnpm/pnpm/issues/12266. Measured on the monorepo itself (fresh state, `install --lockfile-only`, back-to-back against the live registry vs **pnpm 11.6.0**), the real-lockfile document diff drops from **806 to 461 changed lines**. Two consecutive pacquet runs are byte-identical before and after. ### 1. Importer's regular dep wins over its own peer range (`fix(deps-resolver)`) An importer that declares the same alias in both `peerDependencies` and a regular group (e.g. `@pnpm/logger`: `workspace:*` devDependency + `catalog:` peer — 67 importers in this repo) resolved both specs, and the peer's registry resolution overwrote the workspace link in the importers section (`version: 1001.0.1` instead of `link:../../core/logger`, 182 diff lines). `importer_direct_wanted_specs` now merges the groups the way pnpm spreads them in [`getWantedDependencies`](https://github.com/pnpm/pnpm/blob/01b3d45ddb/installing/deps-resolver/src/getWantedDependencies.ts#L32-L43): peers first, then `dependencies` < `devDependencies` < `optionalDependencies`, one wanted spec per alias. The `@pnpm/logger` resolution tallies now match pnpm's exactly. ### 2. A package's peer survives when it also lists the name as a dependency (`fix(deps-resolver)`) Under `autoInstallPeers`, pnpm removes peer-shadowed names from a resolved package's `dependencies` **before** extracting peers ([resolveDependencies.ts#L1527-L1542](https://github.com/pnpm/pnpm/blob/01b3d45ddb/installing/deps-resolver/src/resolveDependencies.ts#L1527-L1542)), so the peer edge supplies the package and the depPath carries the suffix. pacquet did the inverse (dropped the peer, walked the own dep), so `@babel/parser` — whose packageExtensions-added `@babel/types` peer is also its regular dependency — lost its `(@babel/types@7.29.7)` suffix and its `peerDependencies:` block. Also aligns `extract_peer_dependencies` with [`peerDependenciesWithoutOwn`](https://github.com/pnpm/pnpm/blob/01b3d45ddb/installing/deps-resolver/src/resolveDependencies.ts#L1840-L1864): the package's own name counts as an own dep, and a `peerDependenciesMeta`-only entry becomes a peer only when `optional: true`. The non-`autoInstallPeers` arm (omit only peers resolvable from the parent scope) is not ported — pacquet's per-package children cache has no parent context; behavior there matches pnpm whenever the peer is not in scope. ### 3. Overrides apply to `peerDependencies` (`fix(package-manager)`) Ports the peer arm of [`overrideDepsOfPkg`](https://github.com/pnpm/pnpm/blob/01b3d45ddb/hooks/read-package-hook/src/createVersionsOverrider.ts#L68-L129): a matched peer is deleted on `-`, rewritten in place when the override value is a valid peer range, otherwise written into `dependencies` while the declared peer stays. Fixes e.g. `ajv@>=7.0.0-alpha.0 <8.18.0 → >=8.18.0` not rewriting peer ranges and `@yarnpkg/libzip` losing its `(@yarnpkg/fslib@3.x)` suffix. ### 4. Optional-peer hoist: run-extended preferred versions, meta-only peers excluded (`fix(deps-resolver)`) Corrects the model from pnpm/pnpm#12267. pnpm folds **every run-resolved version** into `ctx.allPreferredVersions` ([resolveDependencies.ts#L1483-L1488](https://github.com/pnpm/pnpm/blob/01b3d45ddb/installing/deps-resolver/src/resolveDependencies.ts#L1483-L1488), since pnpm/pnpm#7812) and `getHoistableOptionalPeers` reads that map after each wave — so an optional peer with a **real `peerDependencies` entry** (eslint's `jiti`) resolves against a provider anywhere in the freshly resolved tree. What keeps `debug`'s `supports-color` bare is not a static map but the missing-peer set: [`getMissingPeers`](https://github.com/pnpm/pnpm/blob/01b3d45ddb/installing/deps-resolver/src/resolveDependencies.ts#L1773-L1782) iterates `peerDependencies` only, so a **meta-only** peer never feeds the hoist. Both behaviors verified empirically against pnpm 11.6.0 (`eslint` + `cosmiconfig-typescript-loader` hoists `jiti@2.6.1`; `debug` + `concurrently` stays bare). pacquet's static-snapshot approach got the meta-only case right by the wrong mechanism and under-hoisted every real-entry optional peer — the missing `(jiti@2.6.1)`, `(typanion@3.14.0)`, `(conventional-commits-parser@6.4.0)`, `(@types/node@…)` suffixes and their cascades on the monorepo. The 12267-era regression test asserted the anti-pnpm behavior for the real-entry shape and was inverted; a new test pins the meta-only shape. ## Verification - New unit tests in `pacquet-resolving-deps-resolver` (importer group merge ×4, peer-vs-own-dep shadowing ×3, optional-peer hoist real-entry/meta-only ×2) and `pacquet-package-manager` (peer overrides ×4). - Full workspace suite: 3218 tests pass; clippy `--deny warnings` clean; rustfmt + typos clean. - Whole-monorepo lockfile diff vs fresh pnpm 11.6.0: 806 → 461 changed lines; `@pnpm/logger`, `jiti`, `typanion`, `@babel/types`, `ajv` categories eliminated. Two consecutive pacquet runs byte-identical. |
||
|
Kalven Schraut
|
dfa91df6e8 |
fix: resolve musl pacquet binary on musl-based systems (#12347)
* fix: resolve musl pacquet binary on musl-based systems The pacquet binary packages are split by libc on linux and only the matching one is installed, but resolvePacquetBin always asked for the glibc name. On Alpine and other musl systems the frozen install failed with: Cannot find module '@pacquet/linux-x64/pacquet'. fix pnpm/pnpm#12049 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: sort imports in runPacquet.ts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: verify the musl pacquet binary package on musl-based systems The signature verification hard-coded the glibc platform package name, so on musl systems it verified a package other than the binary that is actually spawned. Share one platform-package-name helper between resolvePacquetBin and collectPacquetPackagesToVerify. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
f648e9b7c4 |
fix: contain hoisted dependency aliases (GHSA-fr4h-3cph-29xv) (#12343)
* fix: contain hoisted dependency aliases (GHSA-fr4h-3cph-29xv) The `nodeLinker: hoisted` install restores its dependency graph straight from the lockfile via `lockfileToHoistedDepGraph`, which joins each dependency alias under a `node_modules` directory and imports the package files there. On a frozen / up-to-date lockfile, resolution is skipped entirely, so the alias validation added for the resolution path never runs. A crafted lockfile alias such as `../../../escape` could therefore escape the install root, and reserved aliases such as `.bin`, `.pnpm`, or `node_modules` could overwrite pnpm-owned layout. Validate every alias at the hoisted-graph directory sink. The shared `safeJoinModulesDir` helper now rejects aliases that are not valid npm package names (path-traversal, absolute, and reserved names) in addition to its containment check, and the hoisted graph routes its `dep.name` sink through it. Pacquet mirrors the boundary: `safe_join_modules_dir` validates the hoister's `dep.0.name` before adding the graph node or recursing, reusing the same dependency-name rule it already applies to direct-dependency aliases. Both stacks surface `ERR_PNPM_INVALID_DEPENDENCY_NAME`. --- Written by an agent (Claude Code, claude-fable-5). * fix: reject invalid dependency aliases at the lockfile verification gate Add an always-on, policy-independent structural check to verifyLockfileResolutions that rejects any importer or package-snapshot dependency alias that is not a valid npm package name. A dependency alias becomes a `node_modules/<alias>` directory at link time, so an alias with path-traversal segments or a reserved name (`.bin`, `.pnpm`, `node_modules`) could escape the install root or overwrite pnpm-owned layout. This complements the linker-sink guards: the verifier runs before any fetch or filesystem work and covers every node linker at once, while the sink guards still protect the `trustLockfile` path the verifier skips. The check runs before the cache lookup so a record written by a version that predates the rule cannot fast-path around it, and before the `packages` guard so a tampered importer alias is caught even when nothing is installed. `isValidDependencyAlias` is now exported from `@pnpm/installing.deps-resolver` and reused here. Pacquet mirrors the gate in its lockfile-verification crate with a matching `ERR_PNPM_INVALID_DEPENDENCY_NAME` verdict. --- Written by an agent (Claude Code, claude-fable-5). * docs(package-manager): drop redundant explicit intra-doc link target `is_valid_dependency_alias` is in scope via `use`, so the bare intra-doc link resolves on its own. The explicit path target tripped `rustdoc::redundant-explicit-links` under the CI Doc job's `cargo doc --document-private-items` (the local pre-push hook runs `cargo doc` without that flag, so it didn't surface). --- Written by an agent (Claude Code, claude-fable-5). * refactor(lockfile-verification): fold the alias check into the single candidate pass The dependency-alias check ran as its own full traversal of the lockfile in addition to collectCandidates' existing pass over every package snapshot. Fold it into that pass instead: collectCandidates now also validates each importer and snapshot dependency alias and returns the invalid ones alongside the resolution-shape violations, so the lockfile is walked once per verification rather than twice. Because collectCandidates runs after the verification-cache lookup, the alias check is now covered by the cache the same way the resolution-shape check is: a new dependencyAliasCheck cache identity makes a record written before this rule existed fail canTrustPastCheck, forcing a re-verification. The shared helper is renamed withOfflineCheckCacheIdentities and appends both offline-structural-check identities. No behavior change for valid lockfiles; the same ERR_PNPM_INVALID_DEPENDENCY_NAME is thrown for invalid aliases. Mirrored in pacquet's lockfile-verification crate. --- Written by an agent (Claude Code, claude-fable-5). * refactor: declare pushInvalidAliases after its caller, trim duplicated comments Move `pushInvalidAliases` below `collectCandidates`, following the repo's declare-after-use convention. Collapse the repeated "an alias becomes a node_modules directory, so a traversal/reserved name escapes or overwrites layout" explanation that was copied across the verifier, the hoisted-graph error, and the pacquet mirror down to a single reference each — the full rationale lives once in the validating sink (`safeJoinModulesDir` / `safe_join_modules_dir`) and the user-facing error hints. --- Written by an agent (Claude Code, claude-fable-5). |
||
|
Zoltan Kochan
|
c16eb0a154 |
perf: run lockfile verification concurrently with frozen install (#12227)
## Problem `pnpm install` with a frozen lockfile got noticeably slower because lockfile verification blocks every later install stage. The verification gate (the `minimumReleaseAge`/`trustPolicy` policy revalidation plus the tarball-URL anti-tamper check) issues a registry round trip per lockfile entry, and the whole install waited for it to finish before any fetching or linking could begin. ## Change (pnpm / TypeScript) Run lockfile verification **concurrently** with fetching and linking instead of blocking on it, while keeping two guarantees intact: 1. **No lifecycle script runs on an unverified lockfile.** A `verifyLockfile` gate is threaded into both `buildModules` call sites — `headlessInstall` (frozen path) and `_installInContext` (full-resolution path) — and awaited immediately before any dependency lifecycle script runs. The projects' own `preinstall`/`postinstall` hooks are held to the same gate at both `runLifecycleHooksConcurrently` call sites, covering the `enableModulesDir: false` path that skips the build phase. If verification failed, the gate throws before a single script executes. 2. **The verdict is always reconciled.** `settleInstall(_install(), verifyLockfilePromise)` awaits the verification verdict first so it takes precedence and fails fast (even mid-install), then surfaces the install's result/error. This also covers paths that skip the build phase entirely (`ignoreScripts`, `lockfileOnly`, empty lockfile). Verification's synchronous prologue (cache lookup, lockfile hash, candidate collection) still runs against the pristine lockfile before `_install()` mutates `ctx.wantedLockfile`, so the concurrent async fan-out reads a stable snapshot — no data race. The verification verdict deliberately takes precedence over a concurrent install error: `pnpm add`'s full-resolution path can throw its own generic "resolution-policy violations produced but no handler wired" for the same underlying violation, and `settleInstall` makes sure the specific `minimumReleaseAge`/`trustPolicy` error is what surfaces. ## Change (pacquet / Rust) Same optimization ported to `pacquet/crates/package-manager/`. `Install::run` builds the resolution verifiers up front but dispatches the verification fan-out per path: - **Frozen materialization path:** verification runs concurrently with `CreateVirtualStore` (the fetch), settled with a `select!` so the verdict takes precedence: a rejected lockfile aborts the fetch in flight (fail-fast), while a fetch failure waits for the verdict and only surfaces once the lockfile is known trusted — an unrelated fetch error can't mask a rejected lockfile. The verdict is always reached before linking and `BuildModules`, so no dependency lifecycle script runs on an unverified lockfile. - **Lockfile-only / up-to-date short-circuits and the fresh-resolve path:** keep an eager blocking gate — they have no fetch to overlap. A verification failure surfaces as the same `InstallError::LockfileVerification` variant regardless of which path produced it. On the pnpr client, a frozen restore now skips resolution entirely: tarball downloads start from the local lockfile at t=0 (filtered through one batched store-index existence probe, so a warm store prefetches nothing) while the server delivers only the trust verdict via the new `POST /v1/verify-lockfile` endpoint, concurrently with the fetch. ## Tests - pnpm: `test/install/trustLockfile.ts` covers the rejection itself, the `trustLockfile` opt-out, and both script gates — a dependency `postinstall` never runs when verification fails, and the projects' own lifecycle hooks never run either, asserted on the `enableModulesDir: false` path with a *slow*-rejecting verifier (an instantly-rejecting one aborts the install before the hooks are attempted, which would hide a missing gate). Existing verification/lifecycle/`minimumReleaseAge` suites pass. - pacquet: existing `frozen_lockfile_gate_rejects_under_huge_minimum_release_age` (unit) and `install_fails_under_huge_minimum_release_age` (CLI) assert the frozen install aborts with no virtual-store materialization on verification failure — proving the fail-fast settle cancels the fetch. New: `without_store_hits` + `StoreIndex::contains_many` unit tests pin the warm-store prefetch filter, and the frozen pnpr CLI test swaps the registry for a zero-expectation server before the restore, proving a warm-store frozen restore makes no registry requests. - pnpr client/server: integration tests cover `/v1/verify-lockfile` accepting a clean lockfile, rejecting a policy violation, honoring `trustLockfile`, and forwarding the client's credential map (each verify call targets a fresh pnpr so no verdict/metadata cache can satisfy it without exercising the credential). - clippy / `cargo doc -D warnings` / rustfmt / eslint clean; package-manager, lockfile-verification, store-dir, pnpr-client, and CLI pnpr-install suites all pass. ## Behavioral nuance On a *rejected* lockfile, fetching/linking may now have partially populated the store/`node_modules` before the abort (previously nothing ran, since verification went first). The command still fails with the same error code and no lifecycle scripts run. |
||
|
Zoltan Kochan
|
66a9078060 |
fix: route pacquet scoped packages through scoped registries (#12340)
* fix: route pacquet scoped packages through scoped registries * fix: satisfy pacquet add test clippy * test: seed scoped registry in modules yaml fixture * test: stabilize latency proxy slow start timing |
||
|
Zoltan Kochan
|
612a2e6a73 | fix: contain patch-remove deletions (#12341) | ||
|
Victor Sumner
|
61810aa684 |
feat: add --frozen-store for installs against a read-only store (#12190)
## What
Adds an opt-in `frozenStore` / `--frozen-store` setting (default `false`) that lets `pnpm install --offline --frozen-lockfile` run against a package store that lives on a **read-only filesystem** — a Nix store, a read-only bind mount, an OCI layer.
## Why
A normal install fails against such a store **not** because it writes package content, but because it unconditionally:
1. opens the SQLite `index.db` in WAL mode, which needs to create `-shm`/`-wal` sidecars in the store directory; and
2. writes a project-registry entry under the store.
Both fail with `attempt to write a readonly database` / `EROFS` on a read-only store directory, even when the store is complete and the lockfile is frozen. This blocks any deployment that wants an immutable, content-addressed store.
## How
When `frozenStore` is enabled, pnpm opens `index.db` through the SQLite **`immutable=1`** URI — which tells SQLite the file cannot change underneath it, so it bypasses the WAL/`-shm` sidecar machinery entirely and reads the raw file with zero sidecar creation — and suppresses every store-write path. Pair it with `--offline --frozen-lockfile` against a fully-populated store. It is incompatible with two settings that would write into the store, and each throws a clear config-conflict error before any network or store access: **`--force`** (which bypasses the no-write-on-hit skip → `ERR_PNPM_CONFIG_CONFLICT_FROZEN_STORE_WITH_FORCE`) and a configured **pnpr server** (which would fetch and write packages into the store → `ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR`).
> Plain `SQLITE_OPEN_READ_ONLY` is **not** sufficient: opening a WAL-mode db read-only still tries to create the `-shm` sidecar, which fails on a read-only *directory*. `immutable=1` is the load-bearing piece.
> **Node.js requirement:** `node:sqlite` only passes `SQLITE_OPEN_URI` to SQLite (so the `immutable=1` query is honored rather than treated as part of a literal filename) starting in **v22.15.0** (22.x line), **v23.11.0**, and every **v24+**. pnpm's `engines` floor is `>=22.13`, so on a runtime older than that the frozen open is detected up front and fails with a clear `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` instead of SQLite's cryptic "unable to open database file". (pacquet uses rusqlite with an explicit `SQLITE_OPEN_URI` flag, so it has no such floor.)
> The `immutable=1` URI path is also percent-encoded (`%`→`%25`, `?`→`%3f`, `#`→`%23`, in that order, leaving `/` literal) so a store path containing those characters doesn't truncate the path or inject a spurious query parameter — applied identically in both stacks.
### Build backstop under the global virtual store
Under the global virtual store (default), a package's directory lives **inside** the store (`{storeDir}/links/...`). Applying a patch or running an allowlisted lifecycle script writes into that directory — so on a frozen store it would crash mid-build with a raw `EROFS`. A fully-seeded store never reaches the build step (patched/built packages are imported from the side-effects cache and filtered out by the `isBuilt` gate), so any residual build candidate means the seed is **missing that package's build output**.
`buildModules` now refuses up front with `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` and actionable guidance ("rebuild the seed with their scripts enabled, or remove them from `onlyBuiltDependencies`") instead of failing cryptically once a script starts. The check is gated on the global virtual store — under the isolated linker, slot directories live in the writable project-local store, so builds there are fine. Non-allowlisted scripts never run, so they are not treated as a blocking write.
Bin-linking has its own read-only-store edge under the global virtual store. On a **warm** checkout (the project's `.bin/<name>` already points at the seed target) `linkBin` returns before touching the store, so it is write-free. But on a **fresh** checkout it (re)creates the bin and calls `fixBin`, whose `chmod` targets the bin's **source file inside the store** (`{storeDir}/links/...`) — which is refused with `EPERM`/`EACCES` on a read-only store, even though a complete seed already ships that bin executable (so the `chmod` is redundant). `@pnpm/bins.linker` now wraps that call in `ensureExecutable`: it swallows the refusal when the target is already executable and rethrows otherwise, so bin-linking is write-free against a frozen store on a cold checkout too, while a genuinely non-executable bin (a broken seed) still surfaces as an error.
The blocking predicate distinguishes the two write kinds: a **patch** is applied regardless of `ignoreScripts`, so a patched package is always blocked; a **lifecycle script** is suppressed under `ignoreScripts`, so an allowlisted build-requiring package is *not* blocked when scripts are off (it would write nothing). This avoids falsely rejecting a valid `--ignore-scripts` frozen install. **Optional dependencies are exempt**: a build or patch failure on an optional dependency is non-fatal at runtime, so a seed missing an optional package's build output skips that build (emitting the `skipped-optional-dependency` log) instead of blocking the install — in both stacks.
### Both stacks (parity rule)
**TypeScript pnpm CLI**
- Config plumbing (`@pnpm/config.reader`): `frozen-store` type, config-file key, default, `Config.frozenStore`.
- The read-only open branch (`@pnpm/store.index`): `immutable=1`, read-only statements, throwing mutators.
- Wiring through `@pnpm/store.controller` and `@pnpm/store.connection-manager` to the sole `StoreIndex` construction site.
- Gating the project-registry write (`@pnpm/installing.context`).
- The `--force` / pnpr-server conflict guards (`@pnpm/installing.deps-installer`) and CLI surface (`@pnpm/installing.commands`).
- **After-install rebuild** (`@pnpm/building.after-install`): the post-install rebuild opens its `StoreIndex` immutably under the flag, so re-reading the store for a rebuild never attempts a writable open against the frozen store.
- **Worker fix:** `@pnpm/worker` opens its *own* writable `StoreIndex` on every `readPkgFromCafs` cache hit, so a pure read crashed on a frozen store. `frozenStore` is threaded through to `getStoreIndex` and keyed into its connection cache.
- **Build backstop** (`@pnpm/building.during-install`): `buildModules` throws `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` for a GVS slot that would build/patch on a frozen store, honoring `ignoreScripts`; threaded from `@pnpm/installing.deps-installer`.
- **Bin-linking on a read-only store** (`@pnpm/bins.linker`): `linkBin` wraps the `fixBin` chmod in `ensureExecutable`, which tolerates `EPERM`/`EACCES` when the bin's store-resident source is already executable (a complete seed) and rethrows otherwise — so a fresh checkout against a frozen store links bins without crashing on the redundant chmod. Catch-on-failure keeps the writable hot path at zero added syscalls.
**Rust pacquet**
- A dedicated `open_immutable` / `shared_immutable_in` opens via `immutable=1`, selected only under the flag. Plain `open_readonly` keeps the ordinary `SQLITE_OPEN_READ_ONLY` open (WAL locking intact) because normal installs read the index while the same process's `StoreIndexWriter` writes it concurrently — an immutable connection skips all locking and change detection, so a concurrent writer would make those reads undefined.
- `--frozen-store` CLI flag + `frozenStore` workspace-yaml setting.
- The store-index writer is replaced with a drain-and-drop stub (`spawn_disabled`) and `init_store_dir_best_effort` is skipped under the flag.
- **Build backstop:** `build_modules` returns `BuildModulesError::FrozenStoreNeedsBuild` (`ERR_PNPM_FROZEN_STORE_NEEDS_BUILD`) under the same GVS + frozen-store condition, threaded from `config.frozen_store`. The gate keys off `should_run_scripts` (which already folds the allow-build policy), so it is correct without an explicit ignore-scripts branch — pacquet has no configurable ignore-scripts mode yet.
pacquet already separated read-only index access (`shared_readonly_in`, or `shared_immutable_in` under the flag) from writes, so it never had the worker-conflation bug; the flag makes the "no writes attempted" contract explicit and gates the remaining best-effort write attempts.
## Testing
- **TS:** `store.index` frozen-mode-on-`0555`-directory test (reads work, writes throw `ERR_PNPM_FROZEN_STORE_WRITE`) plus a path-with-`?` open test — both gated on the runtime's immutable-URI support, with a complementary test asserting `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` fires where that support is absent (the CI Node 22.13.0 path); `config.reader` round-trip; `deps-installer` `--force` and pnpr-server conflict guards; `worker`/`package-requester` unit tests. End-to-end on a `chmod -R 0555` store: install succeeds, `node_modules` materializes, no `-shm`/`-wal`/`-journal` sidecars; negative control without the flag fails as expected; incomplete store → clean offline error.
- **pacquet:** `open_immutable_reads_wal_db_on_readonly_directory` unit test plus the `immutable_sqlite_uri` encoding test and a path-with-`?` open test; yaml + CLI fold tests; **integration test** `frozen_store_installs_against_a_read_only_store` — primes a store, `chmod 0555` the tree, runs `install --frozen-lockfile --frozen-store --offline`, asserts success + materialized `node_modules` + zero sidecars. Confirmed load-bearing by reverting the `immutable=1` fix (test then fails).
- **Build backstop (both stacks):** `building/during-install` unit tests — approved-build-not-cached and patched-not-cached refuse; cached, non-allowlisted, and non-GVS cases pass through; **approved-build-under-`ignoreScripts` passes through while patched-under-`ignoreScripts` still refuses** — and the matching pacquet `build_modules` tests (`frozen_store_gvs_patch_not_seeded_refuses` + GVS-off / frozen-off controls). Each confirmed load-bearing by disabling the relevant guard and watching the corresponding test fail.
- **Bin-linking (`bins/linker`):** `ensureExecutable` tests with `fixBin` mocked to reject with `EPERM` — an already-executable bin source resolves (and `fixBin` is asserted called, so it isn't the warm skip-guard passing), a non-executable one rethrows `EPERM`. Confirmed end-to-end by running the built `linkBins` against a real `chflags uchg`-immutable store: the executable-seed case resolves and links the bin, the non-executable-seed control throws `EPERM`.
A changeset is included with `"pnpm": minor` and `"@pnpm/bins.linker": patch` (the read-only-store bin-linking fix).
|
||
|
Zoltan Kochan
|
a31faa7c19 |
chore: update dependencies (#12346)
* chore: update dependencies
Update all catalog dependencies to their latest versions, except those
held back by pnpm's supported Node.js floor (>=22.13) or known issues;
each held-back entry now carries a comment in pnpm-workspace.yaml
explaining why.
Notable changes:
- msgpackr 1.11.8 -> 2.0.4 (unpinned; types compile again and the
store-index output is byte-compatible with 1.x in both directions)
- typescript 5.9.3 -> 6.0.3, esbuild 0.28, commitlint 21,
concurrently 10, eslint plugin majors (autofixed one import-sort
error they introduced)
- open 11, memoize 11, cli-truncate 6, pidtree 1, @yarnpkg/core 4.8,
@rushstack/worker-pool 0.7.18
- removed unused nock devDependency and the deprecated @types/tar stub
- bole stays on 5: bole 6 is ESM-only and under Jest the workspace
logger's ESM copy and the published @pnpm/logger's CJS bole 5 no
longer share the globalThis.$$bole output registry, breaking
reporter assertions
Held back due to Node >=22.13 support floor: ssri 14,
write-file-atomic 8, validate-npm-package-name 8,
normalize-package-data 9, npm-packlist 11, ini 7 (need ^22.22.2),
undici 8 (needs >=22.19), cspell 10 (needs >=22.18; bumped to 9.8.0
instead).
* chore: add changeset for updated dependency ranges
Patch-bump every published package whose runtime dependency or peer
dependency range changed in the dependency update, following the
precedent of commit
|
||
|
Zoltan Kochan
|
01b3d45ddb | chore: update dependencies (#12339) | ||
|
Zoltan Kochan
|
53b105416f |
chore(release): 11.6.0 (#12336)
* chore(release): 11.6.0 * docs: update CHANGELOG.md --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>v11.6.0 |
||
|
Zoltan Kochan
|
615c6694e1 |
feat: support URL-scoped registry auth via npm_config_// and pnpm_config_// env vars (#12338)
* feat: support URL-scoped registry auth via npm_config_// and pnpm_config_// env vars Adds a file-free way to configure registry authentication, e.g. npm_config_//registry.npmjs.org/:_authToken=<token> pnpm_config_//registry.npmjs.org/:_authToken=<token> These are host-scoped by construction — the registry the value applies to is encoded in the (trusted) variable name — so they cannot be redirected to another host by repository-controlled config. The env value is trusted: it overrides a project/workspace .npmrc but is still overridden by CLI options. pnpm_config_ wins over npm_config_ for the same key. * feat(pacquet): support URL-scoped registry auth via npm_config_// and pnpm_config_// env vars Pacquet parity for the same feature on the JS side: read URL-scoped registry credentials from npm_config_//… and pnpm_config_//… environment variables (e.g. npm_config_//registry.npmjs.org/:_authToken=<token>). These are trusted (sourced from the environment, not the repository) and host-scoped by construction, so they sit at the top of the .npmrc precedence chain — above the project .npmrc. pnpm_config_ wins over npm_config_ for the same key. Adds an EnvVar::vars() enumeration capability (default empty, so existing fakes keep compiling; production providers override it). * fix(pacquet): avoid Unicode ellipsis in a line comment (dylint) * fix: exclude tokenHelper from URL-scoped env auth; add case-insensitive tests Address review feedback on pnpm/pnpm#12338: - A `//host/:tokenHelper` env var would land in authConfig but trip the TOKEN_HELPER_IN_PROJECT_CONFIG guard (which only trusts the user .npmrc), incorrectly failing. tokenHelper names an executable, so it is now excluded from the env-scoped layer entirely. - Add tests for case-insensitive prefix matching and the tokenHelper exclusion. - Add a 'text' language hint to the changeset's fenced block (MD040). * fix(pacquet): avoid panics on non-UTF-8 / non-ASCII env var names Address CodeRabbit review on the pacquet env-auth code: - EnvVar::vars() used std::env::vars(), which panics if any env var name or value is not valid UTF-8. Iterate vars_os() and skip non-UTF-8 entries, matching var()'s .ok() behavior. (SystemEnv and Host.) - parse_url_scoped_env_name sliced with name[..prefix.len()], which panics when the byte index lands inside a multi-byte char. Use boundary-checked name.get(..) instead. - Add a regression test with non-ASCII env var names. * test: cover env-auth precedence and pacquet end-to-end wiring Fill the coverage gaps in the URL-scoped env-auth feature: - JS: assert a CLI-provided //host/:_authToken still beats the same env var (workspace < env < CLI), and that non-token cred fields work while a non-URL-scoped env key is ignored. - pacquet: add end-to-end tests through the full config load — that a npm_config_//… var is honored and outranks a project .npmrc token for the same host, and that the prefix is matched case-insensitively. FakeEnv now enumerates via vars() so the env-scoped reader sees the fixture. |
||
|
Zoltan Kochan
|
84bb4b1a04 |
perf: close the warm-resolve, symlink-churn, and download-concurrency gaps (#12329)
## Motivation The [vlt.sh benchmarks](https://benchmarks.vlt.sh/) (2026-06-11 run, pacquet 0.11.3) show pacquet several times slower than the fastest package managers in the warm-metadata fresh-resolve cells (`cache`: 3.9–8.1x), the cold-cache frozen-install cells (`lockfile`: up to 10x on vue), and `clean`. Profiling the babylon and vue fixtures locally (macOS time profiles of the warm fresh resolve and the install tail) surfaced three independent causes, fixed here. ## Changes ### 1. Deprecation probing without manifest hydration (pacquet) With `minimumReleaseAge` active (the default), every range pick runs `filter_pkg_metadata_by_publish_date`, and any dist-tag pointing outside the maturity cutoff (`next`, `beta`, `canary`, a too-fresh `latest`) repopulates by scanning all candidates and reading each candidate's `deprecated` flag. Each read hydrated the full version manifest — a complete `serde_json` parse including the flattened catch-all map. On babylon's warm fresh resolve this was the single largest CPU consumer (~10 thread-seconds, all on the resolve task's critical path). `PackageVersions::is_deprecated` now answers from the raw fragment (substring pre-check, then a single-field deserialize with the same normalization as `PackageVersion::deprecated`), the tag-repopulation loop parses candidate versions once per filter call (mirroring the `parsedSemverCache` in pnpm's `filterPkgMetadataByPublishDate`), and the deprecated-pick fallback uses the probe instead of hydrating every version. **babylon warm fresh resolve: `resolve_workspace` 7.5s → 2.6s.** ### 2. Relative-symlink up-to-date check (pacquet) `force_symlink_dir` joined an existing link's relative contents onto the link parent and compared the result *verbatim* against the wanted target. Virtual-store links contain `..` segments (`../../<pkg>/node_modules/<name>`), so the joined path never compared equal and every up-to-date symlink was unlinked and recreated. Node's `path.relative` — which upstream `symlink-dir`'s `isExistingSymlinkUpToDate` builds on — resolves its arguments, so pnpm treats those links as current. Both sides now pass through `lexical_normalize`. The babylon install tail was dominated by exactly this unlink+symlink churn. **babylon warm install: 6.8s → 4.7s; warm frozen install: 4.2s → 2.3s.** ### 3. Default network concurrency floor 16 → 64 (pnpm + pacquet) The default was `min(64, max(workers * 3, 16))`. Downloads are I/O-bound, not CPU-bound: on a 4-vCPU CI runner the formula yields 16 concurrent requests, so a low-latency registry drains 600–1300-tarball installs 16 at a time while staying unsaturated — a large share of the cold-cell (`lockfile`/`clean`) gap on the benchmark runners. The default is now `min(96, max(workers * 3, 64))`; the `networkConcurrency` setting still overrides it. Applied to `@pnpm/installing.package-requester`, the lockfile-resolution verifier fan-out that mirrors its floor, and the same two spots in pacquet. Changeset included (minor). **This is a user-visible defaults change on both stacks — flagging it explicitly for review.** ## Local results (M-series macOS, vlt fixtures, isolated store/cache) | cell | before | after | |---|---|---| | vue `cache` | 1159 ms | **479 ms** | | vue `cache+lockfile` | 621 ms | **392 ms** | | vue no-op install | 48 ms | **41 ms** | | babylon `cache` | ~8.8 s | **4.75 s** | | babylon `cache+lockfile` | ~4.2 s | **2.27 s** | vue's warm cells are now ahead of every competitor measured locally; babylon's `cache` cell closed from ~2.5x behind the leader to ~1.35x (the remainder is the per-file store-integrity verify and per-file linking that the pnpm store contract requires). ## Validation - `cargo nextest`: registry, resolving-npm-resolver, resolving-deps-resolver, lockfile-verification, network, fs, tarball, package-manager, cli — 1300+ tests, all green; new unit tests cover the deprecation probe (string/bool/empty/corrupt shapes, nested-key false positives) and cross-parent relative-symlink reuse (fails without the fix). - Lockfile stability: `--lockfile-only` output is byte-identical before/after on vue; on babylon the resolved **package-version sets are identical across 6 runs (3 per binary)**. The babylon lockfile does flap between runs in the peer-suffix shape of `webpack-dev-server@5.2.2` (`(bufferutil@4.1.0)(utf-8-validate@5.0.10)` appearing/disappearing) — this is **pre-existing nondeterminism** reproducible with the unmodified binary against itself, in the optional-peer area; worth a separate issue. - Pre-push checks (fmt, taplo, `cargo doc -D warnings`, dylint) pass; eslint (root config) and `tsgo --build` pass for the two touched TS packages. |
||
|
Zoltan Kochan
|
bc9ed78f48 |
fix: clearer warning when a project .npmrc uses env variables in registry/auth settings (#12333)
* fix: clearer warning when a project .npmrc uses env variables in registry/auth settings
The previous warning only said the setting was ignored. It now explains why
(the project .npmrc is committed to the repository and must not expand secrets
into request destinations or credentials) and how to fix it: move the value to a
trusted source such as the user-level ~/.npmrc or via pnpm config set, with a
link to the docs.
The suggested 'pnpm config set' example is only shown when the key has no
${...} placeholder, so the snippet is always safe to copy-paste (a shell would
otherwise expand a placeholder embedded in the key). The wording does not claim
a specific destination file.
* fix: only suggest a pnpm config set command for shell-safe keys
The key embedded in the warning's suggested 'pnpm config set' command comes
from a repository-controlled .npmrc. The previous guard only suppressed the
example for keys containing a ${...} placeholder, but a shell also expands
$(...), backticks and $VAR inside double quotes — so a crafted key could turn
the suggested copy-paste command into command execution. The example is now
emitted only for keys made up entirely of shell-inert characters.
|
||
|
Zoltan Kochan
|
f11b4fcad7 |
feat(deps-installer): announce reused lockfile-verification verdicts (#12326)
When the lockfile-verification gate short-circuits on a cached verdict, it used to stay completely silent, which made it look like the supply-chain policy gate never ran (pnpm/pnpm#12324). Emit a new `cached` status on the pnpm:lockfile-verification channel carrying the reused record's verifiedAt timestamp, and render it in the default reporter as "Lockfile passes supply-chain policies (verified 2h ago)" (falling back to "previously verified" for records that predate the timestamp). The event fires only when policy verifiers are active, so the shape-only check every install performs stays quiet. Ported to pacquet in the same change: a `Cached` variant on the reporter's LockfileVerificationMessage with the matching camelCase wire shape, emitted from the same cache-hit point in verify_lockfile_resolutions. |
||
|
Zoltan Kochan
|
cb18695c5b |
perf(pacquet): lazy packument hydration, sharded meta cache, and an indexed metadata mirror (#12322)
## Why Profiling a warm babylon resolve (metadata mirrors hot, ~520 MB across 1,476 packuments) showed the dominant cost was not I/O but **hydration**: every version of every packument was deserialized into typed manifests — maps, strings, and `serde_json::Value` trees built, hashed, and dropped — even though a pick consults only the version *strings* plus the handful of manifests it actually considers. typescript ships 3,800 versions; a pick needs one. The `#[serde(flatten)]` catch-all on `PackageVersion` compounded it by routing the whole struct through serde's buffering deserializer. The same hydration cost was paid on cold resolves inside the `spawn_blocking` parses from pnpm/pnpm#12318. ## What Three changes, applied in profile-driven order: **1. Lazy packument hydration** (`df6f70eb57`). `Package::versions` becomes a `PackageVersions` map whose entries hold the raw JSON fragment serde captured (`Arc<RawValue>`, shared not copied) and hydrate into `Arc<PackageVersion>` on first access, cached per slot. Key scans never hydrate; `pinned_version` hydrates only the winner; the publish-date filter moves slots without touching manifests; undecodable fragments degrade to "version absent" (matching the JS implementation's tolerance); serialization re-emits raw fragments verbatim. Picked manifests travel as `Arc<PackageVersion>`. Enables serde_json's `raw_value` feature (already a workspace dep). **2. Sharded in-memory packument cache** (`4c28c6679e`). Every resolve edge consults the shared meta cache, and its single `Mutex<HashMap>` was the top mutex-wait site in the profile; it is now the same `DashMap` shape the crate's other shared maps use. Honest note: wall time was unchanged by this alone — the post-hydration profile shows the warm resolve is critical-path-bound, not lock-bound — but the contention disappears and the cache no longer serializes workers under load. **3. Indexed on-disk metadata mirror** (`126a416ae8`, maintainer-approved cache-format break). The two-line NDJSON mirror (headers + verbatim body) becomes: ``` pacquet-meta-v1 <headers_len> <index_len>\n <headers JSON> etag, modified <index JSON> name, dist-tags, time, homepage, versions: [version, offset, len] <fragments> concatenated raw per-version JSON ``` The loader reads the file once and hands `PackageVersions` byte spans into that buffer — no structural re-scan, hydration parses a slice in place. The writer streams the registry's own bytes (fragments borrow from the lazily-parsed response body), so the **cold-install cost stays one temp-file + rename per package**. Old-format files read as cache misses and are rewritten on the next 200. A span-per-fragment `pread` variant was tried first and measured *worse* (sys 0.4 s → 4.5 s; the pick paths probe many candidate fragments per package), hence the single-buffer design. ## Measurements (warm babylon `--lockfile-only` resolve, 10-core M-series) | build | wall | notes | |---|---|---| | before this PR | ~9.1 s | malloc/free + `deserialize_any` dominate the profile | | + lazy hydration | ~7.7–8 s | parse microbench 3–3.8× (`typescript` 36.1→9.5 ms) | | + indexed mirror | **~5.5–6.7 s** | sys 0.4 s; no whole-body scan | Cold resolves keep the same hydration savings inside the parse tasks; cold write volume and syscall pattern are unchanged. ## Evaluated and deliberately not included Consolidating the install-phase thread pools (tokio + global rayon + the dedicated CAS-write pool + capped blocking threads show ~100k involuntary context switches on cold installs vs ~750 for the pnpm CLI). After the resolution fixes, repeated container A/Bs show no measurable wall-clock cost from the churn — it hides entirely behind network time — and history warns that speculative concurrency reshuffles here regress badly (see the #11903 prefetch revert). Deferred until a benchmark shows it on the critical path. ## Tests New `package_versions` unit tests (hydrate-on-demand + Arc-identity caching, undecodable-fragment-as-absent, verbatim raw round-trip incl. unknown keys, eager construction, hydration-free filtering) and rewritten `mirror` tests (headers/index/fragment round-trip, span hydration, truncation → cache miss, old-format → cache miss, atomic overwrite). Full suites green: `pacquet-registry` (23), `pacquet-resolving-npm-resolver` (235), `pacquet-package-manager` + `pacquet-cli` (768); workspace clippy `-D warnings` (pedantic set), dylint, fmt. |
||
|
Zoltan Kochan
|
657d322b15 |
perf(network): schedule tarball downloads by estimated pipeline work (#12309)
## Summary When the download connection pool saturates, freed slots are granted by a two-class scheduling policy instead of FIFO: - **Latency class** (packument/metadata fetches, which gate resolution progress): served FIFO. - **Throughput class** (tarball downloads): ranked by **estimated total pipeline work** — `unpackedSize + 3000 × fileCount` — so the most expensive download+extract jobs start first (longest-processing-time-first; a large archive that starts last runs alone at single-connection throughput while every other slot idles, see [pnpm/pnpm#12230](https://github.com/pnpm/pnpm/issues/12230)). The per-file term prices the fixed CAS-write/hash overhead, so a many-small-files package ranks as the long job it actually is. - **Neither class can starve the other**: downloads are guaranteed a reserved half of the pool (strict metadata-first was measured to serialize cold installs — no tarball got a slot until resolution drained, costing the whole resolve/fetch overlap), and metadata wins beyond that reserve (a download backlog can't stall resolution). Both directions are work-conserving. ### How the size hints travel - Local fresh installs read `dist.unpackedSize` / `dist.fileCount` off the resolver-fetched manifest (also fixes exact decompression-buffer preallocation on the prefetch path, previously hardcoded `None`). - The pnpr `/v1/resolve` `package` frame carries both as optional `unpackedSize` / `fileCount` fields (omitted when the registry never published them; old clients and servers interoperate unchanged). - pnpr frozen restores: the lockfile records no sizes, but the verification fan-out fetches each entry's metadata anyway — the npm verifier records both stats into an optional `ObservedDistStats` sink as a side product of the tarball-URL binding check, and the frozen fast path announces every verified tarball as a sized `package` frame before `done` (URLs derived by the same `tarball_url_and_integrity` the client materialization uses). Verdict-cache hits fetch no metadata and keep the bare `done` frame. - pnpr's abbreviated metadata now **preserves** `unpackedSize`/`fileCount` instead of stripping them, since pacquet reads both. - Resolve-time tarball fetches (tarball deps' manifests come from their archives) acquire in the latency class — they gate the resolver's walk. ### Benchmark tooling - The integrated benchmark's latency proxy gained `--registry-slow-start`: per-connection TCP slow start (RFC 6928 initial window, doubling per delivered window toward the bandwidth cap), so scheduling effects that depend on per-connection ramp-up are measurable. - Fixed a macOS bug where the proxy's accepted sockets inherited the listener's `O_NONBLOCK` and every proxied connection died on its first read — all shaped benchmark traffic silently failed before this. ## Measurements Fixture: ~110 direct deps / 1308 packages (~90 MB wire), `isolated-linker.fresh-install.cold-cache.cold-store`, local mirror of real npm behind the shaped proxy (30 ms RTT, 80 Mbit/s per-connection cap, TCP slow start). **Drift-controlled interleaved comparison** (4 alternating blocks x 4 runs each; sequential multi-target sessions on this machine showed up to +75% session-order drift, so block-paired ABAB is the only design we trust): | target | mean +/- sd (n=16) | | --- | --- | | baseline FIFO | 14.36 s +/- 0.54 | | this PR | **14.06 s +/- 0.70** | The PR wins **all 4 paired blocks** (-0.18 s to -0.50 s, mean -0.30 s, ~2%). A scheduler ablation (reserve+FIFO, smallest-first, unpackedSize-only, work with K=3k and K=10k per file) ordered as the pipeline model predicts, but the per-variant deltas sit inside the session-drift noise, so only the FIFO-vs-full-design pairing is claimed. K in [3000, 10000] is indistinguishable. **The starvation fix is the load-bearing piece, established mechanistically rather than by wall clock:** with strict metadata-first priority (an intermediate design), cold-install event timelines showed 4-7 s windows at install start with zero tarball activity - downloads never won a slot during the resolution burst, serializing the resolve and fetch phases. The reserved share removes those gaps entirely and the worst observed cold-install runs with it are within ~1 s of the median, where unreserved variants showed multi-second stragglers. Real-registry A/B (15 randomized cold-install pairs against npmjs) is noise-bound on a saturated ~100 Mbit link (+/-3 s registry variance), median -0.17 s in this PR's favor - consistent with "never slower." |
||
|
Khải
|
ac367fce91 |
chore(rust/clippy): pedantic, nursery, and some (#12209)
* chore: enable clippy::pedantic lint group for pacquet workspace * style(pacquet): comply with clippy::pedantic Apply clippy's machine-applicable pedantic fixes across the workspace (inlined format args, removed needless borrows/closures, added must_use, etc.), fix a few doc-comment backtick nits, and drop pointless #[inline(always)] on trivial accessors. Opt specific pedantic lints back out in [workspace.lints.clippy] with documented justifications, grouped into false positives, library-API hygiene that doesn't fit an internal CLI, suggestions that conflict with the cardinal rule of porting pnpm 1:1, and opinionated style. * style: taplo-format Cargo.toml lint table * style(pnpr): comply with clippy::pedantic in merged auth backend code Re-apply pedantic compliance to the networked-SQLite auth backend that landed on main (#12186, #12199/#12206): doc-comment backticks, #[must_use] on constructors and status_code, i64::from over `as`, map_or, and a method-reference closure. * docs(clippy): trim and inline the pedantic allow-list comments * docs(clippy): note perfectionist supersedes many_single_char_names * docs(clippy): note pnpm-mirroring rationale on structure/naming lints * docs(clippy): mark unused_async as deferred pending audit * style: enable clippy::match_wildcard_for_single_variants * refactor: enable clippy::unused_self Convert two self-less private methods (overrides pick_most_specific, tarball head_only_result) to associated functions. * refactor: enable clippy::ref_option Widen engine_json to Option<&str>; #[expect] the two serde serialize_with helpers, which serde must call as f(&field, ser). * perf: enable clippy::trivially_copy_pass_by_ref Pass the 1-byte Copy types NodeLinker and FilterWorkspaceProjectsOptions by value; #[expect] the serde skip_serializing_if helper is_false. * perf: enable clippy::assigning_clones Use clone_from for seven field assignments to reuse allocations. * style: enable clippy::manual_let_else Convert 27 match/if-let guards to let-else; preserve the non-UTF-8 skip rationale comment in the directory walker. * style: enable clippy::default_trait_access Name the concrete type on Default::default() call sites; #[expect] two struct-literal test fixtures where naming each field type would force ~20 imports. * refactor: enable clippy::format_push_string Replace push_str(&format!(...)) with write!/writeln! into the target String (local 'use std::fmt::Write as _'); writeln! preserves the exact LF/CRLF shell-shim output. * refactor: enable clippy::needless_pass_by_value Take by reference where the argument is only read (incl. dropping some redundant clones in resolve_peers' recursion). Where converting would cascade badly, #[expect] with a reason: functions that destructure/consume the arg (build_resolve_result, PrefetchingResolver, S3Store::new), the by-value `impl IntoIterator + Clone` in build_direct_deps_by_importer, and the serde/test helpers whose owned fixtures keep call sites clean. * fix(perfectionist): satisfy dylint after format_push_string changes Add trailing commas to the multi-line writeln! shell-shim templates (macro_trailing_comma) and merge the new `fmt::Write as _` imports into each file's existing `use std::{...}` block (import_granularity). * docs(clippy): explain missing_errors_doc suppression; mark missing_panics_doc deferred * fix(perfectionist): collapse fmt::{self, Write as _} in work_env imports The format_push_string Write import landed as a sibling fmt:: path next to the existing fmt import; merge them so import_granularity passes. * style: enable clippy::return_self_not_must_use Add #[must_use] to the WorkspaceTreeCtx builder methods, matching the #[must_use] already on the parallel TreeCtx builders. * perf: enable clippy::large_stack_arrays Heap-allocate the 64 KiB read buffer in verify_file_integrity with a Vec instead of placing it on the stack. * chore(clippy): enable clippy::nursery group Enable the nursery lint group on the pacquet/pnpr workspace and bring the code into compliance. Fixed in code: - iter_on_single_items: [x].into_iter()/.iter() -> std::iter::once - equatable_if_let: pattern match -> equality check (the install_accelerator rewrite wraps in a multi-line matches!, which gets a trailing comma for perfectionist::macro_trailing_comma) - needless_pass_by_ref_mut: load_pending_row/apply_write_msg take &StoreIndex Opted back out in Cargo.toml, each with a documented justification: use_self, too_long_first_doc_paragraph, missing_const_for_fn, option_if_let_else, significant_drop_tightening, redundant_pub_crate, derive_partial_eq_without_eq, branches_sharing_code, useless_let_if_seq, single_option_map, iter_with_drain, literal_string_with_formatting_args, collection_is_never_read. Dropped the now-redundant individual nursery warns (needless_collect, or_fun_call, redundant_clone) the group now covers, plus the default-on unnecessary_lazy_evaluations. Kept clone_on_ref_ptr and if_then_some_else_none (restriction lints not enabled by any group). * style: bring merged main code into clippy pedantic compliance The 17 commits merged from main predate this branch's pedantic/nursery lint config, so their new code tripped pedantic lints. Apply the machine-applicable fixes (uninlined_format_args, if_not_else, elidable_lifetime_names, must_use_candidate, single_match_else, map_unwrap_or, default_trait_access, assigning_clones, doc_markdown, ...) and re-add the documented #[expect(needless_pass_by_value)] on S3Store::new that this branch had carried on the now-replaced file. * style: bring merged main code into clippy pedantic compliance The 28 commits merged from main predate this branch's lint config, so their new code tripped pedantic lints. Apply the machine-applicable fixes (uninlined_format_args, manual_let_else, needless_raw_string_hashes, redundant_closure_for_method_calls, map_unwrap_or, elidable_lifetime_names, doc_markdown, ...) plus a few by hand: - derive Copy on LinkSlotsParallel (all fields are Copy/refs) to clear needless_pass_by_value without a signature change - deduplicate_all takes &[Vec<DepPath>] (it only borrows the duplicates) - pick_most_specific becomes an associated fn (it never used self) - default_trait_access -> concrete types; assigning_clones -> clone_from; format_push_string -> write! - #[expect] with reasons where a fix would churn main's feature code: needless_pass_by_value on the recursive resolve_node and a test helper, and float_cmp on two deterministic-fixture assertions * style: enable clippy::allow_attributes and allow_attributes_without_reason Both are restriction lints (not implied by any group), enabled alongside the existing clone_on_ref_ptr / if_then_some_else_none. Convert every #[allow(...)] (including one nested in cfg_attr) to #[expect(...)]; all already carried a reason, so allow_attributes_without_reason is satisfied. Drop two now-redundant suppressions surfaced by the conversion: a duplicated #[expect(too_many_arguments)] on fetch_and_extract_zip_once (a prior merge left both an allow and an expect), and the #[expect(dead_code)] on MissingPeerInfo's fields (the #[derive(Debug, Clone)] already reads them, so dead_code never fired). clone_on_ref_ptr was already enabled. mod_module_files is intentionally NOT enabled: it mandates mod.rs, the opposite of the flat module.rs pattern this project requires (CODE_STYLE_GUIDE.md, enforced by perfectionist::flat_module_pattern). * style: enable clippy::mod_module_files to enforce the flat module layout mod_module_files bans mod.rs files, enforcing the flat module.rs pattern this project already uses (0 mod.rs in the tree, so no violations). Update CODE_STYLE_GUIDE.md to cite it as the enforcer; perfectionist's flat_module_pattern is being retired in favor of this Clippy rule. * fix(perfectionist): trailing comma on wrapped assert_eq! in workspace_yaml tests The default_trait_access fix lengthened the assert_eq! so fmt wrapped it to multi-line, which perfectionist::macro_trailing_comma requires to end with a trailing comma. * fix(fs): use cfg_attr expect instead of allow for Windows-unused mode args With clippy::allow_attributes enabled, the #[cfg_attr(windows, allow(unused))] on make_file_executable and the ensure_file/write_atomic mode params fails Windows CI. Switch to #[cfg_attr(windows, expect(unused, reason = ...))]; on Windows the lint fires (Unix mode unused there) so the expectation is fulfilled, and the attribute stays inert on Unix. * fix(fs): drop the Windows unused suppression on ensure_file's mode arg ensure_file forwards mode to verify_or_rewrite unconditionally, so it is used on Windows too; the #[cfg_attr(windows, expect(unused))] was therefore unfulfilled and failed Windows CI under -D warnings. write_atomic and make_file_executable keep their expect — they use mode/file only under #[cfg(unix)], so the lint fires (and the expectation holds) on Windows. * chore(git): revert "fix(fs): drop the Windows unused suppression on ensure_file's mode arg" This reverts commit |
||
|
Zoltan Kochan
|
9cd8070722 |
perf(resolving-npm-resolver): parse packuments off the reactor and release the network permit first (#12318)
A cold resolve parsed every packument body inline on the tokio worker that buffered it — while still holding its network-concurrency permit — and then wrote the multi-megabyte metadata mirror to disk synchronously on the same worker. High-release-cadence packages (`@fluentui/*`, `@types/node`) ship packuments of several megabytes, so a burst of them pinned every reactor worker in serde for hundreds of milliseconds at a time, stalling the sockets those workers pump and parking concurrency permits on CPU work. On a cold babylon resolve (1,476 packuments, ~520 MB) the metadata phase ran at a third of pnpm's effective throughput and the tarball pipeline sat idle behind it (a 15-second dead zone in the download timeline while the `@fluentui` subtree parsed). Both metadata fetchers now drop the request guard as soon as the body is buffered — the semaphore goes back to bounding sockets, matching the buffer-then-release shape the tarball pipeline already uses — and deserialize + persist the mirror inside one `spawn_blocking`, keeping the reactor free to pump the remaining transfers. A 304 releases the guard before the mirror disk read for the same reason. Cold babylon full installs in a Linux container improved from a ~38 s median to ~32.7 s (pnpm: ~27 s), winning 5 of 6 interleaved A/B rounds; cold `--lockfile-only` resolves dropped from 44 s to 24 s in the back-to-back pair. |
||
|
Zoltan Kochan
|
52be454d57 |
fix: infer missing platform fields of optional dependencies from the package name (#12312)
* fix: infer missing platform fields of optional deps from the package name Some registries strip the os/cpu/libc fields (or just libc) from the version objects of the packuments they serve. Resolution then saw every platform-specific optional dependency as platform-unrestricted, so pnpm downloaded and installed the binaries of every platform regardless of supportedArchitectures, and wrote lockfile entries without the platform fields, which broke installs from that lockfile on every machine. Platform-specific binary packages encode their platform in the package name (e.g. @nx/nx-win32-arm64-msvc), so packageIsInstallable now fills the missing platform fields of an optional dependency from the name's tokens. Since every install path decides installability through that check before fetching, foreign-platform binaries are skipped without even downloading them, in fresh resolution and in headless installs with both node linkers alike. A package that declares no platform fields at all is treated as platform-specific only when an operating system is recognized in its name, so a generic name segment (such as 'arm' on its own) never gets a package skipped. Fixes https://github.com/pnpm/pnpm/issues/11702 Fixes https://github.com/pnpm/pnpm/issues/9940 * chore: add platform name tokens to the cspell dictionary * fix(package-is-installable): infer missing platform fields of optional deps from the package name Port of pnpm commit https://github.com/pnpm/pnpm/commit/34875b2d7c (PR https://github.com/pnpm/pnpm/pull/12312). Some registries strip the os/cpu/libc fields (or just libc) from the version objects of the packuments they serve, and lockfile entries written from such metadata lack the fields too, so every platform's binaries were installed regardless of supportedArchitectures. Platform-specific binary packages encode their platform in the package name (e.g. @nx/nx-win32-arm64-msvc), so the installability check now fills the missing platform fields of an optional dependency from the name's tokens: infer_platform_from_package_name + inferred_platform in pacquet-package-is-installable, applied inside package_is_installable (hoisted linker) and in compute_skipped_snapshots (isolated linker, with the check cache keyed by the snapshot's optional flag since the verdicts can differ). The any_installability_constraint fast path now also considers optional snapshots whose names infer a platform their metadata row does not declare, so the inference is reachable on lockfiles without any declared constraint. Same guard rails as upstream: declared fields always win (each field is filled only when missing — a missing libc alone is inferred, disambiguating -gnu vs -musl), and a package declaring no platform fields at all engages the inference only when an operating-system token is recognized in its name, so a generic name segment such as 'arm' on its own never gets a package skipped. Fixes https://github.com/pnpm/pnpm/issues/11702 Fixes https://github.com/pnpm/pnpm/issues/9940 * test: shut the metadata-stripping proxy down cleanly and forward the request method |
||
|
Zoltan Kochan
|
d976edf4ec |
perf: content-check modified manifests and fall back to the current lockfile on the repeat-install fast path (pacquet + pnpm) (#12315)
## Why On [benchmarks.vlt.sh](https://benchmarks.vlt.sh/) (2026-06-10 run, pacquet 0.11.2), pacquet ranked **8th–9th of 10** in every `lockfile+node_modules` variation — slower than pnpm, npm, yarn and vlt — e.g. astro: pacquet 936 ms vs pnpm 502 ms; babylon: pacquet 9.08 s vs pnpm 0.85 s. It also trailed vlt/npm in the `node_modules` and `cache+node_modules` variations (astro 1.5 s / 0.7 s, babylon 8.9 s / 6.4 s). ### Root cause Tracing the actual runner (a `pacquet` PATH shim logging per-invocation file stats) showed the harness's prepare step rewrites `package.json` with **identical content but a fresh mtime** before every timed run, while `clean_all_cache` wipes `~/.cache/pnpm` (the packument cache and `lockfile-verified.jsonl`), and the `node_modules` variations additionally delete `pnpm-lock.yaml`. - **pnpm**: `checkDepsStatus`'s modified-manifests branch re-checks the *content* against the lockfile (`assertWantedLockfileUpToDate`, `assertLockfilesEqual`, `linkedPackagesAreUpToDate`) and reports "Already up to date" with zero network — ~0.5 s is just Node startup. Verified locally: with all caches wiped and the network blocked, `pnpm install` still prints "Already up to date" in 228 ms. - **pacquet**: the optimistic repeat-install check bailed on *any* newer manifest mtime, fell into the full pipeline, and the awaited `minimumReleaseAge` lockfile-verification gate — its verdict cache wiped — re-fetched **one packument per locked package** per run: 0.94 s on astro, 9.1 s on babylon. - With `pnpm-lock.yaml` deleted, both stacks pay a similar fan-out on the synthesized-from-current lockfile (`tryLockfileVerificationCache` bails before the content-hash index when the lockfile file can't be stat'd), which is why even pnpm needs 2.2–11.6 s there. ## What **Commit 1 — port the modified-manifests branch of `checkDepsStatus`** (at pnpm/pnpm@cc4ff817aa) into `optimistic_repeat_install`: - a manifest whose mtime is newer than `lastValidatedTimestamp` is re-checked against the wanted lockfile instead of invalidating the fast path: lockfile-settings drift (`getOutdatedLockfileSetting`), per-importer `satisfiesPackageManifest`, and a port of `linkedPackagesAreUpToDate` for workspace links (`isLocalFileDepUpdated` for `file:` directory specifiers is not ported — those conservatively fall through to the full install); - `assertLockfilesEqual` runs when the wanted lockfile is newer than the reference (workspace: `lastValidatedTimestamp`; single-project: the current lockfile's mtime, mirroring upstream's branch shapes); - the workspace branch refreshes `lastValidatedTimestamp` after a passing content check, like upstream's `updateWorkspaceState` call; - the frozen-dispatch freshness gate is split into reusable pieces (`parse_config_overrides`, `check_lockfile_settings_drift`, `check_importer_satisfies`) shared with the new check, and the per-importer slice is no longer hard-wired to the root importer. **Commit 2 — treat the current lockfile as the wanted one when `pnpm-lock.yaml` is missing (pacquet)** (requested by @zkochan): when `node_modules` is intact, `<virtual_store_dir>/lock.yaml` — the record of what the previous install materialized — stands in as the wanted lockfile for the same content checks, and `pnpm-lock.yaml` is regenerated from it (byte-identical to what the full install's synthesize-from-current path would write) before the fast path reports "Already up to date". Single-project installs with no lockfile on either side still refuse the fast path; `lockfile: false` skips the regeneration; a manifest that no longer matches (e.g. `pacquet add`) still takes the full resolve. ## Validation Re-ran the actual vlt.sh harness (same scripts, ubuntu-24.04-arm runner) with the patched binary swapped into the npm-installed pacquet; all hyperfine runs exited 0: | fixture, variation | pacquet 0.11.2 (official run) | patched | pnpm (same validation run) | |---|---|---|---| | astro, `lockfile+node_modules` | 935.6 ms (rank 9/10) | **38–39 ms** | 599–621 ms | | babylon, `lockfile+node_modules` | 9 084 ms (rank 8/10) | **86.6 ms ± 0.6** | 767.7 ms | | astro, `node_modules` | 1 501 ms (rank 4/10) | **41.2 ms ± 0.8** | 2 226 ms | | astro, `cache+node_modules` | 704 ms (rank 5/10) | **42.9 ms ± 0.9** | 2 017 ms | | babylon, `node_modules` | 8 962 ms (rank 6/10) | **107.8 ms ± 1.0** | 11 566 ms | After this change only aube (~5 ms) and bun (~8 ms) stay ahead in these five variations. `cargo nextest run -p pacquet-package-manager` (438 tests), `-p pacquet-cli` install suites, workspace clippy `-D warnings`, dylint, fmt, taplo and `typos pacquet` are clean. New tests cover the touched-but-identical manifest, a manifest that adds a dependency, a diverged wanted-vs-current lockfile, the state-timestamp refresh, linked siblings inside/outside the manifest range, lockfile regeneration (modified and unmodified manifests, workspace state bump), and `lockfile: false`. Two offline e2e tests additionally pin the "zero network, zero pipeline" property through `Install::run`'s real dispatch: a real install, registry dropped, caches wiped, repeat install pointed at a dead port — both verified discriminating by temporarily disabling the content check. Two existing tests were adjusted: `fresh_install_records_lockfile_verification_for_mtime_bypassed_noop` now disables the optimistic check explicitly so it keeps guarding the verification-cache wiring it was written for, and `optimistic_repeat_install_does_not_short_circuit_when_lockfile_missing` now passes `lockfile: None` (matching the CLI contract for a missing file) and documents that the guard requires *both* lockfiles to be absent. **Commit `1ee88c5107` — the same fallback in the pnpm CLI** (`@pnpm/deps.status` + `@pnpm/installing.commands`): `checkDepsStatus` lets the current lockfile stand in when `pnpm-lock.yaml` is missing (workspace shared-lockfile branch and single-project branch), runs the same content checks against it, and returns it as `wantedLockfileToRestore`; `installDeps` writes `pnpm-lock.yaml` back from it before reporting "Already up to date". Guard rails: no lockfile on either side still refuses the fast path, `useLockfile: false` skips the restore, a failed restore falls through to the full install, and the stand-in is disabled under `useGitBranchLockfile` (there a missing plain `pnpm-lock.yaml` is the steady state and the branch lockfile may legitimately differ from the current one). Verified with the bundled CLI: install → delete `pnpm-lock.yaml` → `pnpm install --registry=http://127.0.0.1:9/` prints "Already up to date" in 29 ms and restores the lockfile byte-identically. Covered by 5 new `checkDepsStatus` unit tests and an `installing/commands` integration test that runs the repeat install against a dead registry. Changeset bumps `@pnpm/deps.status`, `@pnpm/installing.commands`, and `pnpm` (minor). |
||
|
Zoltan Kochan
|
5aed1200ea |
feat: add musl binaries for pacquet and pnpr (#12316)
Summary: - add Linux musl binary package selection to the pacquet and pnpr npm shims - generate linux-x64-musl and linux-arm64-musl native npm packages with libc metadata - build musl Rust release targets for both pacquet and pnpr - update package docs and cspell entries for the touched workflow files |
||
|
Zoltan Kochan
|
d2125b86ac | chore: update lockfile, Node.js, pnpm, and pacquet versions (#12261) | ||
|
Zoltan Kochan
|
b7195db5c8 | chore(release): 11.5.3 (#12305) v11.5.3 | ||
|
Zoltan Kochan
|
65443f4bdf |
fix: validate staged tarball filenames (#12303)
- validate package names and versions from staged tarball manifests before deriving tarball filenames - constrain `pnpm stage download` output paths to the selected download directory - add regression coverage for traversal-bearing manifest metadata Pacquet is not changed because it does not implement the stage/release command surface. |
||
|
Zoltan Kochan
|
bf1b731ee6 |
fix: harden allowBuilds artifact approvals (#12294)
## Summary Package-name `allowBuilds` entries no longer approve lifecycle scripts for artifacts whose identity a name cannot pin: git, git-hosted tarball, direct tarball, and local directory dependencies. To approve such an artifact explicitly, use its peer-suffix-free lockfile depPath as the `allowBuilds` key — error hints, `pnpm ignored-builds`, and `pnpm approve-builds` print exactly that key. - `AllowBuild` policy functions identify packages by `DepPath` instead of caller-supplied name/version. The policy parses name and version out of the depPath itself, so name-keyed rules can never be fed an identity that disagrees with the gated artifact. `AllowBuildContext` carries only an explicit `trustPackageIdentity` override, used to evaluate a previously recorded policy under its original semantics when detecting revoked approvals. - Identity trust is derived from the depPath shape: a registry-style depPath (`name@semver`) is a trusted identity. This is sound because lockfile verification runs a new unconditional, offline structural pass that rejects lockfiles where such a key is backed by a git, directory, or git-hosted tarball resolution (`ERR_PNPM_RESOLUTION_SHAPE_MISMATCH`), while the npm resolution verifier already binds explicit tarball URLs of semver-keyed entries to the registry's own `dist.tarball` unconditionally. The pass runs inside the existing candidate walk and participates in the verification cache key (`resolutionShapeCheck`) on both the gate's and the fresh-resolve record paths, so the stat-only cache fast path stays sound and records written before the rule existed are re-verified. - Installs detect approvals that were revoked (or stopped applying) for git/tarball artifacts and surface those packages as ignored builds; approvals granted for previously ignored builds trigger a rebuild of exactly those packages. - `preparePackage` always treats the fetched manifest as an untrusted identity: it requires a `pkgResolutionId` and gates on the synthesized `name@<resolution id>` depPath. scp-style git URLs are normalized to `ssh://` form in resolution ids, and the git fetcher reuses `createGitHostedPkgId` from the resolver instead of re-deriving ids. - Under the global virtual store, `pnpm rebuild` locates a projection created before the approval was granted by following the project's node_modules link, since the projection hash includes the allowBuilds verdict (relocating the projection instead is tracked in https://github.com/pnpm/pnpm/issues/12302). - New shared helpers: `removePeersSuffix()` in `@pnpm/deps.path` (string-level peer-suffix stripping for user-written keys) and `allowBuildKeyFromIgnoredBuild()` in `@pnpm/building.policy` (the key under which an ignored build is approved). - pacquet mirrors the whole policy: `AllowBuildPolicy::check(dep_path)` derives trust from the dep path, the git-fetcher allow-build closures take only the dep path, `pacquet-lockfile-verification` gains the same structural pass, error code, and cache identity, and dep-path keys normalize via `remove_suffix`. - `shell-quote` is overridden to 1.8.4 (GHSA-w7jw-789q-3m8p / CVE-2026-9277). - Test-harness fix: a module-level drain keeps the global log stream flowing in the deps-installer lifecycle tests, so reporter assertions no longer receive the buffered backlog of earlier installs that ran without a reporter. |
||
|
Scarab Systems
|
2c0b91d6b9 | fix(exec): skip deps auto-install without manifest (#12301) | ||
|
Dasa Paddock
|
46fd26afc9 | feat(view): show deprecation warning and bin info (#12271) | ||
|
Zoltan Kochan
|
3d50680eda |
fix(security): verify Node.js runtime SHASUMS OpenPGP signature (#12295)
Follow-up to #12292 (which verifies the **package-manager** binary). This closes the same class of gap for the **Node.js runtime**. When a repository requests a Node.js runtime — `devEngines.runtime: node@X` (with `onFail: download`, the default) or `useNodeVersion` — pnpm downloads and then executes a Node binary (it's used to run lifecycle / `run` / `exec` scripts). The download **mirror is repository-configurable** via `node-mirror:<channel>` (`nodeDownloadMirrors`) in project `.npmrc`, and the integrity comes from `SHASUMS256.txt` fetched **from that same mirror**. That's a circular check: a malicious mirror serves a tampered `node` tarball **and** a matching `SHASUMS256.txt`, the sha256 check passes, and pnpm runs the binary. Drive-by on a normal command in a cloned repo. ## Fix pnpm now fetches `SHASUMS256.txt.sig` and verifies its **detached OpenPGP signature** against the **Node.js release team's public keys, embedded in the pnpm CLI**, before trusting the hashes. A mirror that serves a tampered binary cannot also produce a valid signature, so verification fails. Any faithful mirror (one that proxies the real signed SHASUMS) keeps working. - `@pnpm/crypto.shasums-file`: new `fetchVerifiedNodeShasums` / `fetchVerifiedNodeShasumsFile` verify the signature via `openpgp` against the embedded keys. - The keys live in a generated file (`src/nodeReleaseKeys.ts`, 28 keys) mirrored from the canonical `nodejs/release-keys` list. `crypto/shasums-file/scripts/update-node-release-keys.mjs` keeps them current (`pnpm check:node-release-keys` / `--update`), and the **create-release-pr** workflow runs the check as a gate so a new release signer can't silently break verification. - `@pnpm/engine.runtime.node-resolver` verifies the **configurable-mirror** SHASUMS. The hardcoded `unofficial-builds.nodejs.org` musl mirror is **not** repo-configurable and is signed by a different key, so it stays trusted over TLS. ## Scope - **Pre-release channels (rc, nightly, …) are not verified** — Node only signs the `release` channel (no `SHASUMS256.txt.sig` exists for them, even on nodejs.org), so they remain unverifiable. Verification is gated on the `release` channel. - **Bun / Deno are unaffected** — their download/SHASUMS URLs are hardcoded to canonical GitHub (`github.com/oven-sh/bun`, `api.github.com/repos/denoland/deno`), not mirror-configurable, so a repo can't redirect them. - **Pacquet parity:** `pacquet/crates/engine-runtime-node-resolver` has the same mirror-configurable SHASUMS logic and needs the equivalent Rust port — tracked as a follow-up (per the repo's parity rule, opening the TS side first). |
||
|
Zoltan Kochan
|
822beb5fa0 |
fix: harden package-manager bootstrap metadata (#12296)
- Resolve package-manager bootstrap metadata through trusted user/CLI registries and trusted network config, defaulting to the public npm registry instead of project/workspace registry settings. - Apply that bootstrap config in `switchCliVersion()` and `syncEnvLockfile()` so repository `.npmrc` proxy/TLS/configByUri values cannot steer package-manager bootstrap traffic. - Validate repository-provided package-manager env-lockfile entries before auto-switch install/execution: dependency paths must be registry package paths and package records must use integrity-only resolutions. - Preserve the fast path for fully resolved, valid package-manager metadata; incomplete metadata is still resolved through trusted bootstrap registries. - Handle peer-suffixed package-manager snapshots by looking up `packages` entries with `removeSuffix(depPath)` while keeping `snapshots` keyed by the full dep path. |
||
|
Zoltan Kochan
|
5f2bb9f5ba |
fix(security): verify npm registry signature before spawning a package-manager binary (#12292)
pnpm can be made to download and execute a native binary through two **repository-controlled** inputs, neither of which was authenticated before this change: 1. **pacquet install engine** — declaring `pacquet` (or `@pnpm/pacquet`) in `configDependencies` (in `pnpm-workspace.yaml`) opts in to pnpm's Rust install engine, and pnpm spawns the platform binary `@pacquet/<platform>-<arch>` during `pnpm install`. 2. **package-manager version switch** — the `packageManager` / `devEngines.packageManager` field makes pnpm download and run a specific pnpm version. This is **on by default** (`onFail` defaults to `download`) and also covers `pnpm self-update` and `pnpm with`. In both cases the repository also controls the lockfile integrity and the registry the bytes are fetched from (via `.npmrc`), so matching the lockfile integrity proves nothing — it matches the hash the attacker wrote. A cloned, untrusted repository could therefore execute an arbitrary native binary just by running a normal pnpm command. ## Fix (corepack-style registry-signature verification) pnpm now verifies the **npm registry signature** of the bytes it is about to spawn, **over the installed integrity**, against npm's public signing keys that **ship embedded in the pnpm CLI** (exactly as corepack does). If the bytes on disk were substituted or tampered with, npm's real signature does not validate over them. - New reusable `verifyInstalledPackageSignatures()` in `@pnpm/deps.security.signatures` verifies `name@version:integrity` against `dist.signatures` using the embedded keys. - Because the keys are **embedded** (not fetched), a registry the user did not vouch for cannot supply its own keypair to forge a signature. The signed packument is fetched from the **configured** registry, so an **npm mirror works transparently** — it proxies the same signed packument, with no configuration. There is intentionally **no runtime override or off-switch** for the keys. - **pacquet** (`installing/commands`): verifies the `pacquet` shim and the host platform binary. It **fails the command** if the signature does not verify or cannot be checked (e.g. registry unreachable); the only graceful fallback to pnpm's own engine is when pacquet has no binary for the current platform. - **pnpm engine** (`engine/pm/commands`): verifies `pnpm`, `@pnpm/exe`, and the host platform binary, **only on a store cache miss** (an actual download), so it adds no network round trip to every command. It **fails closed** — any verification failure, including an unreachable registry, refuses the version switch rather than running an unverified binary. ## Keeping the embedded keys fresh The embedded keys live in a generated file. `deps/security/signatures/scripts/update-npm-signing-keys.mjs` keeps them in sync with npm's keys endpoint (`pnpm check:npm-signing-keys` / `--update`), and the **create-release-pr** workflow runs the check as a gate, so a key rotation cannot silently break verification — a stale key set blocks the release until refreshed. ## Pacquet parity pacquet gained `configDependencies` support on `main` (#12285), but it has **no install-engine-spawn sink** — pacquet *is* the engine, and it does not select/spawn an alternate engine from `configDependencies` (its only config-dependency code-execution path is `updateConfig` plugin pnpmfiles, which it shares with pnpm and which this advisory does not cover). So CAND-PNPM-097 has no pacquet-side analog; no pacquet code change is needed. |
||
|
Zoltan Kochan
|
1017c36776 |
fix: block untrusted request destination env expansion (#12291)
Fixes CAND-PNPM-122 / GHSA-3qhv-2rgh-x77r by making environment expansion trust-aware for registry/auth config and request destinations.
- Stops project `.npmrc` from expanding `${...}` placeholders in registry/proxy request destinations, URL-scoped keys, and registry credential values.
- Stops repository-controlled `pnpm-workspace.yaml` from expanding `${...}` placeholders in request destinations: `registry`, `registries`, `namedRegistries`, and `pnprServer`.
- Preserves env expansion for trusted user/global/auth.ini/CLI/global config/env config so existing token, registry, and pnpr server setup flows continue to work.
- Ports the same trust boundary to pacquet for dependency-management commands.
|
||
|
Zoltan Kochan
|
230df57aa5 |
fix(bins.resolver): reject reserved manifest bin names (#12289)
Manifest bin keys "", ".", "..", and scoped forms such as "@scope/.." passed the bin-name guard because encodeURIComponent leaves them unchanged. When joined to the global bin directory during global remove/update/add operations, "." resolves to the bin directory itself and ".." to its parent, which removeBin then recursively deletes. Reject empty, ".", and ".." bin names after scope stripping in both the TypeScript resolver and the pacquet cmd-shim bin resolver. |
||
|
Zoltan Kochan
|
ea204b07b7 |
feat(pacquet): configurational dependencies support (#12285)
Ports pnpm's `configDependencies` feature to pacquet end-to-end: config dependencies are resolved and installed ahead of regular deps into `node_modules/.pnpm-config/<name>` via the global virtual store, recorded in the **env lockfile** (the first YAML document of `pnpm-lock.yaml`), their `updateConfig` plugin hooks run before the main install (including `patchedDependencies`/`catalogs` injection), and `pnpm add --config` manages them.
The whole path runs via `pacquet install` / `pacquet add --config` and is covered by tests against the mocked registry.
## What's implemented
**Resolve + install** (`pacquet-env-installer`) — clean-specifier resolution + migration of the old inline (`version+integrity`) / object (`{ tarball?, integrity }`) formats; one level of optional subdeps with `os`/`cpu`/`libc` platform fields + host filtering (exact-version-only); env-lockfile pruning of stale entries; pnpm error codes (`ERR_PNPM_BAD_CONFIG_DEP`, `CONFIG_DEP_OPTIONAL_NOT_EXACT`, `ENV_LOCKFILE_CORRUPTED`, `FROZEN_LOCKFILE_WITH_OUTDATED_LOCKFILE`, …).
**`updateConfig` pnpmfile hook** (`pacquet-hooks` + `pacquet-config`) — new `updateConfig` trait method + Node-worker dispatch; `is_plugin_name` + plugin-pnpmfile resolution (`pnpm-plugin-*` / `@pnpm/plugin-*` / `@scope/pnpm-plugin-*`) from `.pnpm-config`; full `Config` round-trip via `WorkspaceSettings` (added `Serialize`), applying only hook-changed keys so `.npmrc`/CLI values aren't clobbered; `catalog`/`catalogs` are seeded into the hook input and captured back into `Config::catalogs` (the install prefers it over re-reading the manifest), so a hook can inject catalogs a `catalog:` dependency then resolves against.
**`pnpm add --config`** (`pacquet-cli` + `pacquet-workspace-manifest-writer`) — resolves + installs the dep and writes the clean specifier into `pnpm-workspace.yaml`'s `configDependencies` block with a format-preserving edit (extends the previously catalog-only writer).
**Wiring** (`pacquet-cli`) — config-dep install + `updateConfig` hooks run at config finalization, before the main install.
**Supporting** — `graph-hasher`: `calc_leaf_global_virtual_store_path` + `calc_global_virtual_store_path_with_subdeps`; `lockfile`: `EnvLockfile` multi-document read/write + env-doc preservation in the wanted-lockfile writer; `reporter`: `pnpm:installing-config-deps` event.
|
||
|
Zoltan Kochan
|
d188316989 |
fix(pacquet): apply overrides during fresh resolution (#12283)
Fixes pnpm/pnpm#12275. ## Summary - Apply parsed `pnpm.overrides` during pacquet fresh-lockfile resolution for importer and transitive manifests. - Write catalog-resolved override values to `pnpm-lock.yaml#overrides`. - Disable prior lockfile resolution reuse when overrides drift so stale transitive subtrees are not reused. |
||
|
Zoltan Kochan
|
8c9edf29c6 |
feat(lockfile): emit patchedDependencies block in pacquet lockfile (#12281)
* feat(lockfile): emit patchedDependencies block in pacquet lockfile pacquet resolved and hashed patches for the depPath `(patch_hash=...)` suffix and at build time, but never wrote the top-level `patchedDependencies:` block into `pnpm-lock.yaml`, so a `pacquet install --lockfile-only` diverged from pnpm (issue item 6 of pnpm/pnpm#12266). Add a `patched_dependencies` field to the `Lockfile` struct in its `sortLockfileKeys` slot (between `pnpmfileChecksum` and `importers`), populated via a new `Config::patched_dependency_hashes()` that ports pnpm's `calcPatchHashes(opts.patchedDependencies)`: resolve each patch path against the workspace dir and hash it, keeping the user's verbatim keys so a bare `foo` and `foo@*` stay separate lockfile keys rather than collapsing into one group bucket. The hashes are computed once per install and threaded through `GraphToLockfileOptions`; the current lockfile (`lock.yaml`) carries them through. * feat(lockfile): check patchedDependencies drift in frozen freshness gate Now that the lockfile records `patchedDependencies` hashes, the frozen-lockfile freshness gate must reject an install when those hashes drift — otherwise editing a patch file would not invalidate the lockfile even though the patch hash participates in `(patch_hash=...)` depPath identity. Port pnpm's `getOutdatedLockfileSetting` patchedDependencies check: `check_lockfile_settings` now takes the current install's `patched_dependency_hashes()` and compares it (order-insensitively) against `lockfile.patched_dependencies`, surfacing a new `StalenessReason::PatchedDependenciesChanged` between the `ignoredOptionalDependencies` and settings checks, matching upstream's order. The pnpr fast-path bails when patches are configured. Addresses review feedback on the patchedDependencies-block PR. * feat(package-manager): detect in-place patch edits in the repeat-install fast path The optimistic repeat-install fast path skipped the whole pipeline before `check_lockfile_freshness` ran, and its workspace-state comparison only checks the `patchedDependencies` key→path map. A patch file edited in place (same config entry, new contents) therefore slipped through: `node_modules` and the lockfile's recorded patch hash could stay stale behind an "Already up to date". Port the patch-file branch of pnpm's `patchesOrHooksAreModified`: a configured patch whose mtime is newer than the workspace state's `lastValidatedTimestamp` invalidates the fast path, checked before the manifest-mtime exit so the patch reason wins. The pnpmfile branch and the `assertWantedLockfileUpToDate` re-verification remain unported. Addresses review feedback on the patchedDependencies-block PR. |
||
|
Zoltan Kochan
|
b73083537d |
feat(lockfile): emit pnpmfileChecksum in pacquet lockfiles (#12280)
* feat(lockfile): emit pnpmfileChecksum in pacquet lockfiles
Port pnpm's `calculatePnpmfileChecksum` so a pacquet install records the
`pnpmfileChecksum` field in `pnpm-lock.yaml` when the project's
`.pnpmfile.{cjs,mjs}` exports a `hooks` object — matching pnpm's value
byte-for-byte (sha256-base64 of the pnpmfile's CRLF-normalized contents)
and its position in the lockfile (right after `packageExtensionsChecksum`).
The checksum value is pure file hashing; only pnpm's
`entries.some(entry => entry.hooks != null)` gate consults the evaluated
module, answered here by a new `hasHooks` query on the existing
long-lived pnpmfile worker. A pnpmfile that exports no hooks records no
checksum, matching pnpm.
Addresses item 5 of pnpm/pnpm#12266.
* test(crypto-hash): isolate CRLF-normalization test with TempDir
`hash_from_file_normalizes_crlf` used a fixed directory under the OS temp
dir, so parallel test runs could contend on the same create/write/remove
path and flake. Use a per-test `tempfile::TempDir` instead and drop the
explicit cleanup.
|
||
|
Zoltan Kochan
|
a06adee919 |
fix(lockfile): match pnpm's runtime dependency lockfile format (#12277)
A `runtime:` dependency (`node@runtime:<ver>`, a `Variations` resolution)
diverged from pnpm's `pnpm-lock.yaml` in three ways. Reconcile all three
toward pnpm so the runtime entry round-trips byte-for-byte:
- The importer recorded `node@runtime:<ver>` instead of the prefix-stripped
`runtime:<ver>`. `real_name` returned `None` for `Variations` resolutions
(the name lives only in the fetched manifest), so `importer_dep_version`
could not strip the `node@` prefix. Read the manifest name for
`Variations`, matching the existing http-tarball path and pnpm's
`depPathToRef`.
- Each variant's `bin` was a bare string (`bin/node`) rather than pnpm's
named map (`{ node: bin/node }`). `bin_spec_for_platform` now returns the
`BinarySpec::Map` form.
- The `packages:` entry omitted `version: <ver>`. pnpm's
`toLockfileDependency` emits it whenever the depPath carries a `:`, the
manifest declares a version, and the resolution isn't a directory. Add a
`version` field to `PackageMetadata` populated under the same condition.
Verified byte-identical against pnpm 11.5.2 on a `node@runtime:26.3.0`
fixture; the whole-monorepo diff is unaffected (no runtime deps there).
Ports the lockfile-shape assertions of
`installing/deps-installer/test/install/nodeRuntime.ts:236-269`.
|
||
|
Zoltan Kochan
|
b1a6f4d4b5 |
fix(deps-resolver): align pacquet peer parent contexts (#12273)
- align pacquet peer parent context handling with pnpm for same-package child providers and peer diamonds - keep optional cached peer resolutions bubbling to later parents without an explicit provider, matching the jest-config and @types/node case - preserve pnpm's duplicated peer-suffix segments for aliased providers that resolve to the same package - include aliased child providers when their real package name is peer-relevant, not only when the install alias is peer-relevant - limit importer-seeded peer parent refs to alias/real names that can affect peer resolution, reducing clone overhead - add focused resolver tests for the issue 12272 lockfile mismatch and related diamond/alias-provider behavior - port the six related pnpm CLI alias-peer install cases to Pacquet CLI tests - pin the alias-peer CLI test peer suffix length so exact lockfile suffix assertions are independent of global config - update the Pacquet coverage upload to codecov-action v6.0.2 so it uses Codecov's current signing-key configuration Fixes pnpm/pnpm#12272. |
||
|
Zoltan Kochan
|
7006bfaf2f |
fix(npm-resolver): render workspace self-dependency as bare link: (#12276)
A workspace package depending on itself (`project_dir == the resolved
package's root_dir`) produced `link:.` because `pathdiff_string` pushed a
`.` for an empty relative path. pnpm renders it as bare `link:`
(`link:${path.relative(projectDir, projectDir)}` → `link:` + `''`).
Drop the `.`-for-empty fallback so the relative path stays empty, matching
`path.relative`. The empty case only arises for self-deps (the only time
`base == target`), so non-self workspace links are unaffected; the `directory`
field isn't serialized for `link:` deps. On the pnpm monorepo this removes all
204 `link:.` self-links (lockfile diff vs fresh pnpm 2,167 -> 1,759).
Regression test `workspace_self_dependency_renders_as_bare_link`.
Refs https://github.com/pnpm/pnpm/issues/12266.
|
||
|
Zoltan Kochan
|
55a4035abf |
fix(pacquet): record catalog snapshot for npm-aliased catalog deps (#12274)
* fix(package-manager): record catalog snapshot for npm-aliased catalog deps
`build_catalog_snapshots` read the resolved version via
`ImporterDepVersion::as_regular`, which returns `None` for `npm:`-aliased
deps (stored as `ImporterDepVersion::Alias`). So a catalog entry like
`js-yaml: npm:@zkochan/js-yaml@0.0.11` was silently dropped from the
`catalogs:` lockfile section — pnpm records `{ specifier:
npm:@zkochan/js-yaml@0.0.11, version: 0.0.11 }`.
Read the version via `ver_peer` instead, which returns the alias's suffix
(`0.0.11`) for `Alias` and the plain version for `Regular`. On the pnpm
monorepo this restores all 8 dropped aliased catalog entries (lockfile diff
vs fresh pnpm 2,191 -> 2,167); a single-entry catalog now matches pnpm
byte-for-byte.
Refs https://github.com/pnpm/pnpm/issues/12266.
* docs(plans): record catalog npm-alias snapshot test porting status
Notes that catalogs.ts:789 "catalog entry using npm alias can be reused"
is covered: its `catalogs:` snapshot assertion by the new
`aliased_catalog_dependency_records_catalog_snapshot` unit test, its reuse
half by the existing single-importer reuse test (the two-project shape
needs workspaces). Also flags the still-unported general catalogs.ts
install/workspace integration cases.
Refs https://github.com/pnpm/pnpm/issues/12266.
|
||
|
Zoltan Kochan
|
bafcfbaefc |
fix(pacquet): match pnpm peer-suffix rendering for walk-ancestor peers (#12267)
* fix(deps-resolver): carry full peer suffix for walk-ancestor peers
When a node's resolved peer is a walk-ancestor still in progress (e.g.
`@eslint-community/eslint-utils` peer-depends on its parent `eslint`),
`build_peer_id` had no depPath for it yet and fell back to the
`name@version` cycle form, freezing a collapsed suffix like
`eslint-utils@4.9.1(eslint@10.4.1)` instead of pnpm's
`eslint-utils@4.9.1(eslint@10.4.1(supports-color@8.1.1))`. Because the
depPath is the graph key, the truncation cascaded to every reference.
Port pnpm's deferred `calculateDepPath`: leave the walk (and the
provisional depPaths `find_hit` reads) untouched, capture each node's
NodeId-level edges, then recompute depPaths in a post-walk pass that
resolves each peer to its full depPath and collapses to `name@version`
only for genuine cycles — detected as peer-graph SCCs via iterative
Tarjan. The graph is rebuilt from the per-node records keyed by the
corrected depPaths, so wrongly-collapsed nodes split and correctly-
collapsed ones still merge.
Also fixes a related divergence: snapshot `dependencies` now carry only
the node's own resolved peers, mirroring pnpm's
`childrenNodeIds: { ...children, ...resolvedPeers }`, so a descendant's
optional peer (e.g. `debug`'s `supports-color`) is no longer materialized
as the parent's dependency.
Refs pnpm/pnpm#12266.
* fix(deps-resolver): don't hoist optional peers against run-resolved versions
`getHoistableOptionalPeers` hoists an optional missing peer only when a
preferred version is already in scope. pnpm reads its static
`ctx.allPreferredVersions` (wanted lockfile + manifests, empty on a fresh
install) for that decision and never folds in versions resolved during
the run. pacquet was feeding every freshly-resolved transitive version
into the same map, so an optional peer like `debug`'s `supports-color`
got hoisted against a deep-tree provider (e.g. `jest-worker`'s
`supports-color`) that pnpm never considers — resolving the peer where
pnpm leaves it bare, and non-deterministically (the outcome depended on
whether the provider landed in the map before the optional pass ran).
Snapshot the preferred-versions map before any run-resolved version is
folded in and use that snapshot for the optional-peer hoist. The
required-peer hoist keeps using the run-extended map: a required peer is
auto-installed either way, and reusing an in-tree version matches pnpm's
picker dedup (covered by the existing auto_install_* tests).
New regression test `optional_peer_only_in_resolved_tree_is_not_hoisted`,
verified to fail without the fix.
Refs pnpm/pnpm#12266.
* fix(deps-resolver): restore min-depth tie-break in build_final_graph
`build_final_graph` keyed a graph node's `depth` off the per-NodeId
`NodeRecord`, but the `pure_pkgs` and `find_hit` fast paths short-circuit
before a `NodeRecord` is created — they only lower the depth on the inline
`self.graph`, which the rebuild discards. So a package first walked at a
deeper depth and later revisited shallower kept the deeper depth, dropping
the `Math.min` tie-break those fast paths' own comments preserve.
Recompute the minimum tree depth per final depPath from `node_dep_paths`
(which records every walked NodeId, revisits included) and use it for each
rebuilt node. No lockfile-visible change today — `DependenciesGraphNode.depth`
isn't serialized and the hoister computes its own BFS depth — but it keeps
the graph's depth correct for any future consumer. Regression introduced by
the deferred depPath rebuild; guarded by a new test.
Refs pnpm/pnpm#12266.
* test(deps-resolver): port resolvePeers "peer's own peer shared with sibling"
Ports upstream's `resolvePeers.ts` "a peer's own peer is shared with a
sibling that peer-depends both" — `plugin`'s `parser` peer must resolve the
`typescript@1.0.0` that `plugin` itself uses, not be shadowed by a top-level
`parser` that resolved `typescript@2.0.0`. Exercises the depPath suffix
machinery reworked for the peer-suffix fixes; pacquet already passes it.
Also records the peer-resolution porting status (resolvePeers.ts +
hoistPeers.test.ts) in plans/TEST_PORTING.md: what's ported/covered and the
remaining gaps (multi-project peer separation, npm-alias peers, the
peer-conflict reporting edge cases, and the unported lockedPeerContext
series).
Refs pnpm/pnpm#12266.
|
||
|
Zoltan Kochan
|
e4d2fe025e | docs: clarify store trust boundary (#12268) | ||
|
Zoltan Kochan
|
c74d4e161a |
docs: document threat model and trust boundaries in security policy (#12269)
* docs: document threat model and trust boundaries in security policy Clarify that pnpm's security boundary is filesystem permissions and that reports assuming pre-existing write access to the store, lockfile, node_modules, or config files are out of scope. Explain that the content-addressable store's integrity check is corruption detection, not tamper resistance against a write-capable local adversary, since the recorded hashes live in the same trust domain as the files. * docs: route pacquet and pnpr security reports to regular issues |