mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-28 09:55:39 -04:00
23c8efeffca2e0bdf1c7995bddf6760935ae2bf6
196 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Ben Scholzen
|
96bdd57bf4 |
fix: dedupe workspace deps with children in single-project mode (#11448)
## Issue When `injectWorkspacePackages: true` is set and a workspace package depends on another workspace package that has its own dependencies, running `pnpm rm` from inside the dependent package's directory switches the lockfile protocol from `link:` to `file:`. Reproduction (workspace where `a` depends on workspace `b`, and `b` has any dependency of its own): ``` cd packages/a pnpm add redis pnpm rm redis # pnpm-lock.yaml: a's "b" entry switched from link:../b to file:packages/b ``` ## Root Cause The fix in #10575 added a defensive guard in `dedupeInjectedDeps` that skipped deduplication whenever the target workspace project's children weren't in `dependenciesByProjectId`: ```ts if (!targetProjectDeps) { if (children.length > 0) continue } ``` In single-project operations (`mutateModulesInSingleProject`, used by `pnpm rm` from inside a package directory) only the operated-on project is resolved. `dependenciesByProjectId` then only has that one project, so the guard fires for any workspace dependency whose target has children, and the protocol stays `file:`. ## Solution In single-project mode the injected dep is resolved against the same workspace package source, so dedupe is safe — *except* for peer-suffixed depPaths, whose resolution depends on the importer's peer context (a plain `link:` would lose it). The new code dedupes whenever `targetProjectDeps` is missing for a known workspace project and the depPath has no peer suffix. The peer-suffix check compares the depPath against its peer-free `pkgIdWithPatchHash` (depPaths are built as `${pkgIdWithPatchHash}${peerDepGraphHash}`), so it's exact rather than a `(`-substring heuristic. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Victor Sumner
|
1c0587698d |
fix: narrow warm install relinking (#11169)
## Problem During warm installs, pnpm relinked existing packages more broadly than necessary when only some child dependencies changed. In the narrowed relinking path, removed child aliases could also remain behind as stale links after dependency updates. ## Solution Only pass changed child edges through the relinking path for existing packages. When a child alias is no longer present in the updated dependency set, remove the obsolete link before relinking. Added regression tests for both cases: - unchanged child dependencies are not relinked unnecessarily - deleted child dependencies do not remain as stale links after a warm install --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Orgad Shaneh
|
9d79ba181e |
fix: expose update --no-save in CLI help (#12091)
The update command already honors --no-save and the docs already mention it, but the flag was missing from the update command metadata. Add the option entry so pnpm update --help shows it and the CLI surface matches the documented behavior. |
||
|
MK (fengmk2)
|
3b54d79521 |
fix(deps-installer): keep catalog-referencing overrides in sync on update (#12158)
* fix(deps-installer): re-resolve catalog-referencing overrides on update
When `pnpm.overrides` reference a catalog (e.g. `overrides: { foo: 'catalog:' }`),
`pnpm update` bumped the catalog entry during resolution but left the resolved
`overrides` in the lockfile pointing at the old version. The lockfile's
`catalogs` advanced while `overrides` stayed stale, producing an internally
inconsistent lockfile that fails a later `pnpm install --frozen-lockfile` with
ERR_PNPM_LOCKFILE_CONFIG_MISMATCH.
After resolution, re-resolve the overrides against the catalog merged with the
update's `updatedCatalogs`, so the lockfile `overrides` track the bumped catalog
just like `catalogs` and direct catalog dependencies do.
* fix(deps-installer): re-resolve catalog overrides before afterAllResolved
Address review feedback:
- Run the catalog-override re-resolution before the `afterAllResolved`
pnpmfile hook instead of after it, so a hook that edits `lockfile.overrides`
still sees and can amend the final value (the block previously ran after the
hook and would clobber its edits whenever a catalog entry was updated).
- Drop the dead `opts.catalogs ?? {}` fallback; `opts.catalogs` is required on
the install options and always defaulted to `{}`, so it is never nullish here.
* test(pacquet): cover catalog-referencing override sync on update --latest
Mirrors pnpm's regression test for keeping lockfile overrides that resolve
through a catalog in sync when `update --latest` bumps that catalog. pacquet
already behaves correctly (it threads the bumped catalogs through to override
parsing), so this is a guard against a future refactor reintroducing the
inconsistency that pnpm/pnpm#12158 fixes on the TypeScript side.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
|
||
|
Abdullah Alaqeel
|
5c12968ad6 |
fix(update): handle mixed direct and transitive selectors (#12105)
* fix(update): handle mixed direct and transitive selectors * test(update): strengthen regression test and port to pacquet The pnpm regression test passed with and without the fix: the fixture's `latest` dist-tag made a fresh install of `^100.0.0` already resolve to 100.1.0, so the assertion was trivially true. Pin the transitive dep-of-pkg-with-1-dep to 100.0.0 before install so the test genuinely fails without the fix and passes with it. Add pacquet parity regression tests for the same mixed direct/transitive selector scenario (exact-name and glob forms). pacquet has no equivalent source change to make — its `update` matches every bare-name/glob selector against direct deps and locked snapshot names in one pass, so a direct selector never gates the transitive one — but the behavior is guarded by tests to lock in pnpm/pnpm#12103 parity. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
morning-verlu
|
531f2a307c |
fix: preserve workspace specs on update (#12140)
## What - preserve existing `workspace:` dependency specifiers when `updateProjectManifest` saves updated direct dependencies and `preserveWorkspaceProtocol` is enabled - keep catalog specifiers taking precedence over resolver-normalized specs - add focused coverage for preserved and normalized local spec behavior - add a changeset for the published `@pnpm/installing.deps-resolver` change ### pacquet parity Ported the same fix to pacquet's `update` command. Previously `pacquet update --latest` routed every direct dependency through a registry `latest` lookup, so a `workspace:` local-path dependency (e.g. `workspace:../packages/foo/dist`) was rewritten into a registry version — corrupting the manifest (in the regression test it became `0.0.1-security`). Both `--latest` rewrite sites now skip registry resolution for such specs via `is_workspace_local_path_specifier`, a faithful port of pnpm's `isWorkspaceLocalPathSpecifier`. The gate is unconditional in the `--latest` path because `preserveWorkspaceProtocol` is always on there (its only override derives from `linkWorkspacePackages` under `--workspace`, which cannot be combined with `--latest`). Fixes #3902 --------- Co-authored-by: morning-verlu <258725120+morning-verlu@users.noreply.github.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Abdullah Alaqeel
|
61969fbddf |
fix(deps-status): detect lockfile-only changes (#12106)
## Summary Fixes `pnpm install` with `optimisticRepeatInstall` incorrectly returning `Already up to date` when `pnpm-lock.yaml` changed but project manifests did not. Fixes #12100. ## Root Cause `checkDepsStatus` used modified manifest mtimes as the only signal for whether it needed to validate dependency status. If no manifest was newer than `workspaceState.lastValidatedTimestamp`, it returned `upToDate: true` before checking whether the wanted lockfile had changed. That skipped lockfile validation for workflows like: - `git checkout HEAD~1 -- pnpm-lock.yaml` - restoring only `pnpm-lock.yaml` from a stash - external tools rewriting the lockfile without touching manifests ## Changes - Check wanted lockfile mtimes before taking the optimistic fast path. - If any wanted lockfile is missing or newer than the workspace state timestamp, validate all projects instead of only modified manifests. - Add a regression test proving a lockfile-only change does not skip wanted-lockfile validation. - Add a patch changeset for `@pnpm/deps.status` and `pnpm`. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Truffle
|
6d35338691 |
fix: detect changes inside file: dependencies on repeat install (pacquet + pnpm) (#12317)
## Summary - `pnpm install` reports "Already up to date" after edits inside a `file:` dependency's directory or after repacking a `file:` tarball. This is a v11 regression from the `optimisticRepeatInstall` default flip in pnpm/pnpm#11158. Fixes pnpm/pnpm#11795. - `checkDepsStatus` gains a `treatLocalFileDepsAsOutdated` option: when set, any project manifest declaring a local file dependency makes the check report not up to date. `installDeps` sets it on the optimistic fast path, so projects with local file dependencies always run a real install, which refetches those dependencies (the v10 behavior). - The predicate covers `file:` specs, path-prefixed specs (`./`, `../`, `~/`, absolute POSIX paths, and Windows drive paths including drive-relative ones like `C:dir`, matching the local resolver's `isFilespec`), and bare tarball file names (`vendor/pkg.tgz`). It is deliberately narrower than the local resolver's bare-path matching: a bare `user/repo` is statically indistinguishable from a git shorthand at this layer, and matching it would kill the fast path for every project with git dependencies, so protocol-carrying and URL specs stay on the fast path. - `pnpm.overrides` entries are scanned with the same predicate: an override mapping to a local file spec redirects every matching dependency in the graph to that directory, so it has the same blind spot as a direct local file dependency. Registry and `link:` overrides keep the fast path. - The option is caller-scoped on purpose. `verifyDepsBeforeRun` also consumes `checkDepsStatus`, and treating `file:` deps as always stale there would force a reinstall before every `pnpm run`. Its behavior is unchanged, and a regression test pins that. - pacquet port in the same commit: `check_optimistic_repeat_install` bails unconditionally on `file:` specifiers, because its only caller is the install command, the one consumer that sets the flag upstream. `link:` specifiers are excluded on both sides: they are symlinked, so changes inside them flow through without a reinstall. ## Why Both branches of `checkDepsStatus` are blind to content changes inside a `file:` dependency. The workspace branch exits early with `upToDate: true` when no project manifest's mtime moved, without ever reaching `linkedPackagesAreUpToDate`. The non-workspace branch exits at the manifest-vs-lockfile mtime gate the same way. Editing a source file inside a `file:` dependency bumps neither, so the fast path can never see it; the fix has to bail before those gates rather than refine them. This is the fix shape (a) I proposed in my diagnosis on the issue thread ([comment](https://github.com/pnpm/pnpm/issues/11795#issuecomment-4504177744)): the cost is a full resolution on repeat installs only for projects that declare `file:` dependencies, which is exactly what v10 did. The manifest-only comparison in `@pnpm/lockfile.verification` (`allProjectsAreUpToDate`) is intentional for the install-proper path and asserted by its tests, so this PR leaves it untouched. ## Checks - `pnpm --filter @pnpm/deps.status test test/checkDepsStatus.test.ts` (31 passed, 13 new) - `pnpm --filter @pnpm/deps.status run compile` and `pnpm --filter @pnpm/installing.commands run compile` (tsgo + eslint clean) - `cargo test -p pacquet-package-manager optimistic_repeat_install` (51 passed, 7 new; run in a rust:1.95.0 container) - `cargo fmt --check -p pacquet-package-manager` - `RUSTDOCFLAGS="-D warnings" cargo doc -p pacquet-package-manager --no-deps` --- Written by an agent (Claude Code, claude-fable-5). --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
c112b6106b |
feat(install): add --dry-run option (npm-style preview) (#12449)
## Description Adds a `--dry-run` option to `pnpm install` with **npm-style preview semantics**: it runs a full dependency resolution and reports what a real install **would** add/remove/update, but writes **nothing** to disk (no lockfile, no `node_modules`, no `.modules.yaml`, no workspace-state file) and **always exits 0**. ``` $ pnpm install --dry-run Dry run complete. A real install would make the following changes (nothing was written to disk): Importers . + is-negative 1.0.0 Packages + is-negative@1.0.0 ``` When the lockfile is already up to date it prints `Dry run complete. pnpm-lock.yaml is up to date; a real install would make no changes.` Resolves https://github.com/pnpm/pnpm/issues/7340. ### Why this shape An earlier attempt (#12270, now closed) implemented `--dry-run` as `--frozen-lockfile --lockfile-only` — i.e. a fail-on-drift *lockfile validator*. That collides with the well-established meaning of `--dry-run` across npm/yarn ("preview, never fail") and duplicated existing behaviour (`pnpm install --frozen-lockfile --lockfile-only` already does that). This PR implements the intuitive preview meaning instead. ### How it works (pnpm) - Reuses the existing `lockfileCheck` callback (resolve fully, skip the lockfile write, hand back the before/after wanted lockfile) plus `lockfileOnly` (skip `node_modules`, the workspace-state file, and metadata-cache writes). - The frozen/headless fast path is disabled whenever `lockfileCheck` is set, so a check-only install always resolves and never materialises anything. - The before/after lockfiles are diffed (reusing the dedupe diff engine, now exported as `calcDedupeCheckIssues`) and rendered into the report. - `--dry-run` with a configured pnpr server is rejected (that path resolves/links through the server). ### Pacquet Ported in the second commit — `pacquet install --dry-run` forces the fresh-resolve path, skips every write (a new `dry_run` flag on `InstallWithFreshLockfile` skips the `pnpm-lock.yaml` save), and a new `dry_run` module diffs the existing lockfile against the freshly-resolved one and prints the same report. |
||
|
Abdullah Alaqeel
|
817f99dbe5 |
fix(resolver): stabilize transitivePeerDependencies in dependency cycles (#12286)
## Summary Fixes lockfile churn where a package's `transitivePeerDependencies` (e.g. `supports-color` via `debug`) could be dropped — and shift between packages — when the package participates in a dependency cycle. Which packages carry a given transitive peer depended on resolution order, so upgrading an unrelated dependency churned the lockfile. ## Root cause When peer resolution walks into a cycle, the cycle is broken by dropping the repeated package's subtree, so the re-entry occurrence resolves against truncated children and looks peer-free. That occurrence was then recorded as "pure" in `purePkgs` — a verdict keyed by package id, not by context. A later occurrence of the same package, reached through a different parent that *can* see the full subtree, hit the `purePkgs` short-circuit and returned an empty peer set, dropping the transitive peers it should have surfaced. Because the outcome depends on which occurrence is walked first, it was order-dependent. ## Fix Don't record a cycle re-entry's resolution in `purePkgs` / `peersCache` (a re-entry is detected when the package id already appears in the ancestor chain). Its truncated peer sets aren't authoritative for the package as a whole, so leaving the caches untouched lets later occurrences resolve correctly — or reuse the package's authoritative, non-truncated entry via `findHit`. This is a minimal guard at the cache-population site: it adds no post-resolution pass and does not change `transitivePeerDependencies` for packages that aren't in cycles. This PR also includes an independent fix: when collecting peer providers from a node's children, match each child's resolved package name in addition to its alias, so `pnpm add my-alias@npm:real-pkg` is visible to peer resolution when `real-pkg` is a peer dependency name. Both the TypeScript pnpm CLI and the Rust (pacquet) port are updated in parity. Fixes pnpm/pnpm#5108 Related `transitivePeerDependencies`-instability reports: pnpm/pnpm#5552, pnpm/pnpm#5794, and the `transitivePeerDependencies` aspect of pnpm/pnpm#9992 (the out-of-scope version drift in pnpm/pnpm#9992 is a separate problem and is not addressed here). --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Abdullah Alaqeel
|
564619f04d |
fix(install): persist revoked builds to .modules.yaml (#12221) (#12224)
* fix(install): persist revoked builds to .modules.yaml (#12221) After a build-script dependency whose approval was revoked (e.g. via git stash dropping allowBuilds from pnpm-workspace.yaml) is re-added, the revocation detection populated ignoredBuilds in memory but the install path's writeModulesManifest had already run, so .modules.yaml never recorded the revoked packages. pnpm approve-builds then read an empty ignoredBuilds and reported 'no packages awaiting approval'. Re-read the manifest from disk after the revocation scan and write back the updated ignoredBuilds, merging with any entries the install path captured. * refactor(install): address review comments - Inline dead ignoredBuildsFromInstall indirection - Drop unsafe allowBuilds cast in the regression test --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
eba03e0d5c |
fix: detect reverted catalog entries on install (#12438)
* fix: detect reverted catalog entries on install After an update bumped a catalog entry in pnpm-workspace.yaml, the workspace state cache stored the pre-update catalog versions, so reverting the entry back to its original version was reported as "Already up to date" instead of reinstalling the previous version. Fold the catalogs written during the install into the catalogs recorded in the workspace state so a later install detects the reverted entry as outdated. Closes https://github.com/pnpm/pnpm/issues/12418 * fix: harden catalog merge against prototype pollution and entry loss Address review feedback on the catalog-merge helper: - mergeCatalogs now builds null-prototype records and copies entries with Object.defineProperty, so a catalog or dependency name like __proto__ (which can flow in from parsed pnpm-workspace.yaml) becomes an ordinary own property instead of corrupting the result's prototype. - The recursive per-project install path now accumulates updatedCatalogs with mergeCatalogs instead of a shallow Object.assign, so two projects updating different entries of the same catalog no longer clobber each other. |
||
|
Zoltan Kochan
|
29ab905c21 |
fix: preserve catalog version range policy on update (#12416)
A named catalog whose name parses as a version (e.g. catalog:express4-21) had its range policy overridden by pnpm update because whichVersionIsPinned misread the catalog: reference in the previous specifier as a pinned version. The catalog reference carries no pinning of its own, so the prefix from the catalog entry is now preserved. Closes https://github.com/pnpm/pnpm/issues/10321 |
||
|
Zoltan Kochan
|
1e82e001cd | chore(release): 11.7.0 (#12414) | ||
|
Zoltan Kochan
|
a6d485abca |
fix: stabilize Windows pacquet install tests (#12410)
- Replace lockfile env-document stream scanning with a FileHandle read loop that closes deterministically, including split-BOM handling. - Align pacquet's default `virtualStoreDirMaxLength` with pnpm's Windows default. - Forward pnpm's effective virtual store max length to delegated pacquet installs through `PNPM_CONFIG_VIRTUAL_STORE_DIR_MAX_LENGTH`, so currently published pacquet versions do not write mismatched `.modules.yaml` on Windows. |
||
|
David Barratt
|
3a271413d8 |
fix: prevent a pinned locked peer provider from leaking to sibling nodes (#12320)
* fix: prevent a pinned locked peer provider from leaking to sibling nodes When the locked-peer-context pinning introduced in pnpm/pnpm#12083 runs for a node that has no child dependencies, parentPkgs aliases the parent's object, so writing the pinned provider into it exposed the provider to every sibling resolved afterwards. Sibling order follows resolution completion order, so optional peers of siblings resolved nondeterministically and "pnpm dedupe --check" failed intermittently in CI. Copy parentPkgs before pinning so the pin stays scoped to the node and its own subtree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf: copy parentPkgs only before the first pin write --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
74a2dc9027 |
feat(installing): delegate resolution to pacquet >= 0.11.7 when configured (#12210)
* feat(installing): delegate resolution to pacquet >= 0.11 when configured When pacquet is declared in configDependencies, pnpm previously always ran it with --frozen-lockfile (pnpm resolved, pacquet materialized). If the installed pacquet is >= 0.11 it ships its own resolver, so a non-frozen plain install on the default isolated linker is now delegated end-to-end: pacquet resolves, writes pnpm-lock.yaml, and materializes in a single pass. Older pacquet keeps the resolve-then-materialize split, and add/update/remove still resolve in pnpm. * test(pnpm): cover pacquet 0.11 resolution delegation end-to-end Bump the e2e pacquet pin to 0.11.0 (published under both pacquet and @pnpm/pacquet) and add tests for the resolve path, the materialize-only fallback (pinned to 0.2.14), and the scoped alias. Un-skip the add/update tests now that pacquet 0.11 writes a compatible .modules.yaml, and fix the update test (is-positive has no v4; update from 1.0.0 instead). Also preserve the configDependencies env document when pacquet resolves: pacquet rewrites pnpm-lock.yaml without the leading env YAML doc, which dropped configDependencies and broke the next --frozen-lockfile install. Capture it before delegating and restore it after. * fix(installing): restore configDependencies env document even when pacquet fails On the pacquet resolve-delegation path the configDependencies env document was only restored after a successful pacquet run. A non-zero exit could leave a rewritten pnpm-lock.yaml without it, breaking the next --frozen-lockfile install's config-deps freshness gate. Restore it in a finally block, swallowing (and warning on) any restore error so it cannot mask the original pacquet failure. * fix(installing): require pacquet 0.11.7 for resolving installs * fix(installing): skip pacquet in lockfile check mode * fix(installing): harden pacquet lockfile handoff * fix(installing): preserve policy handling with pacquet * fix(installing): skip pacquet when lockfile is disabled * fix(installing): skip pacquet with branch lockfiles |
||
|
tsushanth
|
86e70d2896 |
fix(installing.commands): key selectProjectByDir graph by project.rootDir (#12380)
* fix(installing.commands): key selectProjectByDir graph by project.rootDir
`selectProjectByDir` constructs a single-entry `ProjectsGraph` for the
non-workspace install path. It was using `searchedDir` (`opts.dir`) as
the key, but downstream `recursive()` builds `manifestsByPath` from the
projects array (keyed by `project.rootDir`) and then looks up entries
via `manifestsByPath[rootDir]` where `rootDir` is drawn from
`Object.keys(selectedProjectsGraph)`. When `opts.dir` and
`project.rootDir` differ in platform-normalized form (most often on
Windows due to drive-letter casing), the lookup falls through as
`undefined` and `pnpm add <pkg>` crashes with:
Cannot destructure property 'manifest' of 'manifestsByPath[rootDir]' as it is undefined
Pin the graph key to `project.rootDir` in both `installing/commands/src/installDeps.ts`
and `installing/commands/src/import/index.ts`, so the keys stay in sync
with `manifestsByPath`. Closes https://github.com/pnpm/pnpm/issues/12379
Written by an agent (Claude Code, claude-opus-4-7).
* docs: remove redundant comments
* test(installing.commands): cover project graph keying
* Revert "test(installing.commands): cover project graph keying"
This reverts commit
|
||
|
Zoltan Kochan
|
09f0bab194 |
test(deps-installer): make the parallel custom-resolver check deterministic (#12401)
The 'runs checks in parallel' test raced three real setTimeout delays (10/20/30ms) and asserted their exact completion order. Timer scheduling jitter on Windows CI runners reordered the sub-50ms timers, so the suite flaked and failed every Windows test job. Replace the timing race with a start-barrier: each hook blocks until all hooks have started. This only completes if the checks run concurrently -- were they awaited one at a time, the first hook would wait forever for siblings that never start. No timers, no ordering assumptions. |
||
|
tsushanth
|
8dcd9a055c |
fix(installing.commands): show only names in checkbox summary (#12393)
Closes pnpm/pnpm#12386 |
||
|
Zoltan Kochan
|
681b593eb2 |
fix: support scope-specific registry auth tokens (#12392)
pnpm can now use different auth tokens for different package scopes, even when those scopes use the same registry URL. Previously, auth was selected only by registry URL. If `@org-a` and `@org-b` both used `https://npm.pkg.github.com/`, they had to share the same token. This caused problems for registries that issue tokens per organization or per scope. Configure a scope-specific token by adding the package scope after the registry URL in the auth key: ```ini @org-a:registry=https://npm.pkg.github.com/ @org-b:registry=https://npm.pkg.github.com/ //npm.pkg.github.com/:@org-a:_authToken=${ORG_A_TOKEN} //npm.pkg.github.com/:@org-b:_authToken=${ORG_B_TOKEN} //npm.pkg.github.com/:_authToken=${FALLBACK_TOKEN} ``` `pnpm login --registry=https://npm.pkg.github.com --scope=@org-a` writes the token to the same scope-specific auth key. When installing or publishing `@org-a/*`, pnpm uses `ORG_A_TOKEN`. For `@org-b/*`, pnpm uses `ORG_B_TOKEN`. Packages without a matching scope continue to use the registry-wide fallback token. |
||
|
Ahmed Abdelbaset
|
ab0b7d1847 |
feat(link): support --trust-lockfile flag on pnpm link to match other commands (#12374)
|
||
|
Zoltan Kochan
|
5b402ea22b |
test(deps-installer): isolate the heavy deepRecursive test in its own process (#12388)
test/install/deepRecursive.ts resolves @teambit/bit's enormous circular and peer-dependency graph. Measured in a CI-faithful container (Linux, Node 22.13.0, amd64, default ~4 GB heap) it peaks at ~3.6 GB — it fits the default heap on its own, but not with the memory the other deps-installer test files leave behind in the same jest process (the --experimental-vm-modules module registry is not reclaimed between files). That overflow is the "FATAL ERROR: Reached heap limit" the CI suite hit. Run deepRecursive in a dedicated jest process (.test:heavy) so it gets the whole default heap to itself, and run the rest (.test:rest) in a separate process with it excluded via a negative-lookahead path pattern. The two runs cover every test file exactly once. This makes the earlier OOM workarounds unnecessary, so they are reverted: - the 5-way sharding of the suite (deepRecursive was the sole culprit; the remaining files are a subset of what historically ran in one process), and - the workerIdleMemoryLimit in the with-registry jest preset. No global heap bump: every run stays within Node's default ~4 GB, matching the budget pnpm has in production. |
||
|
Zoltan Kochan
|
03143cad22 | test: shard deps-installer integration tests (#12376) | ||
|
Zoltan Kochan
|
4819fb4e66 |
fix(pacquet): match pnpm lockfile resolution (#12372)
## Summary - Match pacquet peer-resolution and lockfile output to pnpm for transitive optional peer variants. - Apply pnpm's Yarn compatibility package extensions in pacquet, with `ignoreCompatibilityDb` support. - Add regression coverage on both the pnpm resolver test and pacquet resolver/install paths. Fixes pnpm/pnpm#12330. |
||
|
Zoltan Kochan
|
3d1a980036 | test: fix CI resource usage (#12373) | ||
|
Zoltan Kochan
|
9b35a6004e |
fix(deps-resolver): make shared children resolution deterministic (#12362)
## Summary - Make shared package child resolution deterministic by choosing the owner by depth, importer order, and parent path instead of async completion timing. - Keep non-owner and stale occurrences lazy while reusing the current owner children and missing-peer context. - Port the same behavior to pacquet and add TypeScript plus Rust regression coverage. ## Verification - `pnpm --filter @pnpm/installing.deps-resolver test test/resolveDependencyTree.test.ts` - `cargo test -p pacquet-resolving-deps-resolver` - Pre-push hook: TypeScript typecheck, pnpm bundle, lint, spellcheck, meta lint, cargo fmt, cargo doc, cargo dylint, taplo format check Fixes pnpm/pnpm#12358 |
||
|
Kalven Schraut
|
dfa91df6e8 |
fix: resolve musl pacquet binary on musl-based systems (#12347)
* fix: resolve musl pacquet binary on musl-based systems The pacquet binary packages are split by libc on linux and only the matching one is installed, but resolvePacquetBin always asked for the glibc name. On Alpine and other musl systems the frozen install failed with: Cannot find module '@pacquet/linux-x64/pacquet'. fix pnpm/pnpm#12049 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: sort imports in runPacquet.ts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: verify the musl pacquet binary package on musl-based systems The signature verification hard-coded the glibc platform package name, so on musl systems it verified a package other than the binary that is actually spawned. Share one platform-package-name helper between resolvePacquetBin and collectPacquetPackagesToVerify. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
f648e9b7c4 |
fix: contain hoisted dependency aliases (GHSA-fr4h-3cph-29xv) (#12343)
* fix: contain hoisted dependency aliases (GHSA-fr4h-3cph-29xv) The `nodeLinker: hoisted` install restores its dependency graph straight from the lockfile via `lockfileToHoistedDepGraph`, which joins each dependency alias under a `node_modules` directory and imports the package files there. On a frozen / up-to-date lockfile, resolution is skipped entirely, so the alias validation added for the resolution path never runs. A crafted lockfile alias such as `../../../escape` could therefore escape the install root, and reserved aliases such as `.bin`, `.pnpm`, or `node_modules` could overwrite pnpm-owned layout. Validate every alias at the hoisted-graph directory sink. The shared `safeJoinModulesDir` helper now rejects aliases that are not valid npm package names (path-traversal, absolute, and reserved names) in addition to its containment check, and the hoisted graph routes its `dep.name` sink through it. Pacquet mirrors the boundary: `safe_join_modules_dir` validates the hoister's `dep.0.name` before adding the graph node or recursing, reusing the same dependency-name rule it already applies to direct-dependency aliases. Both stacks surface `ERR_PNPM_INVALID_DEPENDENCY_NAME`. --- Written by an agent (Claude Code, claude-fable-5). * fix: reject invalid dependency aliases at the lockfile verification gate Add an always-on, policy-independent structural check to verifyLockfileResolutions that rejects any importer or package-snapshot dependency alias that is not a valid npm package name. A dependency alias becomes a `node_modules/<alias>` directory at link time, so an alias with path-traversal segments or a reserved name (`.bin`, `.pnpm`, `node_modules`) could escape the install root or overwrite pnpm-owned layout. This complements the linker-sink guards: the verifier runs before any fetch or filesystem work and covers every node linker at once, while the sink guards still protect the `trustLockfile` path the verifier skips. The check runs before the cache lookup so a record written by a version that predates the rule cannot fast-path around it, and before the `packages` guard so a tampered importer alias is caught even when nothing is installed. `isValidDependencyAlias` is now exported from `@pnpm/installing.deps-resolver` and reused here. Pacquet mirrors the gate in its lockfile-verification crate with a matching `ERR_PNPM_INVALID_DEPENDENCY_NAME` verdict. --- Written by an agent (Claude Code, claude-fable-5). * docs(package-manager): drop redundant explicit intra-doc link target `is_valid_dependency_alias` is in scope via `use`, so the bare intra-doc link resolves on its own. The explicit path target tripped `rustdoc::redundant-explicit-links` under the CI Doc job's `cargo doc --document-private-items` (the local pre-push hook runs `cargo doc` without that flag, so it didn't surface). --- Written by an agent (Claude Code, claude-fable-5). * refactor(lockfile-verification): fold the alias check into the single candidate pass The dependency-alias check ran as its own full traversal of the lockfile in addition to collectCandidates' existing pass over every package snapshot. Fold it into that pass instead: collectCandidates now also validates each importer and snapshot dependency alias and returns the invalid ones alongside the resolution-shape violations, so the lockfile is walked once per verification rather than twice. Because collectCandidates runs after the verification-cache lookup, the alias check is now covered by the cache the same way the resolution-shape check is: a new dependencyAliasCheck cache identity makes a record written before this rule existed fail canTrustPastCheck, forcing a re-verification. The shared helper is renamed withOfflineCheckCacheIdentities and appends both offline-structural-check identities. No behavior change for valid lockfiles; the same ERR_PNPM_INVALID_DEPENDENCY_NAME is thrown for invalid aliases. Mirrored in pacquet's lockfile-verification crate. --- Written by an agent (Claude Code, claude-fable-5). * refactor: declare pushInvalidAliases after its caller, trim duplicated comments Move `pushInvalidAliases` below `collectCandidates`, following the repo's declare-after-use convention. Collapse the repeated "an alias becomes a node_modules directory, so a traversal/reserved name escapes or overwrites layout" explanation that was copied across the verifier, the hoisted-graph error, and the pacquet mirror down to a single reference each — the full rationale lives once in the validating sink (`safeJoinModulesDir` / `safe_join_modules_dir`) and the user-facing error hints. --- Written by an agent (Claude Code, claude-fable-5). |
||
|
Zoltan Kochan
|
c16eb0a154 |
perf: run lockfile verification concurrently with frozen install (#12227)
## Problem `pnpm install` with a frozen lockfile got noticeably slower because lockfile verification blocks every later install stage. The verification gate (the `minimumReleaseAge`/`trustPolicy` policy revalidation plus the tarball-URL anti-tamper check) issues a registry round trip per lockfile entry, and the whole install waited for it to finish before any fetching or linking could begin. ## Change (pnpm / TypeScript) Run lockfile verification **concurrently** with fetching and linking instead of blocking on it, while keeping two guarantees intact: 1. **No lifecycle script runs on an unverified lockfile.** A `verifyLockfile` gate is threaded into both `buildModules` call sites — `headlessInstall` (frozen path) and `_installInContext` (full-resolution path) — and awaited immediately before any dependency lifecycle script runs. The projects' own `preinstall`/`postinstall` hooks are held to the same gate at both `runLifecycleHooksConcurrently` call sites, covering the `enableModulesDir: false` path that skips the build phase. If verification failed, the gate throws before a single script executes. 2. **The verdict is always reconciled.** `settleInstall(_install(), verifyLockfilePromise)` awaits the verification verdict first so it takes precedence and fails fast (even mid-install), then surfaces the install's result/error. This also covers paths that skip the build phase entirely (`ignoreScripts`, `lockfileOnly`, empty lockfile). Verification's synchronous prologue (cache lookup, lockfile hash, candidate collection) still runs against the pristine lockfile before `_install()` mutates `ctx.wantedLockfile`, so the concurrent async fan-out reads a stable snapshot — no data race. The verification verdict deliberately takes precedence over a concurrent install error: `pnpm add`'s full-resolution path can throw its own generic "resolution-policy violations produced but no handler wired" for the same underlying violation, and `settleInstall` makes sure the specific `minimumReleaseAge`/`trustPolicy` error is what surfaces. ## Change (pacquet / Rust) Same optimization ported to `pacquet/crates/package-manager/`. `Install::run` builds the resolution verifiers up front but dispatches the verification fan-out per path: - **Frozen materialization path:** verification runs concurrently with `CreateVirtualStore` (the fetch), settled with a `select!` so the verdict takes precedence: a rejected lockfile aborts the fetch in flight (fail-fast), while a fetch failure waits for the verdict and only surfaces once the lockfile is known trusted — an unrelated fetch error can't mask a rejected lockfile. The verdict is always reached before linking and `BuildModules`, so no dependency lifecycle script runs on an unverified lockfile. - **Lockfile-only / up-to-date short-circuits and the fresh-resolve path:** keep an eager blocking gate — they have no fetch to overlap. A verification failure surfaces as the same `InstallError::LockfileVerification` variant regardless of which path produced it. On the pnpr client, a frozen restore now skips resolution entirely: tarball downloads start from the local lockfile at t=0 (filtered through one batched store-index existence probe, so a warm store prefetches nothing) while the server delivers only the trust verdict via the new `POST /v1/verify-lockfile` endpoint, concurrently with the fetch. ## Tests - pnpm: `test/install/trustLockfile.ts` covers the rejection itself, the `trustLockfile` opt-out, and both script gates — a dependency `postinstall` never runs when verification fails, and the projects' own lifecycle hooks never run either, asserted on the `enableModulesDir: false` path with a *slow*-rejecting verifier (an instantly-rejecting one aborts the install before the hooks are attempted, which would hide a missing gate). Existing verification/lifecycle/`minimumReleaseAge` suites pass. - pacquet: existing `frozen_lockfile_gate_rejects_under_huge_minimum_release_age` (unit) and `install_fails_under_huge_minimum_release_age` (CLI) assert the frozen install aborts with no virtual-store materialization on verification failure — proving the fail-fast settle cancels the fetch. New: `without_store_hits` + `StoreIndex::contains_many` unit tests pin the warm-store prefetch filter, and the frozen pnpr CLI test swaps the registry for a zero-expectation server before the restore, proving a warm-store frozen restore makes no registry requests. - pnpr client/server: integration tests cover `/v1/verify-lockfile` accepting a clean lockfile, rejecting a policy violation, honoring `trustLockfile`, and forwarding the client's credential map (each verify call targets a fresh pnpr so no verdict/metadata cache can satisfy it without exercising the credential). - clippy / `cargo doc -D warnings` / rustfmt / eslint clean; package-manager, lockfile-verification, store-dir, pnpr-client, and CLI pnpr-install suites all pass. ## Behavioral nuance On a *rejected* lockfile, fetching/linking may now have partially populated the store/`node_modules` before the abort (previously nothing ran, since verification went first). The command still fails with the same error code and no lifecycle scripts run. |
||
|
Victor Sumner
|
61810aa684 |
feat: add --frozen-store for installs against a read-only store (#12190)
## What
Adds an opt-in `frozenStore` / `--frozen-store` setting (default `false`) that lets `pnpm install --offline --frozen-lockfile` run against a package store that lives on a **read-only filesystem** — a Nix store, a read-only bind mount, an OCI layer.
## Why
A normal install fails against such a store **not** because it writes package content, but because it unconditionally:
1. opens the SQLite `index.db` in WAL mode, which needs to create `-shm`/`-wal` sidecars in the store directory; and
2. writes a project-registry entry under the store.
Both fail with `attempt to write a readonly database` / `EROFS` on a read-only store directory, even when the store is complete and the lockfile is frozen. This blocks any deployment that wants an immutable, content-addressed store.
## How
When `frozenStore` is enabled, pnpm opens `index.db` through the SQLite **`immutable=1`** URI — which tells SQLite the file cannot change underneath it, so it bypasses the WAL/`-shm` sidecar machinery entirely and reads the raw file with zero sidecar creation — and suppresses every store-write path. Pair it with `--offline --frozen-lockfile` against a fully-populated store. It is incompatible with two settings that would write into the store, and each throws a clear config-conflict error before any network or store access: **`--force`** (which bypasses the no-write-on-hit skip → `ERR_PNPM_CONFIG_CONFLICT_FROZEN_STORE_WITH_FORCE`) and a configured **pnpr server** (which would fetch and write packages into the store → `ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR`).
> Plain `SQLITE_OPEN_READ_ONLY` is **not** sufficient: opening a WAL-mode db read-only still tries to create the `-shm` sidecar, which fails on a read-only *directory*. `immutable=1` is the load-bearing piece.
> **Node.js requirement:** `node:sqlite` only passes `SQLITE_OPEN_URI` to SQLite (so the `immutable=1` query is honored rather than treated as part of a literal filename) starting in **v22.15.0** (22.x line), **v23.11.0**, and every **v24+**. pnpm's `engines` floor is `>=22.13`, so on a runtime older than that the frozen open is detected up front and fails with a clear `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` instead of SQLite's cryptic "unable to open database file". (pacquet uses rusqlite with an explicit `SQLITE_OPEN_URI` flag, so it has no such floor.)
> The `immutable=1` URI path is also percent-encoded (`%`→`%25`, `?`→`%3f`, `#`→`%23`, in that order, leaving `/` literal) so a store path containing those characters doesn't truncate the path or inject a spurious query parameter — applied identically in both stacks.
### Build backstop under the global virtual store
Under the global virtual store (default), a package's directory lives **inside** the store (`{storeDir}/links/...`). Applying a patch or running an allowlisted lifecycle script writes into that directory — so on a frozen store it would crash mid-build with a raw `EROFS`. A fully-seeded store never reaches the build step (patched/built packages are imported from the side-effects cache and filtered out by the `isBuilt` gate), so any residual build candidate means the seed is **missing that package's build output**.
`buildModules` now refuses up front with `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` and actionable guidance ("rebuild the seed with their scripts enabled, or remove them from `onlyBuiltDependencies`") instead of failing cryptically once a script starts. The check is gated on the global virtual store — under the isolated linker, slot directories live in the writable project-local store, so builds there are fine. Non-allowlisted scripts never run, so they are not treated as a blocking write.
Bin-linking has its own read-only-store edge under the global virtual store. On a **warm** checkout (the project's `.bin/<name>` already points at the seed target) `linkBin` returns before touching the store, so it is write-free. But on a **fresh** checkout it (re)creates the bin and calls `fixBin`, whose `chmod` targets the bin's **source file inside the store** (`{storeDir}/links/...`) — which is refused with `EPERM`/`EACCES` on a read-only store, even though a complete seed already ships that bin executable (so the `chmod` is redundant). `@pnpm/bins.linker` now wraps that call in `ensureExecutable`: it swallows the refusal when the target is already executable and rethrows otherwise, so bin-linking is write-free against a frozen store on a cold checkout too, while a genuinely non-executable bin (a broken seed) still surfaces as an error.
The blocking predicate distinguishes the two write kinds: a **patch** is applied regardless of `ignoreScripts`, so a patched package is always blocked; a **lifecycle script** is suppressed under `ignoreScripts`, so an allowlisted build-requiring package is *not* blocked when scripts are off (it would write nothing). This avoids falsely rejecting a valid `--ignore-scripts` frozen install. **Optional dependencies are exempt**: a build or patch failure on an optional dependency is non-fatal at runtime, so a seed missing an optional package's build output skips that build (emitting the `skipped-optional-dependency` log) instead of blocking the install — in both stacks.
### Both stacks (parity rule)
**TypeScript pnpm CLI**
- Config plumbing (`@pnpm/config.reader`): `frozen-store` type, config-file key, default, `Config.frozenStore`.
- The read-only open branch (`@pnpm/store.index`): `immutable=1`, read-only statements, throwing mutators.
- Wiring through `@pnpm/store.controller` and `@pnpm/store.connection-manager` to the sole `StoreIndex` construction site.
- Gating the project-registry write (`@pnpm/installing.context`).
- The `--force` / pnpr-server conflict guards (`@pnpm/installing.deps-installer`) and CLI surface (`@pnpm/installing.commands`).
- **After-install rebuild** (`@pnpm/building.after-install`): the post-install rebuild opens its `StoreIndex` immutably under the flag, so re-reading the store for a rebuild never attempts a writable open against the frozen store.
- **Worker fix:** `@pnpm/worker` opens its *own* writable `StoreIndex` on every `readPkgFromCafs` cache hit, so a pure read crashed on a frozen store. `frozenStore` is threaded through to `getStoreIndex` and keyed into its connection cache.
- **Build backstop** (`@pnpm/building.during-install`): `buildModules` throws `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` for a GVS slot that would build/patch on a frozen store, honoring `ignoreScripts`; threaded from `@pnpm/installing.deps-installer`.
- **Bin-linking on a read-only store** (`@pnpm/bins.linker`): `linkBin` wraps the `fixBin` chmod in `ensureExecutable`, which tolerates `EPERM`/`EACCES` when the bin's store-resident source is already executable (a complete seed) and rethrows otherwise — so a fresh checkout against a frozen store links bins without crashing on the redundant chmod. Catch-on-failure keeps the writable hot path at zero added syscalls.
**Rust pacquet**
- A dedicated `open_immutable` / `shared_immutable_in` opens via `immutable=1`, selected only under the flag. Plain `open_readonly` keeps the ordinary `SQLITE_OPEN_READ_ONLY` open (WAL locking intact) because normal installs read the index while the same process's `StoreIndexWriter` writes it concurrently — an immutable connection skips all locking and change detection, so a concurrent writer would make those reads undefined.
- `--frozen-store` CLI flag + `frozenStore` workspace-yaml setting.
- The store-index writer is replaced with a drain-and-drop stub (`spawn_disabled`) and `init_store_dir_best_effort` is skipped under the flag.
- **Build backstop:** `build_modules` returns `BuildModulesError::FrozenStoreNeedsBuild` (`ERR_PNPM_FROZEN_STORE_NEEDS_BUILD`) under the same GVS + frozen-store condition, threaded from `config.frozen_store`. The gate keys off `should_run_scripts` (which already folds the allow-build policy), so it is correct without an explicit ignore-scripts branch — pacquet has no configurable ignore-scripts mode yet.
pacquet already separated read-only index access (`shared_readonly_in`, or `shared_immutable_in` under the flag) from writes, so it never had the worker-conflation bug; the flag makes the "no writes attempted" contract explicit and gates the remaining best-effort write attempts.
## Testing
- **TS:** `store.index` frozen-mode-on-`0555`-directory test (reads work, writes throw `ERR_PNPM_FROZEN_STORE_WRITE`) plus a path-with-`?` open test — both gated on the runtime's immutable-URI support, with a complementary test asserting `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` fires where that support is absent (the CI Node 22.13.0 path); `config.reader` round-trip; `deps-installer` `--force` and pnpr-server conflict guards; `worker`/`package-requester` unit tests. End-to-end on a `chmod -R 0555` store: install succeeds, `node_modules` materializes, no `-shm`/`-wal`/`-journal` sidecars; negative control without the flag fails as expected; incomplete store → clean offline error.
- **pacquet:** `open_immutable_reads_wal_db_on_readonly_directory` unit test plus the `immutable_sqlite_uri` encoding test and a path-with-`?` open test; yaml + CLI fold tests; **integration test** `frozen_store_installs_against_a_read_only_store` — primes a store, `chmod 0555` the tree, runs `install --frozen-lockfile --frozen-store --offline`, asserts success + materialized `node_modules` + zero sidecars. Confirmed load-bearing by reverting the `immutable=1` fix (test then fails).
- **Build backstop (both stacks):** `building/during-install` unit tests — approved-build-not-cached and patched-not-cached refuse; cached, non-allowlisted, and non-GVS cases pass through; **approved-build-under-`ignoreScripts` passes through while patched-under-`ignoreScripts` still refuses** — and the matching pacquet `build_modules` tests (`frozen_store_gvs_patch_not_seeded_refuses` + GVS-off / frozen-off controls). Each confirmed load-bearing by disabling the relevant guard and watching the corresponding test fail.
- **Bin-linking (`bins/linker`):** `ensureExecutable` tests with `fixBin` mocked to reject with `EPERM` — an already-executable bin source resolves (and `fixBin` is asserted called, so it isn't the warm skip-guard passing), a non-executable one rethrows `EPERM`. Confirmed end-to-end by running the built `linkBins` against a real `chflags uchg`-immutable store: the executable-seed case resolves and links the bin, the non-executable-seed control throws `EPERM`.
A changeset is included with `"pnpm": minor` and `"@pnpm/bins.linker": patch` (the read-only-store bin-linking fix).
|
||
|
Zoltan Kochan
|
a31faa7c19 |
chore: update dependencies (#12346)
* chore: update dependencies
Update all catalog dependencies to their latest versions, except those
held back by pnpm's supported Node.js floor (>=22.13) or known issues;
each held-back entry now carries a comment in pnpm-workspace.yaml
explaining why.
Notable changes:
- msgpackr 1.11.8 -> 2.0.4 (unpinned; types compile again and the
store-index output is byte-compatible with 1.x in both directions)
- typescript 5.9.3 -> 6.0.3, esbuild 0.28, commitlint 21,
concurrently 10, eslint plugin majors (autofixed one import-sort
error they introduced)
- open 11, memoize 11, cli-truncate 6, pidtree 1, @yarnpkg/core 4.8,
@rushstack/worker-pool 0.7.18
- removed unused nock devDependency and the deprecated @types/tar stub
- bole stays on 5: bole 6 is ESM-only and under Jest the workspace
logger's ESM copy and the published @pnpm/logger's CJS bole 5 no
longer share the globalThis.$$bole output registry, breaking
reporter assertions
Held back due to Node >=22.13 support floor: ssri 14,
write-file-atomic 8, validate-npm-package-name 8,
normalize-package-data 9, npm-packlist 11, ini 7 (need ^22.22.2),
undici 8 (needs >=22.19), cspell 10 (needs >=22.18; bumped to 9.8.0
instead).
* chore: add changeset for updated dependency ranges
Patch-bump every published package whose runtime dependency or peer
dependency range changed in the dependency update, following the
precedent of commit
|
||
|
Zoltan Kochan
|
53b105416f |
chore(release): 11.6.0 (#12336)
* chore(release): 11.6.0 * docs: update CHANGELOG.md --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Zoltan Kochan
|
84bb4b1a04 |
perf: close the warm-resolve, symlink-churn, and download-concurrency gaps (#12329)
## Motivation The [vlt.sh benchmarks](https://benchmarks.vlt.sh/) (2026-06-11 run, pacquet 0.11.3) show pacquet several times slower than the fastest package managers in the warm-metadata fresh-resolve cells (`cache`: 3.9–8.1x), the cold-cache frozen-install cells (`lockfile`: up to 10x on vue), and `clean`. Profiling the babylon and vue fixtures locally (macOS time profiles of the warm fresh resolve and the install tail) surfaced three independent causes, fixed here. ## Changes ### 1. Deprecation probing without manifest hydration (pacquet) With `minimumReleaseAge` active (the default), every range pick runs `filter_pkg_metadata_by_publish_date`, and any dist-tag pointing outside the maturity cutoff (`next`, `beta`, `canary`, a too-fresh `latest`) repopulates by scanning all candidates and reading each candidate's `deprecated` flag. Each read hydrated the full version manifest — a complete `serde_json` parse including the flattened catch-all map. On babylon's warm fresh resolve this was the single largest CPU consumer (~10 thread-seconds, all on the resolve task's critical path). `PackageVersions::is_deprecated` now answers from the raw fragment (substring pre-check, then a single-field deserialize with the same normalization as `PackageVersion::deprecated`), the tag-repopulation loop parses candidate versions once per filter call (mirroring the `parsedSemverCache` in pnpm's `filterPkgMetadataByPublishDate`), and the deprecated-pick fallback uses the probe instead of hydrating every version. **babylon warm fresh resolve: `resolve_workspace` 7.5s → 2.6s.** ### 2. Relative-symlink up-to-date check (pacquet) `force_symlink_dir` joined an existing link's relative contents onto the link parent and compared the result *verbatim* against the wanted target. Virtual-store links contain `..` segments (`../../<pkg>/node_modules/<name>`), so the joined path never compared equal and every up-to-date symlink was unlinked and recreated. Node's `path.relative` — which upstream `symlink-dir`'s `isExistingSymlinkUpToDate` builds on — resolves its arguments, so pnpm treats those links as current. Both sides now pass through `lexical_normalize`. The babylon install tail was dominated by exactly this unlink+symlink churn. **babylon warm install: 6.8s → 4.7s; warm frozen install: 4.2s → 2.3s.** ### 3. Default network concurrency floor 16 → 64 (pnpm + pacquet) The default was `min(64, max(workers * 3, 16))`. Downloads are I/O-bound, not CPU-bound: on a 4-vCPU CI runner the formula yields 16 concurrent requests, so a low-latency registry drains 600–1300-tarball installs 16 at a time while staying unsaturated — a large share of the cold-cell (`lockfile`/`clean`) gap on the benchmark runners. The default is now `min(96, max(workers * 3, 64))`; the `networkConcurrency` setting still overrides it. Applied to `@pnpm/installing.package-requester`, the lockfile-resolution verifier fan-out that mirrors its floor, and the same two spots in pacquet. Changeset included (minor). **This is a user-visible defaults change on both stacks — flagging it explicitly for review.** ## Local results (M-series macOS, vlt fixtures, isolated store/cache) | cell | before | after | |---|---|---| | vue `cache` | 1159 ms | **479 ms** | | vue `cache+lockfile` | 621 ms | **392 ms** | | vue no-op install | 48 ms | **41 ms** | | babylon `cache` | ~8.8 s | **4.75 s** | | babylon `cache+lockfile` | ~4.2 s | **2.27 s** | vue's warm cells are now ahead of every competitor measured locally; babylon's `cache` cell closed from ~2.5x behind the leader to ~1.35x (the remainder is the per-file store-integrity verify and per-file linking that the pnpm store contract requires). ## Validation - `cargo nextest`: registry, resolving-npm-resolver, resolving-deps-resolver, lockfile-verification, network, fs, tarball, package-manager, cli — 1300+ tests, all green; new unit tests cover the deprecation probe (string/bool/empty/corrupt shapes, nested-key false positives) and cross-parent relative-symlink reuse (fails without the fix). - Lockfile stability: `--lockfile-only` output is byte-identical before/after on vue; on babylon the resolved **package-version sets are identical across 6 runs (3 per binary)**. The babylon lockfile does flap between runs in the peer-suffix shape of `webpack-dev-server@5.2.2` (`(bufferutil@4.1.0)(utf-8-validate@5.0.10)` appearing/disappearing) — this is **pre-existing nondeterminism** reproducible with the unmodified binary against itself, in the optional-peer area; worth a separate issue. - Pre-push checks (fmt, taplo, `cargo doc -D warnings`, dylint) pass; eslint (root config) and `tsgo --build` pass for the two touched TS packages. |
||
|
Zoltan Kochan
|
f11b4fcad7 |
feat(deps-installer): announce reused lockfile-verification verdicts (#12326)
When the lockfile-verification gate short-circuits on a cached verdict, it used to stay completely silent, which made it look like the supply-chain policy gate never ran (pnpm/pnpm#12324). Emit a new `cached` status on the pnpm:lockfile-verification channel carrying the reused record's verifiedAt timestamp, and render it in the default reporter as "Lockfile passes supply-chain policies (verified 2h ago)" (falling back to "previously verified" for records that predate the timestamp). The event fires only when policy verifiers are active, so the shape-only check every install performs stays quiet. Ported to pacquet in the same change: a `Cached` variant on the reporter's LockfileVerificationMessage with the matching camelCase wire shape, emitted from the same cache-hit point in verify_lockfile_resolutions. |
||
|
Zoltan Kochan
|
52be454d57 |
fix: infer missing platform fields of optional dependencies from the package name (#12312)
* fix: infer missing platform fields of optional deps from the package name Some registries strip the os/cpu/libc fields (or just libc) from the version objects of the packuments they serve. Resolution then saw every platform-specific optional dependency as platform-unrestricted, so pnpm downloaded and installed the binaries of every platform regardless of supportedArchitectures, and wrote lockfile entries without the platform fields, which broke installs from that lockfile on every machine. Platform-specific binary packages encode their platform in the package name (e.g. @nx/nx-win32-arm64-msvc), so packageIsInstallable now fills the missing platform fields of an optional dependency from the name's tokens. Since every install path decides installability through that check before fetching, foreign-platform binaries are skipped without even downloading them, in fresh resolution and in headless installs with both node linkers alike. A package that declares no platform fields at all is treated as platform-specific only when an operating system is recognized in its name, so a generic name segment (such as 'arm' on its own) never gets a package skipped. Fixes https://github.com/pnpm/pnpm/issues/11702 Fixes https://github.com/pnpm/pnpm/issues/9940 * chore: add platform name tokens to the cspell dictionary * fix(package-is-installable): infer missing platform fields of optional deps from the package name Port of pnpm commit https://github.com/pnpm/pnpm/commit/34875b2d7c (PR https://github.com/pnpm/pnpm/pull/12312). Some registries strip the os/cpu/libc fields (or just libc) from the version objects of the packuments they serve, and lockfile entries written from such metadata lack the fields too, so every platform's binaries were installed regardless of supportedArchitectures. Platform-specific binary packages encode their platform in the package name (e.g. @nx/nx-win32-arm64-msvc), so the installability check now fills the missing platform fields of an optional dependency from the name's tokens: infer_platform_from_package_name + inferred_platform in pacquet-package-is-installable, applied inside package_is_installable (hoisted linker) and in compute_skipped_snapshots (isolated linker, with the check cache keyed by the snapshot's optional flag since the verdicts can differ). The any_installability_constraint fast path now also considers optional snapshots whose names infer a platform their metadata row does not declare, so the inference is reachable on lockfiles without any declared constraint. Same guard rails as upstream: declared fields always win (each field is filled only when missing — a missing libc alone is inferred, disambiguating -gnu vs -musl), and a package declaring no platform fields at all engages the inference only when an operating-system token is recognized in its name, so a generic name segment such as 'arm' on its own never gets a package skipped. Fixes https://github.com/pnpm/pnpm/issues/11702 Fixes https://github.com/pnpm/pnpm/issues/9940 * test: shut the metadata-stripping proxy down cleanly and forward the request method |
||
|
Zoltan Kochan
|
d976edf4ec |
perf: content-check modified manifests and fall back to the current lockfile on the repeat-install fast path (pacquet + pnpm) (#12315)
## Why On [benchmarks.vlt.sh](https://benchmarks.vlt.sh/) (2026-06-10 run, pacquet 0.11.2), pacquet ranked **8th–9th of 10** in every `lockfile+node_modules` variation — slower than pnpm, npm, yarn and vlt — e.g. astro: pacquet 936 ms vs pnpm 502 ms; babylon: pacquet 9.08 s vs pnpm 0.85 s. It also trailed vlt/npm in the `node_modules` and `cache+node_modules` variations (astro 1.5 s / 0.7 s, babylon 8.9 s / 6.4 s). ### Root cause Tracing the actual runner (a `pacquet` PATH shim logging per-invocation file stats) showed the harness's prepare step rewrites `package.json` with **identical content but a fresh mtime** before every timed run, while `clean_all_cache` wipes `~/.cache/pnpm` (the packument cache and `lockfile-verified.jsonl`), and the `node_modules` variations additionally delete `pnpm-lock.yaml`. - **pnpm**: `checkDepsStatus`'s modified-manifests branch re-checks the *content* against the lockfile (`assertWantedLockfileUpToDate`, `assertLockfilesEqual`, `linkedPackagesAreUpToDate`) and reports "Already up to date" with zero network — ~0.5 s is just Node startup. Verified locally: with all caches wiped and the network blocked, `pnpm install` still prints "Already up to date" in 228 ms. - **pacquet**: the optimistic repeat-install check bailed on *any* newer manifest mtime, fell into the full pipeline, and the awaited `minimumReleaseAge` lockfile-verification gate — its verdict cache wiped — re-fetched **one packument per locked package** per run: 0.94 s on astro, 9.1 s on babylon. - With `pnpm-lock.yaml` deleted, both stacks pay a similar fan-out on the synthesized-from-current lockfile (`tryLockfileVerificationCache` bails before the content-hash index when the lockfile file can't be stat'd), which is why even pnpm needs 2.2–11.6 s there. ## What **Commit 1 — port the modified-manifests branch of `checkDepsStatus`** (at pnpm/pnpm@cc4ff817aa) into `optimistic_repeat_install`: - a manifest whose mtime is newer than `lastValidatedTimestamp` is re-checked against the wanted lockfile instead of invalidating the fast path: lockfile-settings drift (`getOutdatedLockfileSetting`), per-importer `satisfiesPackageManifest`, and a port of `linkedPackagesAreUpToDate` for workspace links (`isLocalFileDepUpdated` for `file:` directory specifiers is not ported — those conservatively fall through to the full install); - `assertLockfilesEqual` runs when the wanted lockfile is newer than the reference (workspace: `lastValidatedTimestamp`; single-project: the current lockfile's mtime, mirroring upstream's branch shapes); - the workspace branch refreshes `lastValidatedTimestamp` after a passing content check, like upstream's `updateWorkspaceState` call; - the frozen-dispatch freshness gate is split into reusable pieces (`parse_config_overrides`, `check_lockfile_settings_drift`, `check_importer_satisfies`) shared with the new check, and the per-importer slice is no longer hard-wired to the root importer. **Commit 2 — treat the current lockfile as the wanted one when `pnpm-lock.yaml` is missing (pacquet)** (requested by @zkochan): when `node_modules` is intact, `<virtual_store_dir>/lock.yaml` — the record of what the previous install materialized — stands in as the wanted lockfile for the same content checks, and `pnpm-lock.yaml` is regenerated from it (byte-identical to what the full install's synthesize-from-current path would write) before the fast path reports "Already up to date". Single-project installs with no lockfile on either side still refuse the fast path; `lockfile: false` skips the regeneration; a manifest that no longer matches (e.g. `pacquet add`) still takes the full resolve. ## Validation Re-ran the actual vlt.sh harness (same scripts, ubuntu-24.04-arm runner) with the patched binary swapped into the npm-installed pacquet; all hyperfine runs exited 0: | fixture, variation | pacquet 0.11.2 (official run) | patched | pnpm (same validation run) | |---|---|---|---| | astro, `lockfile+node_modules` | 935.6 ms (rank 9/10) | **38–39 ms** | 599–621 ms | | babylon, `lockfile+node_modules` | 9 084 ms (rank 8/10) | **86.6 ms ± 0.6** | 767.7 ms | | astro, `node_modules` | 1 501 ms (rank 4/10) | **41.2 ms ± 0.8** | 2 226 ms | | astro, `cache+node_modules` | 704 ms (rank 5/10) | **42.9 ms ± 0.9** | 2 017 ms | | babylon, `node_modules` | 8 962 ms (rank 6/10) | **107.8 ms ± 1.0** | 11 566 ms | After this change only aube (~5 ms) and bun (~8 ms) stay ahead in these five variations. `cargo nextest run -p pacquet-package-manager` (438 tests), `-p pacquet-cli` install suites, workspace clippy `-D warnings`, dylint, fmt, taplo and `typos pacquet` are clean. New tests cover the touched-but-identical manifest, a manifest that adds a dependency, a diverged wanted-vs-current lockfile, the state-timestamp refresh, linked siblings inside/outside the manifest range, lockfile regeneration (modified and unmodified manifests, workspace state bump), and `lockfile: false`. Two offline e2e tests additionally pin the "zero network, zero pipeline" property through `Install::run`'s real dispatch: a real install, registry dropped, caches wiped, repeat install pointed at a dead port — both verified discriminating by temporarily disabling the content check. Two existing tests were adjusted: `fresh_install_records_lockfile_verification_for_mtime_bypassed_noop` now disables the optimistic check explicitly so it keeps guarding the verification-cache wiring it was written for, and `optimistic_repeat_install_does_not_short_circuit_when_lockfile_missing` now passes `lockfile: None` (matching the CLI contract for a missing file) and documents that the guard requires *both* lockfiles to be absent. **Commit `1ee88c5107` — the same fallback in the pnpm CLI** (`@pnpm/deps.status` + `@pnpm/installing.commands`): `checkDepsStatus` lets the current lockfile stand in when `pnpm-lock.yaml` is missing (workspace shared-lockfile branch and single-project branch), runs the same content checks against it, and returns it as `wantedLockfileToRestore`; `installDeps` writes `pnpm-lock.yaml` back from it before reporting "Already up to date". Guard rails: no lockfile on either side still refuses the fast path, `useLockfile: false` skips the restore, a failed restore falls through to the full install, and the stand-in is disabled under `useGitBranchLockfile` (there a missing plain `pnpm-lock.yaml` is the steady state and the branch lockfile may legitimately differ from the current one). Verified with the bundled CLI: install → delete `pnpm-lock.yaml` → `pnpm install --registry=http://127.0.0.1:9/` prints "Already up to date" in 29 ms and restores the lockfile byte-identically. Covered by 5 new `checkDepsStatus` unit tests and an `installing/commands` integration test that runs the repeat install against a dead registry. Changeset bumps `@pnpm/deps.status`, `@pnpm/installing.commands`, and `pnpm` (minor). |
||
|
Zoltan Kochan
|
b7195db5c8 | chore(release): 11.5.3 (#12305) | ||
|
Zoltan Kochan
|
bf1b731ee6 |
fix: harden allowBuilds artifact approvals (#12294)
## Summary Package-name `allowBuilds` entries no longer approve lifecycle scripts for artifacts whose identity a name cannot pin: git, git-hosted tarball, direct tarball, and local directory dependencies. To approve such an artifact explicitly, use its peer-suffix-free lockfile depPath as the `allowBuilds` key — error hints, `pnpm ignored-builds`, and `pnpm approve-builds` print exactly that key. - `AllowBuild` policy functions identify packages by `DepPath` instead of caller-supplied name/version. The policy parses name and version out of the depPath itself, so name-keyed rules can never be fed an identity that disagrees with the gated artifact. `AllowBuildContext` carries only an explicit `trustPackageIdentity` override, used to evaluate a previously recorded policy under its original semantics when detecting revoked approvals. - Identity trust is derived from the depPath shape: a registry-style depPath (`name@semver`) is a trusted identity. This is sound because lockfile verification runs a new unconditional, offline structural pass that rejects lockfiles where such a key is backed by a git, directory, or git-hosted tarball resolution (`ERR_PNPM_RESOLUTION_SHAPE_MISMATCH`), while the npm resolution verifier already binds explicit tarball URLs of semver-keyed entries to the registry's own `dist.tarball` unconditionally. The pass runs inside the existing candidate walk and participates in the verification cache key (`resolutionShapeCheck`) on both the gate's and the fresh-resolve record paths, so the stat-only cache fast path stays sound and records written before the rule existed are re-verified. - Installs detect approvals that were revoked (or stopped applying) for git/tarball artifacts and surface those packages as ignored builds; approvals granted for previously ignored builds trigger a rebuild of exactly those packages. - `preparePackage` always treats the fetched manifest as an untrusted identity: it requires a `pkgResolutionId` and gates on the synthesized `name@<resolution id>` depPath. scp-style git URLs are normalized to `ssh://` form in resolution ids, and the git fetcher reuses `createGitHostedPkgId` from the resolver instead of re-deriving ids. - Under the global virtual store, `pnpm rebuild` locates a projection created before the approval was granted by following the project's node_modules link, since the projection hash includes the allowBuilds verdict (relocating the projection instead is tracked in https://github.com/pnpm/pnpm/issues/12302). - New shared helpers: `removePeersSuffix()` in `@pnpm/deps.path` (string-level peer-suffix stripping for user-written keys) and `allowBuildKeyFromIgnoredBuild()` in `@pnpm/building.policy` (the key under which an ignored build is approved). - pacquet mirrors the whole policy: `AllowBuildPolicy::check(dep_path)` derives trust from the dep path, the git-fetcher allow-build closures take only the dep path, `pacquet-lockfile-verification` gains the same structural pass, error code, and cache identity, and dep-path keys normalize via `remove_suffix`. - `shell-quote` is overridden to 1.8.4 (GHSA-w7jw-789q-3m8p / CVE-2026-9277). - Test-harness fix: a module-level drain keeps the global log stream flowing in the deps-installer lifecycle tests, so reporter assertions no longer receive the buffered backlog of earlier installs that ran without a reporter. |
||
|
Zoltan Kochan
|
5f2bb9f5ba |
fix(security): verify npm registry signature before spawning a package-manager binary (#12292)
pnpm can be made to download and execute a native binary through two **repository-controlled** inputs, neither of which was authenticated before this change: 1. **pacquet install engine** — declaring `pacquet` (or `@pnpm/pacquet`) in `configDependencies` (in `pnpm-workspace.yaml`) opts in to pnpm's Rust install engine, and pnpm spawns the platform binary `@pacquet/<platform>-<arch>` during `pnpm install`. 2. **package-manager version switch** — the `packageManager` / `devEngines.packageManager` field makes pnpm download and run a specific pnpm version. This is **on by default** (`onFail` defaults to `download`) and also covers `pnpm self-update` and `pnpm with`. In both cases the repository also controls the lockfile integrity and the registry the bytes are fetched from (via `.npmrc`), so matching the lockfile integrity proves nothing — it matches the hash the attacker wrote. A cloned, untrusted repository could therefore execute an arbitrary native binary just by running a normal pnpm command. ## Fix (corepack-style registry-signature verification) pnpm now verifies the **npm registry signature** of the bytes it is about to spawn, **over the installed integrity**, against npm's public signing keys that **ship embedded in the pnpm CLI** (exactly as corepack does). If the bytes on disk were substituted or tampered with, npm's real signature does not validate over them. - New reusable `verifyInstalledPackageSignatures()` in `@pnpm/deps.security.signatures` verifies `name@version:integrity` against `dist.signatures` using the embedded keys. - Because the keys are **embedded** (not fetched), a registry the user did not vouch for cannot supply its own keypair to forge a signature. The signed packument is fetched from the **configured** registry, so an **npm mirror works transparently** — it proxies the same signed packument, with no configuration. There is intentionally **no runtime override or off-switch** for the keys. - **pacquet** (`installing/commands`): verifies the `pacquet` shim and the host platform binary. It **fails the command** if the signature does not verify or cannot be checked (e.g. registry unreachable); the only graceful fallback to pnpm's own engine is when pacquet has no binary for the current platform. - **pnpm engine** (`engine/pm/commands`): verifies `pnpm`, `@pnpm/exe`, and the host platform binary, **only on a store cache miss** (an actual download), so it adds no network round trip to every command. It **fails closed** — any verification failure, including an unreachable registry, refuses the version switch rather than running an unverified binary. ## Keeping the embedded keys fresh The embedded keys live in a generated file. `deps/security/signatures/scripts/update-npm-signing-keys.mjs` keeps them in sync with npm's keys endpoint (`pnpm check:npm-signing-keys` / `--update`), and the **create-release-pr** workflow runs the check as a gate, so a key rotation cannot silently break verification — a stale key set blocks the release until refreshed. ## Pacquet parity pacquet gained `configDependencies` support on `main` (#12285), but it has **no install-engine-spawn sink** — pacquet *is* the engine, and it does not select/spawn an alternate engine from `configDependencies` (its only config-dependency code-execution path is `updateConfig` plugin pnpmfiles, which it shares with pnpm and which this advisory does not cover). So CAND-PNPM-097 has no pacquet-side analog; no pacquet code change is needed. |
||
|
Zoltan Kochan
|
e4d2fe025e | docs: clarify store trust boundary (#12268) | ||
|
David Barratt
|
29a496ac7c |
fix: make peer-dependent deduplication deterministic (#12179)
* fix(deps-resolver): make peer-dependent deduplication deterministic When a peer-suffixed package variant is a subset of two or more mutually incompatible larger variants, `deduplicateDepPaths` chose which one to collapse it into based on the order dep paths were inserted into the per-pkgId set, which reflects importer/resolution order and varies between platforms. The same workspace could then resolve to different lockfiles on different machines, making `pnpm dedupe --check` alternate between pass and fail. The depth-count sorter `nodeDepsCount(a) - nodeDepsCount(b)` is not a total order, so equal-count variants keep their (order-dependent) relative position. Tie-break on the dep path string to give a deterministic winner regardless of insertion order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(deps-resolver): assert resolved depPath is defined before order check The order-invariance assertion compared two undefined values, which would pass silently if the depPath never resolved. Assert both are defined first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pacquet): port dedupePeerDependents collapse Port pnpm's `dedupePeerDependents` pass (resolvePeers tail block + `deduplicateAll` / `deduplicateDepPaths` / `nodeDepsCount` / `isCompatibleAndHasMoreDeps`) into pacquet, carrying the pnpm/pnpm#12179 determinism fix: the collapse target is chosen by a total order over `(dep count, dep path)` so it no longer depends on importer/resolution order. Runs in `resolve_peers_workspace` after `dedupe_injected_deps`, gated on `config.dedupe_peer_dependents` (default true) threaded through `WorkspaceResolveOptions`. Duplicate variant groups are reconstructed by grouping the finished graph on `resolved_package_id` instead of threading pnpm's `depPathsByPkgId` through the walk. Since pacquet has no unified post-resolve lockfile pruner, the pass reuses `dedupe_injected_deps::prune_unreachable` to drop collapsed orphans so they don't surface in the lockfile. Both unit tests from pnpm's dedupeDepPaths.test.ts are ported (the version-mismatch collapse and the importer-order determinism case), plus end-to-end remap+prune and incompatible-variant coverage. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
089484aca8 |
perf(pnpr): resolve server-side and fetch tarballs directly (#12232)
## Summary Reworks pnpr from an install/file accelerator into a resolve-only accelerator: - `POST /v1/resolve` resolves against the client-supplied registries and returns a gzipped JSON lockfile response - pacquet/pnpm clients then fetch tarballs normally from registries with their own credentials and existing parallel fetch/integrity paths - pnpr no longer serves package file bytes or store-index rows, so the server-side file diff, file-frame response, grant table, and public-package byte-gating code are removed The follow-up resolution fast paths are included on the new measured path: - repeated public no-lockfile resolves use a bounded in-memory TTL cache - fresh frozen input lockfiles skip the server-side lockfile-only pacquet resolve after verification proves the lockfile is usable - input lockfile verification and the verdict cache are preserved ## Benchmark Integrated benchmark on Linux shows small improvements in all pnpr rows, with the clearest movement in hot restore. This should be treated as an incremental win rather than a large install-speed change. | Scenario | `pnpr@HEAD` | `pnpr@main` | Change | | --- | ---: | ---: | ---: | | fresh restore, cold cache + cold store | `1.677 s ± 0.090` | `1.686 s ± 0.070` | ~0.6% faster | | fresh restore, hot cache + hot store | `492.5 ms ± 18.1` | `521.9 ms ± 33.4` | ~5.6% faster | | fresh install, cold cache + cold store | `1.997 s ± 0.025` | `2.003 s ± 0.038` | ~0.3% faster | | fresh install, hot cache + hot store | `1.211 s ± 0.024` | `1.236 s ± 0.038` | ~2.0% faster | ## Trade-off Going registry-direct means pnpr no longer gates tarball bytes itself. Private package access is enforced by the upstream registry when the client fetches tarballs. Resolution policy still runs server-side: lockfile verification, release-age policy, trust policy, and resolved package selection continue to happen before the client fetches bytes. |
||
|
Zoltan Kochan
|
4b4d38361c | chore(release): 11.5.2 (#12207) | ||
|
shiminshen
|
e7e99f04e4 |
fix: do not crash when a catalog specifier is a range (#11706)
## Summary `pnpm update --recursive --lockfile-only <pkg>@<version>` crashed with `Invalid Version: <range>` when the catalog entry for `<pkg>` was a range (e.g. `^21.2.10`) and `catalogMode` was `strict` or `prefer`. This is the exact command Renovate's pnpm artifact updater runs; monorepos using `catalog:` with any range specifier were blocked from Renovate-driven lockfile updates. **Root cause:** in `installSome`, the catalog-match short-circuit guards `semver.eq(wantedDep.bareSpecifier, catalogDepSpecifier)` with `semver.validRange` on both sides. `validRange` returns truthy for ranges, but `semver.eq` constructs `new SemVer(...)` internally and throws on a range. **Fix:** use `semver.valid` instead of `semver.validRange` on both sides of the equality guard. Range specifiers now fall through to the existing mismatch handling (`CatalogVersionMismatchError` in `strict` mode, warn-and-use-direct in `prefer` mode) instead of crashing. Behavior for concrete-on-both-sides is unchanged. Closes #11570 ## Behavior after the fix This turns a crash into pnpm's normal catalog-mismatch handling; it does **not** make a strict-mode update succeed when the catalog is a range: - **`catalogMode: strict`** — rejects with `ERR_PNPM_CATALOG_VERSION_MISMATCH` (clean, actionable error instead of a stack trace). - **`catalogMode: prefer`** — warns and uses the direct version. - **concrete-vs-concrete** — unchanged (`semver.eq` still runs). ## pacquet parity The TypeScript fix patches a crash inside pnpm's `catalogMode` mismatch gate — a feature pacquet had not ported at all (`catalog-mode` was in the config parity test's `NOT_PORTED` list). Rather than just the one-liner, this PR ports that gate to pacquet so the two stacks match: - **config:** new `CatalogMode { Manual, Strict, Prefer }` enum (default `manual`), `Config.catalog_mode`, wired through `pnpm-workspace.yaml` (`catalogMode:`) and the env overlay; `catalog-mode` moved from `NOT_PORTED` to a mapped row in the `pnpm_default_parity` contract test. - **package-manager:** `check_catalog_mode` + `CatalogVersionMismatchError` (`ERR_PNPM_CATALOG_VERSION_MISMATCH`), invoked from `update` before the manifest is mutated. The comparison only treats both sides as equal when each parses as a concrete semver version, so a ranged catalog entry falls through to the mismatch path instead of reaching an exact-version comparison — the Rust analogue of the `semver.valid` guard above. The crash itself can't occur in pacquet (Rust's `node-semver` returns a `Result` rather than throwing); the port is the *feature* with the range-correct comparison built in, so pacquet behaves like fixed pnpm. **Not ported** (the surrounding pieces pacquet still lacks, so wiring them would diverge from pnpm rather than match it): the `add`-path cataloging that relies on `defaultCatalog` rewriting, and the `saveCatalogName` → `pnpm-workspace.yaml` auto-cataloging half. The gate is therefore wired into `update <pkg>@<version>` / `--latest` (the Renovate scenario), not `add`. |
||
|
Zoltan Kochan
|
5192edf40e |
feat(pnpr): forward credentials and add per-user access grants for external private registries (#12184) (#12189)
Closes #12184 (part 2). #12181 shipped the per-caller access gate on `POST /v1/install`, which authorizes every served package against pnpr's own `packages:` policy — the complete answer **while pnpr fetches anonymously**. This PR adds the remaining piece: forwarding the caller's per-registry credentials so the accelerator can resolve/fetch **external private** content as the caller, and gating that content per user against the registry that actually owns it. ## Credential forwarding (issue steps 1–2) - **Wire:** `POST /v1/install` gains an `authHeaders` body map (`{ "//host/path/": "Bearer …" }`, the shape `AuthHeaders::from_map` consumes / `getAuthHeadersFromCreds` produces) plus an HTTP `Authorization` header. The body map carries the *upstream* registry tokens; the header identifies the caller to pnpr's own gate and keys the grant table. - **pacquet plumbing:** a request-scoped `Arc<AuthHeaders>` is threaded via a new `Install.auth_override` field and an `auth_override` param on `build_resolution_verifiers`, so resolution/verification run as the caller **without** baking per-user auth into the interned `&'static Config` (which would leak one config per user). - **Server:** `handle_install` builds the per-request `AuthHeaders` and threads it through resolve, verify, and `fetch_uncached` (which now returns the freshly-fetched set). - **Clients:** pacquet `pnpr-client` and `@pnpm/pnpr.client` send `registry` / `namedRegistries` / `authHeaders` + `Authorization`; the TS path sources them from the caller's registry credentials via `@pnpm/network.auth-header` (`getAuthHeadersFromCreds` is newly re-exported). `@pnpm/worker` is unchanged — downloads happen server-side. - **Credential scope:** both clients forward the caller's *full* credential map, not a subset scoped to the declared registries. The registries a dependency graph touches aren't knowable up front — a transitive package can be scope-routed to another registry or pinned to a tarball URL on a host that's in `.npmrc` but isn't a declared registry — so pnpr attaches the right token per fetched URL exactly as a local install does. These are package-fetch credentials going to the very service the caller configured to fetch its packages. ## Per-user grant table (issue steps 3–4) Externally-resolved private content carries no pnpr policy, so the store's possession of the bytes must not authorize a user the upstream never cleared. A served package is dispatched by **whether a forwarded credential was used to fetch it**: - **No forwarded cred → pnpr-as-authority:** the existing local `packages:` policy check, unchanged. - **Forwarded cred → upstream-as-authority:** gated against a persistent `(user, name@version)` grant table (SQLite, modeled on `VerdictCache`). Freshly fetched this request ⇒ record + allow (the upstream just accepted the token). Cache hit with a standing grant ⇒ allow, no upstream trip. Cache hit, no grant ⇒ re-verify against the owning registry with the caller's credential — record on success; **clear-on-discovery** (purge the user's grants for the package) + deny on `401`/`403`. TTL is the `installAccelerator.grantTtl` config knob (default: permanent). ## Public vs private (no per-user gating for public packages) A forwarded credential matching a registry doesn't mean a package is *private* — in a mixed proxy (one registry serving a company's private packages **and** public ones), the token matches everything, and gating public content per user would cost a grant row and a re-verify round trip per user for bytes anyone may read. So before the per-user path, a not-yet-classified cache hit is probed **anonymously**: a `2xx` classifies the package public in a global set (no user pays for it again, no grant, no further round trip); a `401`/`403` means it's genuinely private and falls through to the grant / re-verify path above. Public packages thus cost **one anonymous probe across the whole fleet**, not one per user. ## Tests - pnpr: grant-table + public-set mechanics, regime dispatch, the upstream-authorization paths (fresh-fetch, granted cache hit, private re-verify-and-record, denied-clears-grants, public-classified-once-then-free), and forwarded-cred-routes-around-local-policy. - pacquet `pnpr-client`: a test asserting `authHeaders` + `Authorization` travel on the wire. - Full suites green: `pnpr` (237), `pacquet-package-manager` (389), `pacquet-pnpr-client` (12), `pacquet-network`/`config` (325); clippy `-D warnings`, `cargo fmt`, rustdoc `-D warnings --document-private-items`, `typos`, and the TS compile all clean. ## Scoped follow-ups (not in this PR) - Clear-on-discovery fires at the re-verify hook only. A `401`/`403` during the cold resolve aborts the request anyway (nothing is served); threading the offending package out of the deep resolve error to also clear stale grants for *future* requests needs structured auth errors. - Per-scope external registries route via the default registry, since pacquet doesn't yet surface `@scope:registry` routing in `collect_packages`. |
||
|
Zoltan Kochan
|
a358ee09ab |
fix: don't catalog runtime: dependencies under strict catalog mode (#12188)
A `runtime:` specifier (e.g. node from `devEngines.runtime` or `pnpm runtime set`) round-trips to `devEngines.runtime`, which only recognizes the `runtime:` protocol. Under `catalogMode` strict/prefer the auto-save loop promoted it into a catalog and rewrote the manifest entry to `catalog:`, which broke that round-trip and stranded it in `devDependencies`. Skip `runtime:` specifiers in that loop. |
||
|
Zoltan Kochan
|
a017bf3394 |
refactor: rename the agent client and agent setting to pnpr (#12155)
* refactor: rename the agent client + setting to pnpr The pnpm-side client and its config setting still carried the old "agent" name after the server moved to pnpr. Align both with pnpr (and with pacquet, which already uses `pnprServer`): - Move `agent/client` → `pnpr/client` and rename the package `@pnpm/agent.client` → `@pnpm/pnpr.client` (exported `AgentProject` type → `PnprProject`). - Rename the config setting `agent` → `pnprServer` (`--pnpr-server` CLI flag), matching pacquet's setting name. - Rename the internal install-path symbols and the user-facing log / error strings that mentioned "pnpm agent" to "pnpr". No behavioral change — only names. The e2e suite now drives `--config.pnprServer`. * fix: forward optionalDependencies to the pnpr server `PnprProject` and the install-request body only carried `dependencies` and `devDependencies`, so a project's `optionalDependencies` were dropped on the way to the pnpr server — it resolved as if they didn't exist, producing a different lockfile than the local resolver. Thread `optionalDependencies` through the client request shape, the deps-installer single-project and workspace request builders, and the pnpr server (`InstallRequestProject` / `InstallRequest` + the throwaway manifest it writes for resolution). Adds an e2e case asserting an optional dependency is resolved through `pnprServer`. |
||
|
Zoltan Kochan
|
2b788d53fd |
refactor: replace the experimental pnpm-agent server with pnpr (#12151)
The experimental TypeScript `pnpm-agent` install-accelerator server is superseded by the `pnpr` server, which implements the same protocol. Remove `agent/server` and route the agent e2e test through pnpr. The pnpm TypeScript client (`@pnpm/agent.client`) is kept and made compatible with pnpr. The wire protocol carries the on-disk lockfile format, while pnpm keeps an in-memory `LockfileObject` in process: - Incoming: the agent's response lockfile is converted to the in-memory shape via `convertToLockfileObject`. - Outgoing: the existing lockfile is read in its on-disk shape with the new `readWantedLockfileFile` and forwarded as-is — no in-memory round-trip. pnpr now resolves multi-project workspaces by reconstructing the workspace on disk (root manifest + `pnpm-workspace.yaml` + member manifests) and letting pacquet's install path discover every importer. Member dirs are written as quoted YAML scalars; importer dirs are validated against path traversal (rejecting absolute, `..`, backslash, and slashes-only inputs) and de-duplicated; synthetic manifest names map injectively from dirs. The CI test job builds the `pnpr` server from source (cached on the Rust sources) so the agent e2e tests run against the current server. The published `@pnpm/pnpr` is dropped as a test dependency: running the suite already requires building `pnpr-prepare` from source (no npm fallback), so the toolchain to build `pnpr` is always present, and the published binary can predate the server protocol the tests exercise. |
||
|
Sharmila
|
1c73e8303c |
fix(deps-resolver): prefer locked peer contexts during resolution by default (#12083)
## Summary Preserve compatible peer contexts already recorded in the lockfile during a writable re-resolution. A fresh install still resolves peers normally. When a lockfile already records multiple valid peer contexts, pnpm keeps those contexts instead of collapsing them into one compatible context and rewriting unrelated lockfile entries. ## Why [#12075](https://github.com/pnpm/pnpm/pull/12075) fixed optional-peer candidate selection: pnpm no longer discards a compatible optional-peer version merely because it came from the lockfile. This PR addresses a separate source of lockfile churn. A writable install could still replace one valid peer context with another valid peer context even when the existing provider remained present and satisfied the peer range. Public reproduction: <https://github.com/sharmila-oai/pnpm-optional-peer-lockfile-repro> The nested reproduction starts with two valid `vitest@3.2.4` contexts: ```text context-low -> vitest@3.2.4(jsdom@26.1.0) context-high -> vitest@3.2.4(jsdom@27.4.0) ``` Running a writable lockfile regeneration should retain both contexts: ```sh ./reproduce-nested-context.sh ``` ## Behavior pnpm reuses a locked peer provider only when: - The provider is still present in the current dependency graph. - The provider still satisfies the peer range. Current manifest choices remain authoritative. In particular, pnpm does not replace: - A newly added direct peer provider. - An explicitly updated direct peer provider. - A changed nested provider. - A direct provider installed through an alias. The reuse pass runs only when the dependency tree contains locked peer contexts, so fresh installs do not pay for a second peer-resolution pass. ## Tradeoff This change favors lockfile stability over reducing the number of peer contexts. A writable install may retain multiple compatible peer contexts where a fresh install would select one. ## Implementation The resolver performs its normal peer-resolution pass first. When the dependency tree contains locked peer contexts, it performs a second pass that may reuse compatible provider paths from the lockfile while respecting current manifest choices. pacquet now mirrors this behavior. Its lockfile-reuse path rebuilds child dependencies from the package manifest and skips peer dependencies recorded in the snapshot, so the peer pass derives each dependency instance's peer context. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |