mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-28 09:55:39 -04:00
76587d3def6fabf2ec59458a2d00a56069a29e5b
154 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Kochan
|
089484aca8 |
perf(pnpr): resolve server-side and fetch tarballs directly (#12232)
## Summary Reworks pnpr from an install/file accelerator into a resolve-only accelerator: - `POST /v1/resolve` resolves against the client-supplied registries and returns a gzipped JSON lockfile response - pacquet/pnpm clients then fetch tarballs normally from registries with their own credentials and existing parallel fetch/integrity paths - pnpr no longer serves package file bytes or store-index rows, so the server-side file diff, file-frame response, grant table, and public-package byte-gating code are removed The follow-up resolution fast paths are included on the new measured path: - repeated public no-lockfile resolves use a bounded in-memory TTL cache - fresh frozen input lockfiles skip the server-side lockfile-only pacquet resolve after verification proves the lockfile is usable - input lockfile verification and the verdict cache are preserved ## Benchmark Integrated benchmark on Linux shows small improvements in all pnpr rows, with the clearest movement in hot restore. This should be treated as an incremental win rather than a large install-speed change. | Scenario | `pnpr@HEAD` | `pnpr@main` | Change | | --- | ---: | ---: | ---: | | fresh restore, cold cache + cold store | `1.677 s ± 0.090` | `1.686 s ± 0.070` | ~0.6% faster | | fresh restore, hot cache + hot store | `492.5 ms ± 18.1` | `521.9 ms ± 33.4` | ~5.6% faster | | fresh install, cold cache + cold store | `1.997 s ± 0.025` | `2.003 s ± 0.038` | ~0.3% faster | | fresh install, hot cache + hot store | `1.211 s ± 0.024` | `1.236 s ± 0.038` | ~2.0% faster | ## Trade-off Going registry-direct means pnpr no longer gates tarball bytes itself. Private package access is enforced by the upstream registry when the client fetches tarballs. Resolution policy still runs server-side: lockfile verification, release-age policy, trust policy, and resolved package selection continue to happen before the client fetches bytes. |
||
|
Zoltan Kochan
|
4b4d38361c | chore(release): 11.5.2 (#12207) | ||
|
shiminshen
|
e7e99f04e4 |
fix: do not crash when a catalog specifier is a range (#11706)
## Summary `pnpm update --recursive --lockfile-only <pkg>@<version>` crashed with `Invalid Version: <range>` when the catalog entry for `<pkg>` was a range (e.g. `^21.2.10`) and `catalogMode` was `strict` or `prefer`. This is the exact command Renovate's pnpm artifact updater runs; monorepos using `catalog:` with any range specifier were blocked from Renovate-driven lockfile updates. **Root cause:** in `installSome`, the catalog-match short-circuit guards `semver.eq(wantedDep.bareSpecifier, catalogDepSpecifier)` with `semver.validRange` on both sides. `validRange` returns truthy for ranges, but `semver.eq` constructs `new SemVer(...)` internally and throws on a range. **Fix:** use `semver.valid` instead of `semver.validRange` on both sides of the equality guard. Range specifiers now fall through to the existing mismatch handling (`CatalogVersionMismatchError` in `strict` mode, warn-and-use-direct in `prefer` mode) instead of crashing. Behavior for concrete-on-both-sides is unchanged. Closes #11570 ## Behavior after the fix This turns a crash into pnpm's normal catalog-mismatch handling; it does **not** make a strict-mode update succeed when the catalog is a range: - **`catalogMode: strict`** — rejects with `ERR_PNPM_CATALOG_VERSION_MISMATCH` (clean, actionable error instead of a stack trace). - **`catalogMode: prefer`** — warns and uses the direct version. - **concrete-vs-concrete** — unchanged (`semver.eq` still runs). ## pacquet parity The TypeScript fix patches a crash inside pnpm's `catalogMode` mismatch gate — a feature pacquet had not ported at all (`catalog-mode` was in the config parity test's `NOT_PORTED` list). Rather than just the one-liner, this PR ports that gate to pacquet so the two stacks match: - **config:** new `CatalogMode { Manual, Strict, Prefer }` enum (default `manual`), `Config.catalog_mode`, wired through `pnpm-workspace.yaml` (`catalogMode:`) and the env overlay; `catalog-mode` moved from `NOT_PORTED` to a mapped row in the `pnpm_default_parity` contract test. - **package-manager:** `check_catalog_mode` + `CatalogVersionMismatchError` (`ERR_PNPM_CATALOG_VERSION_MISMATCH`), invoked from `update` before the manifest is mutated. The comparison only treats both sides as equal when each parses as a concrete semver version, so a ranged catalog entry falls through to the mismatch path instead of reaching an exact-version comparison — the Rust analogue of the `semver.valid` guard above. The crash itself can't occur in pacquet (Rust's `node-semver` returns a `Result` rather than throwing); the port is the *feature* with the range-correct comparison built in, so pacquet behaves like fixed pnpm. **Not ported** (the surrounding pieces pacquet still lacks, so wiring them would diverge from pnpm rather than match it): the `add`-path cataloging that relies on `defaultCatalog` rewriting, and the `saveCatalogName` → `pnpm-workspace.yaml` auto-cataloging half. The gate is therefore wired into `update <pkg>@<version>` / `--latest` (the Renovate scenario), not `add`. |
||
|
Zoltan Kochan
|
5192edf40e |
feat(pnpr): forward credentials and add per-user access grants for external private registries (#12184) (#12189)
Closes #12184 (part 2). #12181 shipped the per-caller access gate on `POST /v1/install`, which authorizes every served package against pnpr's own `packages:` policy — the complete answer **while pnpr fetches anonymously**. This PR adds the remaining piece: forwarding the caller's per-registry credentials so the accelerator can resolve/fetch **external private** content as the caller, and gating that content per user against the registry that actually owns it. ## Credential forwarding (issue steps 1–2) - **Wire:** `POST /v1/install` gains an `authHeaders` body map (`{ "//host/path/": "Bearer …" }`, the shape `AuthHeaders::from_map` consumes / `getAuthHeadersFromCreds` produces) plus an HTTP `Authorization` header. The body map carries the *upstream* registry tokens; the header identifies the caller to pnpr's own gate and keys the grant table. - **pacquet plumbing:** a request-scoped `Arc<AuthHeaders>` is threaded via a new `Install.auth_override` field and an `auth_override` param on `build_resolution_verifiers`, so resolution/verification run as the caller **without** baking per-user auth into the interned `&'static Config` (which would leak one config per user). - **Server:** `handle_install` builds the per-request `AuthHeaders` and threads it through resolve, verify, and `fetch_uncached` (which now returns the freshly-fetched set). - **Clients:** pacquet `pnpr-client` and `@pnpm/pnpr.client` send `registry` / `namedRegistries` / `authHeaders` + `Authorization`; the TS path sources them from the caller's registry credentials via `@pnpm/network.auth-header` (`getAuthHeadersFromCreds` is newly re-exported). `@pnpm/worker` is unchanged — downloads happen server-side. - **Credential scope:** both clients forward the caller's *full* credential map, not a subset scoped to the declared registries. The registries a dependency graph touches aren't knowable up front — a transitive package can be scope-routed to another registry or pinned to a tarball URL on a host that's in `.npmrc` but isn't a declared registry — so pnpr attaches the right token per fetched URL exactly as a local install does. These are package-fetch credentials going to the very service the caller configured to fetch its packages. ## Per-user grant table (issue steps 3–4) Externally-resolved private content carries no pnpr policy, so the store's possession of the bytes must not authorize a user the upstream never cleared. A served package is dispatched by **whether a forwarded credential was used to fetch it**: - **No forwarded cred → pnpr-as-authority:** the existing local `packages:` policy check, unchanged. - **Forwarded cred → upstream-as-authority:** gated against a persistent `(user, name@version)` grant table (SQLite, modeled on `VerdictCache`). Freshly fetched this request ⇒ record + allow (the upstream just accepted the token). Cache hit with a standing grant ⇒ allow, no upstream trip. Cache hit, no grant ⇒ re-verify against the owning registry with the caller's credential — record on success; **clear-on-discovery** (purge the user's grants for the package) + deny on `401`/`403`. TTL is the `installAccelerator.grantTtl` config knob (default: permanent). ## Public vs private (no per-user gating for public packages) A forwarded credential matching a registry doesn't mean a package is *private* — in a mixed proxy (one registry serving a company's private packages **and** public ones), the token matches everything, and gating public content per user would cost a grant row and a re-verify round trip per user for bytes anyone may read. So before the per-user path, a not-yet-classified cache hit is probed **anonymously**: a `2xx` classifies the package public in a global set (no user pays for it again, no grant, no further round trip); a `401`/`403` means it's genuinely private and falls through to the grant / re-verify path above. Public packages thus cost **one anonymous probe across the whole fleet**, not one per user. ## Tests - pnpr: grant-table + public-set mechanics, regime dispatch, the upstream-authorization paths (fresh-fetch, granted cache hit, private re-verify-and-record, denied-clears-grants, public-classified-once-then-free), and forwarded-cred-routes-around-local-policy. - pacquet `pnpr-client`: a test asserting `authHeaders` + `Authorization` travel on the wire. - Full suites green: `pnpr` (237), `pacquet-package-manager` (389), `pacquet-pnpr-client` (12), `pacquet-network`/`config` (325); clippy `-D warnings`, `cargo fmt`, rustdoc `-D warnings --document-private-items`, `typos`, and the TS compile all clean. ## Scoped follow-ups (not in this PR) - Clear-on-discovery fires at the re-verify hook only. A `401`/`403` during the cold resolve aborts the request anyway (nothing is served); threading the offending package out of the deep resolve error to also clear stale grants for *future* requests needs structured auth errors. - Per-scope external registries route via the default registry, since pacquet doesn't yet surface `@scope:registry` routing in `collect_packages`. |
||
|
Zoltan Kochan
|
a358ee09ab |
fix: don't catalog runtime: dependencies under strict catalog mode (#12188)
A `runtime:` specifier (e.g. node from `devEngines.runtime` or `pnpm runtime set`) round-trips to `devEngines.runtime`, which only recognizes the `runtime:` protocol. Under `catalogMode` strict/prefer the auto-save loop promoted it into a catalog and rewrote the manifest entry to `catalog:`, which broke that round-trip and stranded it in `devDependencies`. Skip `runtime:` specifiers in that loop. |
||
|
Zoltan Kochan
|
a017bf3394 |
refactor: rename the agent client and agent setting to pnpr (#12155)
* refactor: rename the agent client + setting to pnpr The pnpm-side client and its config setting still carried the old "agent" name after the server moved to pnpr. Align both with pnpr (and with pacquet, which already uses `pnprServer`): - Move `agent/client` → `pnpr/client` and rename the package `@pnpm/agent.client` → `@pnpm/pnpr.client` (exported `AgentProject` type → `PnprProject`). - Rename the config setting `agent` → `pnprServer` (`--pnpr-server` CLI flag), matching pacquet's setting name. - Rename the internal install-path symbols and the user-facing log / error strings that mentioned "pnpm agent" to "pnpr". No behavioral change — only names. The e2e suite now drives `--config.pnprServer`. * fix: forward optionalDependencies to the pnpr server `PnprProject` and the install-request body only carried `dependencies` and `devDependencies`, so a project's `optionalDependencies` were dropped on the way to the pnpr server — it resolved as if they didn't exist, producing a different lockfile than the local resolver. Thread `optionalDependencies` through the client request shape, the deps-installer single-project and workspace request builders, and the pnpr server (`InstallRequestProject` / `InstallRequest` + the throwaway manifest it writes for resolution). Adds an e2e case asserting an optional dependency is resolved through `pnprServer`. |
||
|
Zoltan Kochan
|
2b788d53fd |
refactor: replace the experimental pnpm-agent server with pnpr (#12151)
The experimental TypeScript `pnpm-agent` install-accelerator server is superseded by the `pnpr` server, which implements the same protocol. Remove `agent/server` and route the agent e2e test through pnpr. The pnpm TypeScript client (`@pnpm/agent.client`) is kept and made compatible with pnpr. The wire protocol carries the on-disk lockfile format, while pnpm keeps an in-memory `LockfileObject` in process: - Incoming: the agent's response lockfile is converted to the in-memory shape via `convertToLockfileObject`. - Outgoing: the existing lockfile is read in its on-disk shape with the new `readWantedLockfileFile` and forwarded as-is — no in-memory round-trip. pnpr now resolves multi-project workspaces by reconstructing the workspace on disk (root manifest + `pnpm-workspace.yaml` + member manifests) and letting pacquet's install path discover every importer. Member dirs are written as quoted YAML scalars; importer dirs are validated against path traversal (rejecting absolute, `..`, backslash, and slashes-only inputs) and de-duplicated; synthetic manifest names map injectively from dirs. The CI test job builds the `pnpr` server from source (cached on the Rust sources) so the agent e2e tests run against the current server. The published `@pnpm/pnpr` is dropped as a test dependency: running the suite already requires building `pnpr-prepare` from source (no npm fallback), so the toolchain to build `pnpr` is always present, and the published binary can predate the server protocol the tests exercise. |
||
|
Sharmila
|
1c73e8303c |
fix(deps-resolver): prefer locked peer contexts during resolution by default (#12083)
## Summary Preserve compatible peer contexts already recorded in the lockfile during a writable re-resolution. A fresh install still resolves peers normally. When a lockfile already records multiple valid peer contexts, pnpm keeps those contexts instead of collapsing them into one compatible context and rewriting unrelated lockfile entries. ## Why [#12075](https://github.com/pnpm/pnpm/pull/12075) fixed optional-peer candidate selection: pnpm no longer discards a compatible optional-peer version merely because it came from the lockfile. This PR addresses a separate source of lockfile churn. A writable install could still replace one valid peer context with another valid peer context even when the existing provider remained present and satisfied the peer range. Public reproduction: <https://github.com/sharmila-oai/pnpm-optional-peer-lockfile-repro> The nested reproduction starts with two valid `vitest@3.2.4` contexts: ```text context-low -> vitest@3.2.4(jsdom@26.1.0) context-high -> vitest@3.2.4(jsdom@27.4.0) ``` Running a writable lockfile regeneration should retain both contexts: ```sh ./reproduce-nested-context.sh ``` ## Behavior pnpm reuses a locked peer provider only when: - The provider is still present in the current dependency graph. - The provider still satisfies the peer range. Current manifest choices remain authoritative. In particular, pnpm does not replace: - A newly added direct peer provider. - An explicitly updated direct peer provider. - A changed nested provider. - A direct provider installed through an alias. The reuse pass runs only when the dependency tree contains locked peer contexts, so fresh installs do not pay for a second peer-resolution pass. ## Tradeoff This change favors lockfile stability over reducing the number of peer contexts. A writable install may retain multiple compatible peer contexts where a fresh install would select one. ## Implementation The resolver performs its normal peer-resolution pass first. When the dependency tree contains locked peer contexts, it performs a second pass that may reuse compatible provider paths from the lockfile while respecting current manifest choices. pacquet now mirrors this behavior. Its lockfile-reuse path rebuilds child dependencies from the package manifest and skips peer dependencies recorded in the snapshot, so the peer pass derives each dependency instance's peer context. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
f429f93e8b |
feat(pnpr): support --lockfile-only on the pnpr install path (resolve-only mode) (#12148)
* feat(pnpr): support --lockfile-only on the pnpr install path Add a server-side resolve-only mode so `--lockfile-only` (and the `lockfileOnly` setting) is honored when a pnprServer / agent is configured: resolve and write the lockfile, fetch nothing, link nothing. The pnpr server skips the tarball fetch and file diff when the request sets `lockfileOnly`, returning just the lockfile. The pnpr client and the TypeScript agent client forward the flag and ignore any file/index lines an older server still streams, keeping the store untouched. The pacquet CLI no longer errors and stops after writing the lockfile. Closes #12146 * fix(agent.client): observe fileDownloads in lockfile-only path; correct minimumReleaseAge unit doc Address PR review: avoid a possible unhandled rejection if the NDJSON stream errors after the L frame in lockfile-only mode, and fix the `minimumReleaseAge` JSDoc (minutes, not seconds) to match the resolver and the Rust client. |
||
|
Zoltan Kochan
|
6d17b669b4 |
fix: verify lockfile tarball URL matches registry metadata (#12134)
## What The lockfile resolution verifier now confirms that a registry entry pinning an explicit `tarball` URL points at the artifact the registry's own metadata lists for that `name@version`. A mismatch — or any entry that can't be confirmed against the registry — is rejected with `ERR_PNPM_TARBALL_URL_MISMATCH`. ## Why Follow-up to the design discussion on #12122. The verifier checked the age/trust of `name@version` against the registry packument but never bound the lockfile's `tarball` URL to it. For the non-standard entries pnpm preserves a tarball URL for (npm Enterprise, GitHub Packages — see `toLockfileResolution`), pnpm fetches straight from that URL. So a **tampered lockfile could pair a trusted `name@version` with an attacker-chosen tarball URL** (plus a matching integrity for the attacker's bytes); verification passed against the legitimate version while the install fetched the attacker's bytes. Defending a checked-in lockfile is explicitly in this feature's threat model. ## How - For a registry-keyed entry that pins an explicit `tarball`, fetch the packument and assert the URL equals `versions[v].dist.tarball`. The comparison canonicalizes away benign differences — http/https scheme, default ports (`:443`/`:80`), and `%2f` scope-separator encoding (case-insensitive) — so only real mismatches are flagged. The packument is fetched from the user's configured registry (the lockfile's tarball host can't redirect it), and named-registry routing uses the same canonicalization so a scheme/`%2f`-only difference doesn't route to the wrong packument. - **The binding is unconditional.** It runs regardless of `minimumReleaseAge`/`trustPolicy` and is **not** narrowed by their exclude lists, because it guards *integrity*, not *maturity/trust*. Disabling the age/trust policies must not silently disable anti-tamper. (`createNpmResolutionVerifier` therefore always returns a verifier.) - **It is fail-closed.** An entry passes only when the registry metadata affirmatively lists the version with a matching tarball URL. If the metadata can't be fetched, doesn't list the version, or omits `dist.tarball`, the entry is rejected — otherwise a tampered lockfile could smuggle a malicious URL past the check by pointing it at a `name@version` the registry can't vouch for. - **Behavior change:** as a result, an install that re-verifies a lockfile (its content changed since the last verified run, so the verification cache no longer short-circuits) now requires the configured registry to be reachable. `trustLockfile` is the opt-out for environments that treat the on-disk lockfile as already trusted. - **Verification cache.** The policy snapshot records a `tarballUrlBinding` marker and `canTrustPastCheck` requires it, so a cache record written before this rule existed is re-verified rather than trusted (closing an upgrade-time bypass). - Entries with no explicit `tarball` reconstruct the URL from name+version+registry and are inherently bound (no check). `file:`/git-hosted resolutions stay out of scope (#12122). - Threads `nonSemverVersion` to the verifier so URL-keyed tarball deps (a remote `https:` tarball that carries a semver `version` copied from its manifest) are recognized as deliberate non-registry deps and skipped — also fixing a latent release-age over-match on them. The candidate dedupe key includes `nonSemverVersion` so a registry snapshot and a URL-keyed snapshot sharing a `name@version` and serialized resolution stay distinct. Mirrored in pacquet (`create_npm_resolution_verifier`). The dedupe-key change is TS-only: pacquet's candidate `version` comes from the lockfile key suffix, so the two shapes never share a key there. ## Tests - TS: confirmed mismatch → violation; non-standard URL matching metadata → pass; default-port/scheme difference → pass; URL-keyed dep → skipped; URL binding runs (and fails closed) with no age/trust policy configured; `canTrustPastCheck` rejects a cache record lacking the binding marker. Regression-verified (the mismatch test fails when the check is disabled). - pacquet: mirror tests + the no-policy / `minimumReleaseAge: 0` / `trustPolicy: off` cases, default-port/scheme equivalence, and the missing-`tarballUrlBinding` cache rejection. A few install-dispatch / resolution-reuse tests that pin a deliberately bogus tarball URL (or run against an unreachable registry to prove resolution reuse) now set `trustLockfile`, since the always-on fail-closed tarball-URL check would otherwise flag the fixture before the path under test runs. - `clippy --deny warnings`, `fmt`, and `dylint` clean. |
||
|
Zoltan Kochan
|
0f509d055f | chore(release): 11.5.1 (#12126) | ||
|
Sharmila
|
122ab0a1ed |
fix(deps-resolver): preserve locked optional peer candidates (#12075)
## Summary Preserve compatible optional-peer versions already recorded in the lockfile when pnpm re-resolves a workspace. ## Reproduction Public reproduction: <https://github.com/sharmila-oai/pnpm-optional-peer-lockfile-repro> The workspace contains: ```text packages/uses-vitest -> vitest@3.2.4 packages/older -> jsdom@26.1.0 ``` Vitest declares `jsdom` as an optional peer dependency. The committed lockfile was generated when another workspace package also depended on `jsdom@27.4.0`. At that time, pnpm selected the higher compatible version for Vitest: ```text vitest@3.2.4(jsdom@27.4.0) ``` That additional direct dependency was then removed from its `package.json`, without regenerating the lockfile. This simulates a normal manifest edit. Running: ```sh pnpm install --lockfile-only --no-frozen-lockfile ``` unnecessarily rewrites Vitest's still-valid optional-peer context: ```diff - version: 3.2.4(jsdom@27.4.0) + version: 3.2.4(jsdom@26.1.0) ``` Both versions satisfy Vitest's optional peer range. The existing `27.4.0` resolution remains valid and should not be discarded while pnpm updates the lockfile. ## Cause Preferred versions loaded from the wanted lockfile are stored as weighted selectors: ```ts { selectorType: 'version', weight: EXISTING_VERSION_SELECTOR_WEIGHT } ``` `getHoistableOptionalPeers()` only recognized the plain string form: ```ts specType === 'version' ``` As a result, it ignored the compatible locked `27.4.0` candidate. It only saw `26.1.0`, which was rediscovered from `packages/older/package.json`, and rewrote the peer context. ## Fix Normalize the selector before checking its type, matching the handling already used by `hoistPeers()` for required peers: ```ts const specType = typeof selector === 'string' ? selector : selector.selectorType ``` This restores lockfile-seeded versions to the candidate set. It does not add a new preference rule or force pnpm to keep every locked version. Optional-peer auto-installation continues to choose the highest version satisfying every recorded peer range. The equivalent fix is included in pacquet, pnpm's Rust port. ## Validation - Added matching TypeScript and Rust regression tests. - Verified the public reproduction against `pnpm@11.4.0` and the patched CLI. - Ran the focused TypeScript resolver checks and pacquet test, clippy, format, and `cargo nextest` checks. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Adam Lyrén
|
6f382f42ee |
fix: preserve integrity of remote tarball dependencies on re-resolution (#12096)
* fix: preserve integrity of remote tarball dependencies on re-resolution Re-resolving a remote tarball dependency without re-fetching it (e.g. `pnpm update`) produced a resolution with no integrity, so the previously recorded integrity was dropped from the lockfile, breaking later installs with ERR_PNPM_MISSING_TARBALL_INTEGRITY. Carry the integrity over from the previous lockfile entry when the rebuilt tarball resolution lost it and the URL is unchanged. This complements #12040, which fixes the same class of bug in the package-requester layer but does not cover this re-resolution path. Closes #12067. * test: cover integrity carryover on tarball re-resolution * refactor: check integrity before type in the tarball carryover guard * perf(pacquet): reuse the warm-store tarball on re-resolution instead of re-downloading --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
morning-verlu
|
118e9be809 |
fix: set user agent in headless lifecycle scripts (#12092)
Co-authored-by: morning-verlu <258725120+morning-verlu@users.noreply.github.com> |
||
|
Zoltan Kochan
|
1db05c6fca |
fix: inconsistent resolution of a peer shared through a diamond (#12081)
* fix: inconsistent resolution of a peer shared through a diamond When a package peer-depends both another package and one of that package's own peer dependencies (e.g. @typescript-eslint/eslint-plugin peer-depends both @typescript-eslint/parser and typescript, and @typescript-eslint/parser peer-depends typescript), pnpm reused a hoisted instance of the shared peer that was resolved against a different version, producing an inconsistent resolution. Close #12079 * test: cover the peer-diamond resolution in pnpm and pacquet Add @pnpm.e2e/peer-diamond-* fixtures modeling #12079 (plugin peer-depends both parser and ts; parser peer-depends ts) and integration tests on both stacks. The pnpm test guards the fix; the pacquet test confirms pacquet already resolves the diamond consistently (its merge always prefers the node's own child). * docs: fix grammar in changeset (peer-depends on both) |
||
|
Zoltan Kochan
|
b741d91e67 | chore(release): 11.5.0 (#12068) | ||
|
Marvin Hagemeister
|
49e6074644 |
test: replace @pnpm/registry-mock with an in-repo in-process registry (#11927)
Replace the external `@pnpm/registry-mock` (Verdaccio) test dependency with an in-repo, in-process registry that serves package fixtures to **both** the pacquet Rust tests and the pnpm CLI (Jest) tests. No separately managed registry process is needed. ### How it works - **Fixtures** live at `registry/.fixtures/packages/<name>/<version>/…`, moved verbatim from [`pnpm/registry-mock`](https://github.com/pnpm/registry-mock) (keyed by each `package.json`'s `name`+`version`). - **`pnpm-registry-fixtures`** builds verdaccio-shaped storage from those fixtures; the in-tree **`pnpm-registry`** crate serves it. - Files whose names differ only by case (`@pnpm.e2e/with-same-file-in-different-cases`) and `bundleDependencies` trees are composed **in memory** by the builder, since neither can be committed to the working tree. - **pacquet**: `pacquet-testing-utils`' `TestRegistry` starts the server lazily (once per process) in proxy mode, serving `@pnpm.e2e` fixtures locally and falling through to the npm uplink for real packages (`is-positive`, `is-negative`, …) — matching how registry-mock behaved. - **pnpm CLI**: the `with-registry` Jest `globalSetup` builds storage from the fixtures via the new `pnpm-registry-prepare` binary (built from source in the Test CI job) and serves it with `pnpm-registry`. `REGISTRY_MOCK_PORT` / `REGISTRY_MOCK_CREDENTIALS` / `getIntegrity` now come from `@pnpm/testing.registry-mock`. ### Result `@pnpm/registry-mock` is removed from every manifest, the catalog, and `packageExtensions`; `cargo test` / `cargo nextest run` / `just test` and the pnpm CLI Jest suites all run registry-backed tests without launching Verdaccio. |
||
|
Ethan Setnik
|
3cf2b86579 |
fix: preserve tarball dependency integrity in the lockfile (#12040)
* fix: preserve tarball dependency integrity in the lockfile URL/tarball resolvers do not return an integrity (it is only known after the tarball is downloaded). When a remote-tarball dependency was reused from the lockfile without being re-fetched, the freshly resolved resolution had no integrity and the existing one was dropped, breaking subsequent --frozen-lockfile installs under the lockfile-integrity hardening (ERR_PNPM_MISSING_TARBALL_INTEGRITY). Carry the integrity over from the current resolution instead. Closes #12001 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(package-requester): simplify tarball integrity carryover guard Align the integrity carryover added in the previous commit with its sibling block in the download path: use `!resolution.type` (the idiom already used there) and drop the `newIntegrity == null` clause, which is redundant once `resolution` is the freshly resolved resolution. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(pacquet): cover tarball-dependency integrity preservation (#12001) pnpm #12040 fixes dropping a remote-tarball dependency's integrity when an unrelated package is installed afterwards. Pacquet can't reach that scenario yet: a non-registry https-tarball direct dependency hits the TarballResolver, which returns no name_ver/integrity, so lockfile build panics with MissingSuffix. Add the regression test for the target behavior, gated with allow_known_failure! until external tarball deps install. Tracked in #12053. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
a39a83d19e |
feat: support nodeLinker: hoisted on fresh installs + add hoistingLimits setting (#12041)
## 1. Support `nodeLinker: hoisted` on the fresh-lockfile install path (pacquet) Closes #11871. Until now pacquet's `Install::run` hard-refused `nodeLinker: hoisted` without a checked-in lockfile (`ERR_PNPM_…UNSUPPORTED_FRESH_INSTALL_NODE_LINKER`). - Extracted a shared `run_hoisted_linker` helper from the frozen path's hoisted branch (walker → `link_hoisted_modules` → `SymlinkDirectDependencies { link_only: true }` → `pkg_root_by_key` → walker-skip folding), so both install paths run identical logic. - Fresh path now threads `node_linker` + `supported_architectures`, hands `CreateVirtualStore` the real linker (populating `cas_paths_by_pkg_id`), branches on `is_hoisted`, and returns `hoisted_locations` so `.modules.yaml` round-trips. - Removed the guard and the dead `UnsupportedFreshInstallNodeLinker` error variant. Ported upstream's `hoistedNodeLinker/install.ts` into `crates/cli/tests/hoisted_node_linker.rs` (real tests for the core layout, no-lockfile, `externalDependencies`, `autoInstallPeers`, and `hoistingLimits`; the rest stubbed as `known_failures` against `pnpm add`/update (#433) and build-phase (#11870) gaps), and ticked the boxes in `plans/TEST_PORTING.md`. ## 2. Add the `hoistingLimits` setting (pnpm CLI **and** pacquet) Revives the stale #6468 (closes #6457) and brings both stacks to parity. `hoistingLimits` mirrors yarn's `nmHoistingLimits`: `none` (default — hoist as far as possible), `workspaces` (hoist only as far as each workspace package), `dependencies` (hoist only up to each workspace package's direct deps). It was previously a programmatic-only option in pnpm (no config surface) and a pacquet-only raw-map yaml field. **pnpm CLI:** `config/reader` (`types.ts` enum + `Config.ts` + `configFileKey.ts`), `installing/linking/real-hoist`'s new `getHoistingLimits` (mode → the `@yarnpkg/nm` hoister's per-locator border map), and the install/add/recursive command option lists. Tests: `hoistedNodeLinker/install.ts` (`dependencies` mode) + `real-hoist` `getHoistingLimits` unit tests. Changeset included (minor). **pacquet:** replaced the raw-map config with the same enum; added `get_hoisting_limits` (port of `getHoistingLimits`); and **fixed `real-hoist`'s border semantics** — a name in the limits marks a *border* whose descendants stay nested beneath it, not a leaf to block. (The earlier leaf-blocking behavior was the divergence flagged while porting; its unit tests were rewritten to the corrected semantics.) |
||
|
Abdullah Alaqeel
|
2cadfb5d3d |
refactor: replace enquirer with @inquirer/prompts (#11942)
Replaces the unmaintained `enquirer` package with `@inquirer/prompts` for all interactive CLI prompts. Fixes the `update -i` scrolling overflow bug where long choice lists were clipped in the terminal. Fixes #6643 ## User-facing changes - **`pnpm update -i` / `pnpm update -i --latest`**: Scrolling now works correctly when many packages are available; the new library uses visual-line-aware pagination via `usePagination` - **`pnpm audit --fix -i`**: Same scrolling fix for vulnerability selection - **`pnpm approve-builds`**: Interactive build approval prompts updated - **`pnpm patch`**: Version selection and "apply to all" prompts updated - **`pnpm patch-remove`**: Patch removal selection updated - **`pnpm publish`**: Branch confirmation prompt updated - **`pnpm login`**: Credential prompts updated - **`pnpm run` / `pnpm exec`** (with `verifyDepsBeforeRun=prompt`): Confirmation prompt updated ## Internal changes - `OtpEnquirer` DI interface changed from `{ prompt }` to `{ input }` - `LoginEnquirer` DI interface changed from `{ prompt }` to `{ input, password }` - `enquirer` removed from catalog and all 8 package.json files - `@inquirer/prompts` v8.4.3 added to catalog and all 8 package.json files - Removed `OtpPromptOptions` and `OtpPromptResponse` exports from `@pnpm/network.web-auth` (no longer needed) --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
39101f5e37 |
fix: hang on cyclic aliased peer dependency (#12018)
- `pnpm i nuxt@npm:nuxt-nightly@5x` (and similar aliased installs) hung at 0% CPU during peer resolution after `resolved N, reused 0, downloaded N, added 0`. - `resolvePeers.calculateDepPath` only short-circuited cycles whose members included `currentAlias`. When two peers form a mutual cycle (e.g. `vite` ↔ `@vitejs/devtools`) and both hit the `findHit` cache instead of running their own `calculateDepPath`, the cycle surfaced at a level where no participant could break it — a sibling's `calculateDepPath` saw the cycle in the `cycles` argument but kept awaiting `pathsByNodeIdPromises` on cyclic peer node IDs. - The fix expands `cyclicPeerAliases` to also include any cycle that intersects the current call's pending peers, so awaiting siblings emit the `name@version` peer id and the cached promise gets released. - Pacquet's `resolve_peers` walks synchronously with an `in_progress` set and returns already-realized `DepPath` values from `find_hit`, so the deadlock does not occur there. A pacquet regression test locks in that the aliased-install + transitive-mutual-peer scenario terminates with the expected graph entries. Closes #11999. |
||
|
Zoltan Kochan
|
a33c4bfcb0 |
perf: skip resolution when only pnpm-lock.yaml is missing (pnpm + pacquet) (#12004)
* perf: skip resolution when only pnpm-lock.yaml is missing
When pnpm-lock.yaml is absent but node_modules/.pnpm/lock.yaml exists and still
satisfies the manifest, reuse the materialized snapshot to regenerate the
wanted lockfile instead of walking the registry to rebuild it. Closes the
cache+node_modules variation gap in the vlt.sh benchmarks for the pnpm CLI
side; the pacquet port is tracked separately at #11993.
`--frozen-lockfile` still fails when pnpm-lock.yaml is absent: the regenerated
file must be committed, so failing loudly is the correct behavior for CI.
* perf(pacquet): port the cache+node_modules shortcut
When `pnpm-lock.yaml` is absent but `node_modules/.pnpm/lock.yaml` exists
and still satisfies the manifest, synthesize the wanted lockfile from the
materialized snapshot and take the frozen-install path. The install skips
resolution and regenerates `pnpm-lock.yaml` from the synthesized object.
Mirrors the pnpm-side change at
|
||
|
Zoltan Kochan
|
72d997cc34 | chore(release): 11.4.0 (#11989) | ||
|
Zoltan Kochan
|
aa6149df65 |
fix: fail by default when a tarball does not match the locked integrity (#11968)
`pnpm install` (non-frozen) used to react to `ERR_PNPM_TARBALL_INTEGRITY` by logging the error, silently re-resolving from the registry, and overwriting the locked integrity. The lockfile's integrity was effectively advisory by default — a compromised registry, proxy, or republished version could substitute attacker-controlled content on a clean machine even though the project shipped a committed `pnpm-lock.yaml`. Integrity mismatches against the lockfile now fail by default. The **only** opt-in is **`pnpm install --update-checksums`** — a new flag, narrowly scoped to refreshing the locked integrity values. Mirrors yarn's flag of the same name. A warning still prints when the bypass takes effect so the rewrite stays auditable. `--force` and `pnpm update` deliberately do **not** bypass the integrity check. They are routine refresh operations; silently overwriting a locked integrity in those flows would erase the protection a committed lockfile is supposed to provide. `--frozen-lockfile` behavior is unchanged. `--fix-lockfile` keeps its documented purpose (filling in missing lockfile entries) and is also not a bypass. Combining `--frozen-lockfile` with `--update-checksums` errors out — frozen mode refuses to rewrite the lockfile, which is exactly what `--update-checksums` is for. `--update-checksums` also bypasses the resolver's on-disk metadata cache fast path (`pickPackage.ts:271`, `pick_package.rs:531`). Without that, a stale on-disk packument that already contained the pinned version would short-circuit the registry entirely and the flag would silently no-op on dev machines. With the gate, every first-encounter goes through a conditional GET; the in-memory cache is left alone so second-and-onward references within the same install still hit cached fresh data (one network round-trip per *unique* package, not per reference). ## Reported by Reported privately via the security channel. The reproduction: 1. Publish `example-package@1.0.0` with content `v1` and install with pnpm; lockfile records the `v1` integrity. 2. Replace the registry's tarball+metadata for the same `1.0.0` with content `v2`. 3. On a clean store/cache, run `pnpm install`. Before this fix, pnpm logged `ERR_PNPM_TARBALL_INTEGRITY` but exited 0 with `v2` installed and the lockfile rewritten to the new integrity. After this fix, the same install exits non-zero. ## Prior art - **npm** ([sebhastian](https://sebhastian.com/npm-err-code-eintegrity/)): hard-fails with `EINTEGRITY`. No dedicated override flag — recovery is `npm cache clean --force`, manually editing the lockfile, or deleting it. - **yarn** ([Sean C Davis](https://www.seancdavis.com/posts/fix-yarn-integrity-check-failed/)): hard-fails with "Integrity check failed". Has a dedicated **`yarn install --update-checksums`** flag — pnpm now adopts the same name. ## Pacquet parity Pacquet was already fail-hard on integrity mismatch by default (no auto-repair path to remove). This PR brings the rest of the surface into line so `pnpm install --update-checksums` keeps working when pacquet is the materialization target, and `pacquet install --update-checksums` behaves identically standalone: - New `--update-checksums` flag on `pacquet install` (`crates/cli/src/cli_args/install.rs`), plumbed through `Install` and `InstallWithFreshLockfile` into the resolver. - When the flag is set, pacquet skips the frozen-lockfile fast path and routes through the fresh-resolve path so locked integrity values get rewritten from the registry. - `--frozen-lockfile + --update-checksums` errors with `pacquet_package_manager::frozen_lockfile_with_outdated_lockfile`, mirroring pnpm's `ERR_PNPM_FROZEN_LOCKFILE_WITH_OUTDATED_LOCKFILE`. - `pacquet_tarball::verify_checksum_error` now carries a help hint pointing at `--update-checksums` and calling out the supply-chain implication, matching the updated pnpm `TarballIntegrityError`. - The disk fast-path gate is mirrored in `crates/resolving-npm-resolver/src/pick_package.rs:531`, with the flag threaded from `ResolveOptions` → `PickPackageOptions`. |
||
|
Zoltan Kochan
|
ad84fffd46 |
fix: reject path-traversal segments in dependency aliases (#11954)
* fix: reject path-traversal segments in dependency aliases A transitive registry package can use a dependency-alias key like `@x/../../../../../.git/hooks` to make `pnpm install` create a symlink outside the intended `node_modules` directory, since pnpm passes the alias straight into `path.join(modulesDir, alias)` without checking that the joined path stays inside `modulesDir`. Reject aliases that aren't a single `name` or `@scope/name` shape at manifest-read time (both the importer's manifest and every transitive package manifest) and re-check at the symlink layer as defense in depth. Mirror the fix in pacquet's deps-resolver. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(pacquet): use raw strings in alias validator tests for dylint Perfectionist's `prefer-raw-string` lint rejects the two backslash-escaped test inputs. --- Written by an agent (Claude Code, claude-opus-4-7). * refactor: tighten dependency-alias validator to validate-npm-package-name An alias is the directory name pnpm creates inside `node_modules`, so the only valid shapes are a single `name` or `@scope/name` consisting of URL-friendly characters with no leading `.` / `_`, and not equal to reserved names such as `node_modules`. That's the same `validForOldPackages` rule `parseWantedDependency` already applies to CLI-given names — the manifest-read path should match. Route both stacks through it so `.bin`, `.pnpm`, `node_modules`, `favicon.ico`, whitespace, and non-URL-friendly characters are all rejected alongside the path-traversal shapes the narrow validator caught. --- Written by an agent (Claude Code, claude-opus-4-7). * refactor: collapse symlink-layer assertion + path.join into safeJoinModulesDir The two-step pattern of "assert the alias stays in the dir" then "join the dir and the alias" left it possible for a caller to use the join without the assertion. Fold them into a single `safeJoinModulesDir` that returns the joined path and throws on escape, so the check is unmissable. --- Written by an agent (Claude Code, claude-opus-4-7). * test(symlink-dependency): cover the path-equals-dir guard branch The earlier tests only exercised the `!startsWith` branch with `'../sibling'` and `'@x/../../../etc'`. Add `''` and `'.'` as alias cases — both resolve to the modules dir itself and hit the `resolvedLink === resolvedDir` branch of `safeJoinModulesDir`. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
a23956e3ab |
fix(config/reader): pin unscoped per-registry settings to their source's registry at load time (#11953)
* fix(config/reader): drop user-level default auth when workspace overrides registry
When a workspace `.npmrc` overrides `registry=` to a different value than the
user's `~/.npmrc` or `~/.config/pnpm/auth.ini` would have set, do not bind
unscoped/default credentials (`_authToken`, `_auth`, `username`/`_password`)
from the user-level config to the workspace-selected registry. The previous
behavior leaked user-trusted credentials to whatever registry an untrusted
workspace `.npmrc` pointed at. Reported by JUNYI LIU.
* chore(cspell): allow JUNYI in changeset and tests
* fix(config/reader): also defend when pnpm-workspace.yaml overrides registry
Move the rebind defense to after all config layers (CLI, env vars,
pnpm-workspace.yaml, .npmrc) have settled. Compare the final resolved
default registry against what the user-level config alone would produce,
and skip the check entirely if the user requested a registry via CLI/env
themselves.
* feat(config/reader): deprecate unscoped authentication credentials
Emit a per-file warning whenever an .npmrc or auth.ini contains an
unscoped auth value (_authToken, _auth, username, _password,
tokenHelper). URL-scoped tokens have been npm's recommended pattern
since npm@9, and unscoped credentials are slated for removal in a
future major. The warning fires independently of whether the rebind
defense rejects the credentials, so users see the deprecation even when
their setup happens to be safe today.
* refactor(config/reader): rescope unscoped credentials at load time instead of detecting rebinds post-merge
Each .npmrc / auth.ini / CLI source's unscoped credential keys
(_authToken, _auth, username, _password, tokenHelper) are rewritten to
their URL-scoped equivalent during load, using the same source's
registry= value (or the npmjs default if it declares none). A later
layer overriding registry= can no longer rebind a credential to its own
registry — the credential is already pinned to the URL its author
intended.
This removes the post-merge source-tracking defense and replaces it
with the simpler per-source normalization. Each rescope emits a
deprecation warning so users migrate to writing the URL-scoped form
directly.
* refactor(network/auth-header): drop empty-string default-registry slot
After load-time rescoping, no source can populate configByUri[''] —
every credential is either URL-scoped from the start or rewritten to
the URL-scoped form during the .npmrc / auth.ini / CLI parse. The
runtime fallback that re-keyed configByUri[''] onto the merged default
registry, and the publish-side fallback that read it, are both dead
code.
Removed:
- empty-string handling in getAuthHeadersFromCreds, including its
defaultRegistry parameter
- defaultRegistry parameter from createGetAuthHeaderByURI
- the corresponding dedicated unit test
- the configByUri['']?.creds fallback in publishPackedPkg.ts
- empty-key assertions in config/reader tests
Updated all ~16 call sites of createGetAuthHeaderByURI to drop the now
unused second argument.
* feat(config/reader): extend per-source rescoping to client TLS cert/key
The same trust-boundary issue that affected unscoped credentials applies
to client TLS settings: an unscoped cert=/key= would be presented to
whatever registry the merged config settles on, even if a later layer
(workspace .npmrc, pnpm-workspace.yaml, CLI flag) overrode it. The
existing rescope helper now also rewrites unscoped `cert` and `key`
to their URL-scoped form, pinning them to the registry their author
named in the same source.
`ca`/`cafile` are intentionally left unscoped: they're trust anchors,
not credentials, and corporate MITM-proxy setups depend on them
applying to every HTTPS request. The default-registry override can't
weaponize an unscoped CA — the attacker would need a cert signed by it.
`certfile`/`keyfile` (file-path variants) are not rescoped either:
`certfile` isn't read unscoped by pnpm today (asymmetric vs. `keyfile`
in NPM_AUTH_SETTINGS), and supporting only one of them would be
confusing. Users wanting the path form can write it URL-scoped
directly.
* chore(config/reader): remove dead unscoped `keyfile` allowlist entry
`keyfile` was listed in NPM_AUTH_SETTINGS so unscoped `keyfile=<path>`
passed the .npmrc filter and ended up in authConfig — but nothing in
the codebase ever read it from there. The dispatcher uses `opts.key`
(inline PEM) and `configByUri[host].tls.key` (URL-scoped path/inline
content), neither of which is populated from unscoped `keyfile=`.
`certfile` was already absent from the allowlist for the same reason,
so this also removes the asymmetry between the two file-path variants.
URL-scoped `//host/:certfile=...` and `//host/:keyfile=...` continue
to work via `tryParseSslKey` and are unaffected.
* test(network/auth-header): drop test for removed default-registry slot
This test exercised the configByUri[''] re-keying path that was
removed in the rescope-at-load refactor. With createGetAuthHeaderByURI
no longer accepting a defaultRegistry parameter and unscoped
credentials no longer reaching the merged config, the scenario the
test described is structurally unreachable.
* fix(config/reader): handle empty/invalid registry value in rescope
Two CI fixes:
1. When a source's `registry=` resolves to an empty string (e.g. an
unresolved `${ENV_VAR}` placeholder), `new URL(...)` inside
`nerfDart` throws. Guard the call with try/catch: drop the
unscoped per-registry keys (a bare token has nowhere safe to bind)
and emit a warning naming the offending source.
2. Update `.npmrc does not load pnpm settings` to expect the rescoped
form of unscoped `_authToken`/`username` in `authConfig` — they
now appear as `//registry.npmjs.org/:_authToken` etc. since the
test's .npmrc declares no `registry=` of its own.
* chore(cspell): allow "rescoping"
* test(installing/deps-installer): drop "legacy way" auth test
This test passed credentials via the configByUri[''] empty-string slot,
which the auth-header layer re-keyed to the merged default registry at
request time. That slot was removed in the rescope-at-load refactor —
credentials are now always URL-scoped before they reach configByUri,
so the empty-key entry is unreachable from any code path.
The scenario the test covered (basicAuth via username/password) is
already exercised by the existing "installing a package that need
authentication, using password" test using the URL-scoped form.
|
||
|
Zoltan Kochan
|
ae2175829a |
feat(registry-access): extract dist-tag + adduser helpers, dogfood from tests (#11926)
* feat(registry-access): extract setDistTag and dogfood from tests
Add `@pnpm/registry-access.commands#setDistTag` — the low-level PUT to
`/-/package/:pkg/dist-tags/:tag`. The CLI `dist-tag add` handler now
calls it instead of issuing the fetch inline.
Tests in this monorepo now use a thin new package
`@pnpm/testing.registry-mock` (REGISTRY_MOCK_PORT + REGISTRY_MOCK_CREDENTIALS
baked in) that delegates to `setDistTag`, replacing `addDistTag` from
`@pnpm/registry-mock`. That dropped helper relied on
`anonymous-npm-registry-client` and a verdaccio-era
fetch-then-DELETE-then-PUT dance that is no longer needed against
pnpm-registry.
39 test files swapped from `@pnpm/registry-mock` to
`@pnpm/testing.registry-mock`.
* fix: move setDistTag to its own package to break tsconfig project-reference cycle
testing/registry-mock → registry-access.commands → releasing/commands
→ installing/commands → installing/deps-installer → testing/registry-mock.
Extract setDistTag into @pnpm/registry-access.set-dist-tag (only depends
on @pnpm/error, @pnpm/network.fetch, @pnpm/npm-package-arg). Both
@pnpm/registry-access.commands and @pnpm/testing.registry-mock import
from it. Cycle gone.
* feat(registry-access): extract addUser helper, dogfood from login + tests
Add @pnpm/registry-access.add-user — a small helper that PUTs to
/-/user/org.couchdb.user:<name> and returns { token }. The CLI's
classicLogin (pnpm login fallback path) now calls it, and tests
use it via @pnpm/testing.registry-mock instead of the legacy
addUser from @pnpm/registry-mock.
Swapped 3 call sites: globalSetup.js, installing/deps-installer's
auth.ts, and pnpm/test/dlx.ts. AddUserHttpError exposes status +
text + parsed-json-if-applicable + headers so the CLI can still
do its OTP detection. One webauth-OTP login test mock had to be
adjusted to provide its body via `text` (JSON-stringified) rather
than `json` only, since the helper consumes the body via `text()`.
* refactor: consolidate set-dist-tag + add-user helpers into one @pnpm/registry-access.client package
One shared package is better than splitting per endpoint. Future endpoints
(publish, deprecate, etc.) can land here without another wrapper.
No behavioral change — same setDistTag and addUser exports as before,
just under one roof. Callers updated: registry-access.commands,
auth.commands, testing.registry-mock.
* fix(registry-access): sort imports
|
||
|
shiminshen
|
572842a039 |
fix(installing.commands): clarify "loose mode" wording in minimumReleaseAge log (#11763)
* fix(installing.commands): clarify "loose mode" wording in minimumReleaseAge log The log line printed when pnpm auto-adds entries to `minimumReleaseAgeExclude` referred to internal "loose mode" terminology, which doesn't appear in the docs and isn't discoverable. Point users at the actual setting name they need to flip. Closes #11747 * Update installing/commands/src/policyHandlers.ts Co-authored-by: Zoltan Kochan <z@kochan.io> * fix(installing.commands): name the value in minimumReleaseAgeStrict log hint Change "set minimumReleaseAgeStrict to gate these updates with a prompt" to "set minimumReleaseAgeStrict to true to ..." so the value is explicit. --------- Co-authored-by: shiminshen <16914659+shiminshen@users.noreply.github.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
f2a4d2caef | chore(release): 11.3.0 (#11894) | ||
|
Zoltan Kochan
|
155af87585 |
fix(env-installer): prune env lockfile when updating a config dep (#11892)
`pnpm add --config <pkg>` (via `resolveConfigDeps`) wrote the env lockfile without pruning, so optional subdependencies from the previously resolved version remained as orphans. Mirror the prune call from `resolveAndInstallConfigDeps`. |
||
|
Zoltan Kochan
|
e0bd879dea |
fix(deps-resolver): restore index-based pairing so git/tarball deps aren't dropped (#11890)
PR #11711 switched updateProjectManifest and the catalog-update loop in resolveDependencies to look up wantedDependencies by alias, but parseWantedDependency returns `{ alias: undefined, bareSpecifier }` for inputs like `pnpm/foo#sha` or tarball URLs whose alias is only known after fetching the package's package.json. Those entries collided under the `undefined` Map key, so the alias-keyed lookup of the resolved dep returned undefined, the filter dropped them from specsToUpsert, and they silently disappeared from the manifest update and pendingBuilds. This restored the index-based pairing the code used before #11711. catalog: preservation isn't affected: it's driven by rdd.catalogLookup.userSpecifiedBareSpecifier in the spec object, not by how wantedDep is looked up. The premise in the removed comment ("linked deps like workspace:* are excluded from directDependencies") was also wrong — linked deps stay in directDependencies with isLinkedDependency: true, they're not dropped. Restores building/commands/test/build/index.ts: rebuilds dependencies, rebuilds specific dependencies, rebuild with pending option. |
||
|
Totoro
|
ae42a7adc1 |
fix: preserve catalog: protocol references on upgrade (#11711)
* fix: preserve catalog: protocol references on upgrade (issue #11658) * refactor: address review feedback on catalog: preservation fix - Fix typo in 3 test assertions (`@pnpm.e2e.foo` → `@pnpm.e2e/foo`) that made `.toBeFalsy()` pass vacuously - Use `Map` for alias→wantedDependency lookup in `updateProjectManifest` to match the pattern in `index.ts` --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
212315de16 |
fix: cap lockfile verification memory and add trustLockfile opt-out (#11878)
* fix: cap lockfile verification memory and add trustLockfile opt-out Verifying a multi-thousand-entry lockfile against `minimumReleaseAge` or `trustPolicy: no-downgrade` retained every fetched packument in a per-install cache for the entire install. On large workspaces this OOM'd CI runners with a 2GB heap cap. Project both caches down to just the fields each check reads (per-version trust evidence + the `time` map for trust; package-level `modified` + version-name set for the abbreviated shortcut) so the bulk packument is GC'd as soon as the fetch returns. Also adds a `trustLockfile` setting (default `false`) that skips the verification pass entirely for environments where the lockfile is already part of the trusted base. Mirrored in pacquet. Closes #11860. * perf: share resolver packument cache with the lockfile verifier The verifier kept its own per-install dedup Maps and re-fetched every packument the resolver had already pulled during the same install. Plumb the resolver's per-install `PackageMetaCache` through to the verifier (via `createNpmResolutionVerifier` / `build_resolution_verifiers`) so a name already in the resolver's LRU short-circuits the verifier's disk/network round-trip — fast path only, the cached document is projected for the trust check so the verifier's memory footprint stays bounded. In pnpm, `installing/client` now constructs one LRU and hands it to both `createResolver` and `createResolutionVerifiers`. In pacquet, the `InMemoryPackageMetaCache` is lifted to `Install::dispatch` and passed to both `build_resolution_verifiers` and `InstallWithFreshLockfile`. |
||
|
David Barratt
|
3422cecfd3 |
fix(installing.deps-resolver): deterministically order cyclic peer suffixes (#11826)
* fix(installing.deps-resolver): deterministically order cyclic peer suffixes (#8155)
`resolveDependencies` was pushing onto `pkgAddresses`, `postponedResolutionsQueue`,
and `postponedPeersResolutionQueue` from inside `Promise.all`-spawned callbacks,
so the order of items in those arrays reflected completion timing rather than
the order of `extendedWantedDeps`. That ordering then flowed downstream into
`resolvePeers` and the cyclic-peer suffix assignment, so two packages with
transitive peer dependencies on each other (e.g. `@aws-sdk/client-sts` and
`@aws-sdk/client-sso-oidc`) flipped between two equally-valid lockfile forms
across consecutive installs.
The fix awaits `Promise.all` to a temporary array and drains it with `for…of`
so the per-edge results land in input order. This matches the existing pattern
200 lines earlier in `resolveDependenciesOfImporters`.
End-to-end repro from the issue (`pnpm add @aws-sdk/client-s3@3.588.0` then
loop `pnpm dedupe --check`): 33/50 failures without the fix → 0/100 with it.
---
Written by an agent (Claude Code, claude-opus-4-7).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(installing.deps-installer): replace all slashes in mock metadata path
Addresses CodeQL incomplete-string-escaping finding: `replace('/', '%2F')`
only swaps the first occurrence. Scoped names in this test only have one
slash so the behavior is unchanged, but switching to `replaceAll` clears
the warning and is more defensible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(installing.deps-installer): assert raw snapshot key order
Removed the .sort() applied to the lockfile snapshot keys in the cyclic
peer determinism test so the comparison reflects the actual order
emitted by the lockfile writer. The deterministic ordering guaranteed
by
|
||
|
Zoltan Kochan
|
501681044e | chore(release): 11.2.2 (#11817) | ||
|
Zoltan Kochan
|
881a86541b |
fix(installing.commands): forward pnpm install flags to pacquet (#11781)
* fix(installing.commands): forward `pnpm install` flags to pacquet When the install engine is delegated to pacquet via configDependencies, pnpm hard-coded the args to `install --frozen-lockfile --reporter=ndjson` and silently dropped the user's other CLI flags. `pnpm install --no-runtime` therefore still installed the workspace's runtime devDependency, clobbering the Node version the surrounding tooling had set up — visible as the `Verify Node version` failure on PR #11765 where setup-pnpm provisions Node 24.0.0 but pacquet then materializes node 24.6.0. Pacquet's `install` subcommand already mirrors pnpm's surface for the common flags (`--no-runtime`, `--prod`, `--dev`, `--no-optional`, `--node-linker`, `--offline`, `--prefer-offline`, `--cpu`/`--os`/`--libc`). Forward the user's argv verbatim when the command is `install`/`i`; `add`/`update`/`dedupe` still don't forward — their flag surfaces don't line up with pacquet's `install`. * fix(installing.commands): pass --ignore-manifest-check to pacquet `pnpm up` / `add` / `remove` were aborting with `pacquet_package_manager::outdated_lockfile` whenever pacquet was declared in `configDependencies`. After resolving and writing the updated lockfile, pnpm hands materialization off to pacquet but hasn't yet written the post-mutation `package.json` — that write happens after `mutateModules` returns. Pacquet's frozen-lockfile freshness gate then saw the new lockfile paired with the pre-mutation manifest and refused to install. Pass pacquet's new `--ignore-manifest-check` flag (pacquet PR #11811) on every delegation. The flag is narrow: it only skips `satisfies_package_manifest`. Settings drift like `overrides` is still enforced, and pnpm already re-validated the lockfile before delegating, so re-checking the manifest here was redundant work that only ever fired false positives on the mutate-then-materialize path. Requires a pacquet release that ships the flag; bump `PACQUET_VERSION` in `pnpm/test/install/pacquet.ts` once it does, or the existing e2e tests will fail against pacquet 0.2.2-9 (which doesn't recognize the flag and clap would reject). Closes #11797. --- Written by an agent (Claude Code, claude-opus-4-7). * fix: update pacquet in tests * fix(installing.commands): strip positionals + always-injected flags when forwarding to pacquet `collectForwardedFlags` checked `argv[0] === 'install'` to find the command token to strip. Any global flag the user typed before `install` (e.g. `--config.registry=...` in the e2e test) shifted the token out of position, so the function returned the full argv and pacquet saw `install` twice — `error: unexpected argument 'install' found`. Use the parsed argv that `@pnpm/cli.parse-cli-args` already produced: `remain` lists positionals (the `install`/`i` token and nothing else on this code path, since `isInstallCommand` is only true when no package params are present), and `original` preserves the user's exact tokens. Drop positionals + the flags we always inject (`--reporter=ndjson`, `--frozen-lockfile`, `--ignore-manifest-check`) so clap doesn't reject duplicates either. `original` over `cooked` deliberately: nopt's `cooked` splits `--key=value` into two tokens, which would break pacquet's `--config.<key>=<value>` parser (it requires the `=` form). * fix(installing.commands): make argv.cooked/remain optional on InstallCommandOptions Widening these to required broke test fixtures elsewhere (publish/pack/ deprecate/dist-tag/deploy) that construct minimal `argv: { original }` options for code paths that never reach pacquet. Only the pacquet delegation actually reads `remain`, so make the two new fields optional on the shared options type and supply a default at the runPacquet call site. The runtime path through main.ts already populates all three. * fix(installing.commands): strip any user-supplied --reporter when forwarding to pacquet Pacquet's `--reporter` is a clap value option with last-value-wins semantics, so `pnpm install --reporter=silent` (or `--reporter silent` two-token form) reached pacquet and overrode the `--reporter=ndjson` pnpm injects, breaking the NDJSON-to- streamParser plumbing the default reporter depends on. The previous filter only matched the exact `--reporter=ndjson` token. Walk argv with a lookahead so both `--reporter=<value>` and `--reporter <value>` are dropped without consuming an adjacent flag. * fix(installing.commands): drop negated/value forms of always-injected flags `collectForwardedFlags` only matched the exact positive tokens `--frozen-lockfile` and `--ignore-manifest-check`, so a user typing `pnpm install --no-frozen-lockfile` (or `--frozen-lockfile=false`) forwarded the negation to pacquet, which then saw both our injected `--frozen-lockfile` and the user's `--no-frozen-lockfile` and crashed clap with "unexpected argument". Match every shape the user can write the same flag in: positive, `--no-` negated, and any `=value` form. Can't blindly strip `--no-` either way — pacquet has flags whose literal name starts with `no-` (`--no-runtime`, `--no-optional`); those must still forward. The user's `--no-frozen-lockfile` intent is honored upstream — pnpm did a fresh resolve before delegating; pacquet's role here is just lockfile-driven materialization, which is always frozen. * fix(installing.commands): match positionals by index, hide reporter from dropped-flags warning `collectForwardedFlags` matched positionals via `new Set(argv.remain)`, which strips by value: a flag value that happened to equal a positional token (e.g. `pnpm install --node-linker install`) was wrongly dropped from the forwarded list, costing pacquet the value of `--node-linker`. Walk `argv.original` with a subsequence pointer into `argv.remain` so only the actual positional indexes get skipped. `collectDroppedFlags` still surfaced `--reporter foo` / `--reporter=foo` in the "may not be honored" warning on `add`/`update`/`dedupe`, but pnpm honors reporter selection itself before delegation — so the warning was misleading. Route both helpers through the same `isAlwaysInjected` check and consume `--reporter` and its value the same way `collectForwardedFlags` already does. |
||
|
Zoltan Kochan
|
11a43b15da | chore(release): 11.2.1 (#11777) | ||
|
Zoltan Kochan
|
2061c55b2a |
fix(env-installer): mark optional config subdep snapshots with optional: true (#11770)
Match how optional packages are recorded elsewhere in pnpm-lock.yaml so non-host platform variants pulled in via a config dep's optionalDependencies aren't treated as required. |
||
|
Zoltan Kochan
|
e5e7b7241d |
fix(env-installer): suppress 'Installing config dependencies...' on no-op installs (#11766)
* fix(env-installer): only print "Installing config dependencies..." when work is actually being done Previously the message was emitted unconditionally for every config dependency, before any of the "do we need to fetch / re-symlink?" checks. As a result the banner printed on every install even when everything was already cached and correctly linked. Emit the started event lazily — at most once per install, and only when an orphan is being removed, a parent or subdep needs fetching, a parent symlink needs (re)creating, or orphan subdep siblings are being pruned. --- Written by an agent (Claude Code, claude-opus-4-7). * test(env-installer): assert installing-config-deps events fire only when work happens Captures `streamParser` events around `resolveAndInstallConfigDeps` to verify the lazy emission introduced in the previous commit: - fresh install emits both `started` and `done`, - a follow-up no-op install emits neither, - removing a config dep still emits `started` (orphan cleanup work). --- Written by an agent (Claude Code, claude-opus-4-7). * test(env-installer): subscribe to streamParser once at module load `streamParser` is a `split2` Transform stream that buffers writes until the first 'data' listener attaches and then drains the whole buffer into it. Subscribing per-test made the new install-config-deps test capture events from every earlier test in the file. Move the subscription to module load and have each test drain the accumulated events around its own call. Also drop the "removal" assertion: `resolveAndInstallConfigDeps` does not prune entries that disappear from the configDeps argument (lockfile pruning happens at a higher layer), so the scenario it claimed to test never actually fired the orphan-cleanup path. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(env-installer): emit started when only the sibling symlink needs relinking If a config dep's optional subdep is already cached in the global virtual store but the sibling symlink under the parent's node_modules is missing or points at a stale target, symlinkDir() does real work without reportStarted ever firing. Check whether the link already points at the expected target and only fire reportStarted + symlinkDir when it doesn't, mirroring the parentSymlinkAlreadyCorrect path. Also clean up the test-level streamParser listener in afterAll so the subscription doesn't outlive the test file. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
0fb723323f | chore(release): 11.2.0 (#11764) | ||
|
Santiago
|
a62055786b |
fix: handle minimumReleaseAge policy violations in global installs (#11753)
* fix: handle release-age policy in global installs * refactor: dedupe global policy-callback wiring Collapse setupPolicyHandlers + createResolutionPolicyManifestUpdater into one createGlobalPolicyCallbacks helper used by both global add and global update entry points. --- Written by an agent (Claude Code, claude-opus-4-7). --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Eyal Mizrachi
|
9cb48bb27b |
fix: injectWorkspacePackages crashes — lean resolution defense-in-depth + lifecycle re-import (#11662)
Fixes two **independent** crashes hitting `pnpm install --frozen-lockfile` on workspaces with `injectWorkspacePackages: true` (or `dependenciesMeta.*.injected`), surfaced via `turbo prune --docker` pipelines. ## Bug 1 — peer-variant snapshot missing `resolution` (lean, defense-in-depth) A peer-variant injected workspace snapshot (`@scope/pkg@file:packages/pkg(peerA@1)(peerB@2)`) inherits its `resolution` from the base `packages:` entry (`@scope/pkg@file:packages/pkg`). When a tool prunes the lockfile and drops that base entry, readers that deref `pkgSnapshot.resolution` crash with the cryptic: ``` Cannot use 'in' operator to search for 'directory' in undefined ``` **The root cause is upstream of pnpm**: the pruner (e.g. `turbo prune`) emits an internally inconsistent lockfile. Fixed at the source in **vercel/turborepo#12825** (retain the base entry for peer-variant injected deps; minimal repro in **vercel/turborepo#12824**) — empirically verified to produce a correct pruned lockfile for a real multi-service workspace. **pnpm side (this PR): one lean normalization at the read layer** — in `convertToLockfileObject`, where base→variant inheritance already happens via `Object.assign`. When the base entry is absent, reconstruct the directory resolution from the `file:` depPath. This is *reconstruction, not guessing*: for a workspace `file:` dep the directory **is** the depPath suffix — exactly what pnpm's own writer emits. It is **defense-in-depth, not load-bearing**: with a well-formed lockfile (turbo#12825 or any correct input) the branch never fires. Because the normalization sits at the single shared read layer, it also covers the sibling `Cannot use 'in' operator … 'integrity' in undefined` on the `pnpm deploy` path (same `resolution === undefined` root, different deref site). Per review feedback: the earlier per-reader `inheritOrSynthesizeResolution` helper across 5 call sites is **removed**; normalization lives in exactly one place (`convertToLockfileObject`), and the readers are back to `main`. ## Bug 2 — lifecycle re-import wipes `.bin/<tool>` (pure pnpm; the substantive fix) `runLifecycleHooksConcurrently` re-imports an injected workspace package into its targets after `prepare`/`postinstall`. The 2022 `scanDir`-into-`filesMap` workaround (#4299) fed target-internal paths to `importPackage`; once #11088 made `importIndexedDir`'s `makeEmptyDir` fast path the default, that path wipes the target's `node_modules` before copying, so the re-import dies with `ERR_PNPM_ENOENT` on `node_modules/.bin/<tool>`. Fix: drop the `scanDir` workaround and pass `keepModulesDir: true` so `importIndexedDir` skips the destructive fast path and preserves the target's existing `node_modules` (bin symlinks + transitive deps) via its staging/move path. Stays on `storeController.importPackage`, so source files keep their **hardlinks** (no copy-loop regression). Net reduction vs `main`: the `scanDir` helper and the `node:fs` / `FilesMap` imports are removed. ## Tests - The `deps-restorer` regression fixture `peer-variant-missing-resolution` **omits the base `packages:` entry**, so it encodes the actual pruned shape and reproduces the crash on `main`: reverting the `convertToLockfileObject` change yields `resolution: undefined` for the peer-variant (→ the `lockfileToDepGraph` crash); with this PR it is reconstructed as `{ type: 'directory', directory: … }`. - A `lockfile.fs` unit test pins the heuristic boundary: a directory resolution is synthesized for a pruned `file:` peer-variant but **never** for a `file:` tarball. - A `deps-installer` regression test covers the Bug 2 re-import (injected dep with a `prepare` script + a bin-having dependency). ## Validation End-to-end on a real `injectWorkspacePackages` monorepo (`turbo prune --docker` → `pnpm install --frozen-lockfile`), on services that crash on **both** bugs with stock pnpm: - pnpm with both fixes: the crashing services build. - **vercel/turborepo#12825 + pnpm with only Bug 2** (Bug 1 fully reverted): the crashing services still **build** → confirms Bug 1 here is genuine defense-in-depth and turbo#12825 owns the root cause. - Bug 2 reproduces on stock pnpm regardless of turbo (it is purely pnpm's importer fast-path). Pairs with **vercel/turborepo#12825** (Bug 1 root cause; minimal repro **vercel/turborepo#12824**). Tracks #11663. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> Co-authored-by: Eyalm321 <eyal@sunsationsusa.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: UApply Developer <developer@uapply.ai> |
||
|
Zoltan Kochan
|
b206a15395 |
feat(installing): delegate fetch / import / link to pacquet when configured (#11734)
When `configDependencies` declares pacquet (under either the unscoped `pacquet` or the scoped `@pnpm/pacquet` alias), pnpm delegates the fetch / import / link / build phases of an install to the pacquet Rust binary. Pnpm keeps owning dependency resolution — pacquet's resolver isn't ready yet — and hands pacquet a freshly-written lockfile to materialize. Covered install shapes: - frozen install (`tryFrozenInstall` → pacquet, no resolve needed) - default isolated `nodeLinker` (`installInContext`: lockfileOnly resolve via JS, then pacquet) - hoisted `nodeLinker` (same resolve-then-materialize shape) - workspace partial install (subset of workspace projects mutated) - agent-server install (`@pnpm/agent.client` resolves, pacquet materializes) ```yaml # pnpm-workspace.yaml configDependencies: "@pnpm/pacquet": "^0.2.0" # or unscoped `pacquet` ``` ## How it works - `installing/commands/src/runPacquet.ts` resolves the platform binary via `createRequire(realpath(.pnpm-config/<name>/package.json))` — same algorithm the upstream wrapper uses, but skipping the JS shim's extra Node startup. - Pacquet's NDJSON stderr is forwarded through `@pnpm/logger`'s global `streamParser` so `@pnpm/cli.default-reporter` renders its events the same way it renders pnpm's own. Non-JSON stderr lines pass through verbatim. - A few pnpm-side log emits (`importing_done` placeholder, `pnpm:summary`) are suppressed when pacquet will take over so the reporter doesn't close streams or lock in empty diffs before pacquet's real events arrive. Pacquet's duplicate `pnpm:progress status:resolved` events are filtered on the resolve-then-materialize paths so the reporter doesn't double-count. - `installing/deps-installer/src/install/index.ts` gates the delegation on a `runPacquet?: () => Promise<void>` callback in `StrictInstallOptions`. The CLI layer in `installing/commands/src/installDeps.ts` constructs the callback, threaded through both the single-project and workspace-recursive paths. - The `pacquet` and `@pnpm/pacquet` npm packages ship the same JS shim from `pacquet/npm/pacquet/scripts/generate-packages.mjs`; per-platform binaries stay under the existing `@pacquet/<plat>-<arch>` scope and aren't duplicated. |
||
|
Zoltan Kochan
|
1627943d2a |
feat(outdated): include node, deno, and bun runtimes (#11739)
`pnpm outdated` and `pnpm update --interactive` previously skipped runtime dependencies (`node`/`deno`/`bun` installed via the `runtime:` protocol). Both commands go through `outdatedDepsOfProjects` → `outdated()`, and the inner loop bailed out for anything `parseBareSpecifier` couldn't parse — which is everything `runtime:`-shaped. A runtime was only ever reported if the current install differed from the wanted lockfile entry, so the latest available version was never surfaced. The same gap silently affected `jsr:` and named-registry deps too. Commits, smallest fix first → progressively cleaner architecture: 1. **`feat(outdated)`** — minimal fix: special-case runtime deps in `outdated.ts` so they appear in the table and the interactive update picker. 2. **`refactor(outdated)`** — per-resolver dispatch. Each protocol resolver gets its own "what's the latest?" function; `@pnpm/resolving.default-resolver` composes them. 3. **`refactor(outdated)`** — rename to `resolveLatest` (the function returns info regardless of whether the dep is outdated; "outdated" described a state, not an action). 4. **`refactor(outdated)`** — let the local-resolver own the `link:`/`file:` skip, drop the matching short-circuit in `outdated.ts`. 5. **`refactor(outdated)`** — slim `LatestQuery` / `LatestInfo` to the bare essentials; move `pickRegistryForPackage` into the npm-resolver where it belongs; derive `current`/`wanted` display from `pkgSnapshot.version` in `outdated.ts`. 6. **`chore(outdated)`** — drop stale tsconfig project reference left behind by #5. 7. **`refactor(outdated)`** — drop `wantedRef` from the query; resolvers detect protocol from `bareSpecifier` alone. ## Final architecture `@pnpm/resolving.resolver-base` defines a single tiny protocol: ```ts interface LatestQuery { wantedDependency: WantedDependency compatible?: boolean } interface LatestInfo { latestManifest?: PackageManifest } type ResolveLatestFunction = (query: LatestQuery, opts: ResolveOptions) => Promise<LatestInfo | undefined> ``` - `undefined` from a resolver means "I don't claim this dep — try the next one." - `{}` means "I claim it, but I can't tell you what's latest" (policy-blocked, network unavailable, or a protocol with no concept of latest — git/tarball). - `{ latestManifest }` is the happy path. Each protocol resolver (npm/jsr/named-registry, git, tarball, local, node/bun/deno runtimes) exports its own `resolveLatest*` function alongside its `resolve*`. `@pnpm/resolving.default-resolver` composes them into a single first-match dispatcher, surfaced through `@pnpm/installing.client` as `createResolver(...).resolveLatest`. `outdated.ts` is protocol-agnostic: dispatches, then derives `current`/`wanted` display from `pkgSnapshot.version` (falling back to the raw ref for URL-shaped refs where the URL is the only diff signal between commits), uses raw `wantedRef !== currentRef` for the lockfile-shifted check, and pulls `packageName` from `dp.parse(relativeDepPath).name` so aliased deps still report under the real package name. Per-resolver responsibilities: - **npm-resolver** (`resolveLatestFromNpm` / `resolveLatestFromJsr` / `resolveLatestFromNamedRegistry`): match their respective spec shapes, call the matching `resolveFromX` with `'latest'` (or the original spec under `--compatible`), handle `MINIMUM_RELEASE_AGE_VIOLATION` and `ERR_PNPM_NO_MATCHING_VERSION` so policy-blocked deps don't surface as available updates. Picks the per-package registry internally via its ctx. - **node/bun/deno runtime resolvers**: claim deps via `bareSpecifier.startsWith('runtime:')` + alias match, query their release sources for the latest version (only the version — no asset-hash fetches), return `{ latestManifest }`. - **git / tarball resolvers**: claim deps via spec shape, return `{}` (no concept of "latest"); the caller still surfaces a ref-mismatch report if the lockfile shifted to a different commit/URL. - **local-resolver**: returns `undefined` so `link:`/`file:`/`workspace:` deps fall through and get silently skipped. |
||
|
Zoltan Kochan
|
c8d8fde6ca |
feat(config-deps): support optionalDependencies with platform filtering (#11725)
Extends `configDependencies` to resolve and install one level of `optionalDependencies`, with `os` / `cpu` / `libc` platform filtering applied at install time. Closes the prerequisite called out in #11723: this is what makes the esbuild/swc-style platform-binary pattern viable for config dependencies (e.g. shipping pacquet as a config dep with native binaries via `optionalDependencies`). ### What lands - **Resolution** (`resolveOptionalSubdeps.ts`, wired into `resolveConfigDeps` and `resolveAndInstallConfigDeps`): after each top-level config dep resolves, walks one level of `optionalDependencies`, resolves each, and records them in the env lockfile with `os`/`cpu`/`libc` preserved. The parent's snapshot gets `optionalDependencies: { … }`. All variants are recorded regardless of host platform, so the env lockfile stays portable across machines. - **Install** (`installConfigDeps.ts`): after the parent is installed into its GVS leaf, fetches each platform-compatible subdep into its own GVS leaf and creates a sibling symlink inside the parent leaf's `node_modules/`. Node's `realpath`-based resolution then makes `require('pkg-platform-arch')` from inside the parent resolve correctly. Stale siblings are pruned, so platform changes between runs produce a clean layout. - **GVS hash** (new `calcGlobalVirtualStorePathWithSubdeps` in `graph-hasher`): the parent's GVS leaf hash now folds in the optional subdeps' full pkg ids. Without this, changing a subdep version while keeping the parent pinned would land in the same leaf and silently overwrite the sibling symlinks. The leaf function keeps its original "no children" contract; the new function is a separate entry point that pacquet can mirror cleanly. - **Re-install detection**: the "skip if already installed" check compares the existing `.pnpm-config/{name}` symlink's `realpath` against the expected GVS leaf, not the package.json's name/version. With subdep versions now feeding the leaf hash, name/version alone isn't sufficient. The check only short-circuits the parent's re-import and re-symlink — `installOptionalSubdeps` always runs so platform-specific siblings get pruned and relinked when the host's effective platform changes (Rosetta x64 ↔ arm64, etc.). - **Exact versions only**: subdep specifiers must be valid semver exact versions (e.g. `"1.2.3"`). Ranges (`"^1.0.0"`) and tags (`"latest"`) are rejected up-front with a `CONFIG_DEP_OPTIONAL_NOT_EXACT` error. With the parent pinned by integrity, the subdep's resolved version mustn't drift between machines. - **Error handling**: optional-subdep resolution failures are logged via `skippedOptionalDependencyLogger` with `reason: 'resolution_failure'` (same shape as `installing/deps-resolver`) and the install continues — except for `ERR_PNPM_TRUST_DOWNGRADE`, which is a security signal that must still abort the install. ### Scope Only one level deep. Transitive `dependencies` and lifecycle scripts remain unsupported — pacquet doesn't need them yet, and they carry meaningful security and complexity tradeoffs that deserve a separate discussion. The env lockfile schema needs no changes: `LockfilePackageInfo` already carries `os`/`cpu`/`libc`, and `LockfilePackageSnapshot.optionalDependencies` already exists for recording the parent→child edge. ## Known limitation If a workspace already had a resolved config dep in the env lockfile (`snapshots[pkgKey] = {}`) before this PR, optional subdeps won't be retroactively discovered on subsequent installs. Workaround: `pnpm update <pkg>` (or remove + re-add). In practice no published package today relies on `optionalDependencies` in a config dep — they couldn't, since the feature didn't exist — so the practical exposure is narrow. See the inline review thread for the design rationale. |
||
|
Zoltan Kochan
|
cd80b2c8ae | chore(release): 11.1.3 (#11717) | ||
|
Zoltan Kochan
|
2a9bd897bf |
perf: record locally-resolved lockfile in verification cache (#11714)
The lockfile verification cache currently only records the lockfile that exists at the **start** of an install. So a flow like:
```
pnpm install <pkg>
rm -rf node_modules
pnpm install
```
re-runs the per-package registry round-trip against the newly written lockfile, even though the local resolver already enforced the policy when picking those versions. The fresh lockfile is now recorded immediately after each install-time write, so the second install takes the cache fast path.
## Implementation
### Recording the post-resolution lockfile
- New helper `recordLockfileVerified` (in `installing/deps-installer/src/install/`). Gated on `cacheDir` + non-empty `resolutionVerifiers` — same gate the pre-resolution verifier uses.
- Two thin combiners over the lockfile writers: `writeWantedLockfileAndRecordVerified` and `writeLockfilesAndRecordVerified`. The install paths use these so the record always runs alongside the write.
### Hash stability: writer returns the canonical lockfile
The cache stores `hashObject(LockfileObject)` and the next install computes the same hash off the file it loads from disk. For the hashes to match, both ends must compute over structurally identical objects. They don't, naïvely: the in-memory write object can carry `undefined` optional fields (e.g. `settings.dedupePeers = undefined` from `opts.dedupePeers || undefined` in install code) that YAML drops on serialize — `object-hash` treats undefined vs missing as distinct values.
- `writeWantedLockfile` / `writeLockfiles` (in `@pnpm/lockfile.fs`) now return the canonical post-write `LockfileObject`: `convertToLockfileObject(stripUndefinedDeep(lockfileFile))`. The strip walks the existing object graph in memory rather than going through a `yaml.load` round-trip, so non-cache callers (deploy, deps-restorer, make-dedicated-lockfile, agent server) pay near-zero cost.
- Install hooks hash the writer's returned value, not the raw in-memory input. Guaranteed by construction to match what the next reader produces.
### `useGitBranchLockfile` correctness
The pre-resolution verification gate and the new post-write recorder were both keying cache records on a hard-coded `pnpm-lock.yaml`. Under `useGitBranchLockfile` the actual file is `pnpm-lock.<branch>.yaml`, so the stat shortcut hit `ENOENT` and the cache effectively never engaged for git-branch users. Both sites now resolve the real filename via `getWantedLockfileName`. The wrappers compute it once and pass it to the writer via a new optional `lockfileName` opt so `useGitBranchLockfile` installs don't fork `getCurrentBranch` twice per write.
### Bug fix unrelated to the cache, found during review
`writeLockfiles`' differs branch was deciding whether to remove or keep `node_modules/.pnpm/lock.yaml` based on `isEmptyLockfile(wantedLockfile)`. Filtered-current callers (deps-restorer) pass an empty current against a non-empty wanted, so this could leave a stale current lockfile on disk. Fixed to key off the current.
### Comments policy
`AGENTS.md` (and `pacquet/AGENTS.md`) now spell out the comment defaults: write self-documenting code, do not restate at call sites what the callee's JSDoc / doc comment already says, comments are reserved for the non-obvious *why*. The pruning pass in this PR brings the changed code in line.
## API surface
- `@pnpm/lockfile.fs` (minor):
- `writeWantedLockfile`: return widened from `Promise<void>` to `Promise<LockfileObject>`. New optional `lockfileName` opt.
- `writeCurrentLockfile`: return widened to `Promise<LockfileObject | undefined>` (undefined when the empty-lockfile branch unlinks).
- `writeLockfiles`: return widened from `Promise<void>` to `Promise<{ wantedLockfile, currentLockfile }>`. New optional `wantedLockfileName` opt. New exported `WriteLockfilesResult` type.
- New export: `getWantedLockfileName`.
- `@pnpm/installing.deps-installer` (patch): internal-only wrappers; no external API change.
|
||
|
Zoltan Kochan
|
4a79336473 |
feat: report lockfile verification progress (#11712)
* feat: report lockfile verification progress The lockfile resolution verifier introduced in #11705 runs an unbounded registry round-trip on cache miss and was previously silent — on a cold registry cache users saw nothing for several seconds. Emit pnpm:lockfile-verification log events (started/done) around the actual verification pass and render them in the default reporter as a transient progress line that collapses into a final "verified" summary with entry count and elapsed time. The cached short-circuit stays silent. * feat: include lockfile path in verification log and render when non-standard Add `lockfilePath` to the `pnpm:lockfile-verification` event payload so consumers always know which lockfile a `started`/`done` pair refers to. In the default reporter, render the path in the message only when the lockfile lives outside the workspace root (or, for non-workspace installs, outside cwd) — the common case stays uncluttered, while custom `lockfileDir` setups now surface in the verification line. * feat: name what the lockfile verification actually checks in the rendered message "Verifying lockfile" was opaque about *what* was being verified. Reword the rendered messages to explicitly name the check ("supply-chain policies"), so users on a cold-cache pause understand what's happening instead of just seeing the pause. * fix: skip lockfile verification emission for empty candidate set A non-empty lockfile.packages whose snapshots all fail name/version extraction would still emit a "Verifying lockfile (0 entries)" line even though no verifier work runs. Bail before emission when the candidate map is empty so the no-op branch stays silent, matching the contract for the other no-op branches (empty verifiers, no lockfile.packages). * fix(reporter): always close out the verifying-lockfile frame Address two Copilot review points on #11712: 1. The verifier emitted `started` but no terminal event when violations were found or when the registry fan-out threw, leaving "Verifying lockfile…" as the last frame for that block in ansi-diff mode (and an unmatched line in CI logs). Add a `failed` status to the logger, wrap the fan-out in try/finally so a terminal event is emitted on every exit path that emitted `started`, and render a brief failure line so the spinner-style frame is replaced before the PnpmError block prints. 2. The path-suppression heuristic used strict `===` between path.dirname(lockfilePath) and expectedDir, which broke on trailing separators and slash-direction differences. Switch to a path.relative-based check so a workspaceDir like `/repo/` or a Windows path with mixed slashes still correctly suppresses the redundant "at <path>" suffix. * docs: update lockfile verification logging behavior The lockfile verifier now emits log events during the registry round-trip pass, improving user visibility into the process. |
||
|
Zoltan Kochan
|
4195766f10 |
feat: tighten minimumReleaseAge — auto-exclude, lockfile verification, and interactive prompt (#11705)
Three coordinated changes that close the silent-bypass gap in loose `minimumReleaseAge` mode AND the discover-by-loop UX problem in strict mode (#10488), plus a parallel hardening of the lockfile verifier: 1. **Auto-collect into `minimumReleaseAgeExclude` (loose mode)** — fresh resolutions that fall back to a version newer than the cutoff are auto-recorded into the workspace manifest's `minimumReleaseAgeExclude`. A single info message lists what was persisted. The workspace manifest writer dedupes against existing entries. 2. **Lockfile verifier runs in loose mode too** — `createNpmResolutionVerifier` no longer gates on `minimumReleaseAgeStrict`. With auto-collect keeping the exclude list explicit, every accepted-immature pin must be on the list — same contract strict mode enforces. Lockfiles produced under a weaker (or absent) policy that still hold immature entries are rejected the same way strict mode would. 3. **Strict mode prompts on the aggregate set instead of throwing on the first** — the resolver always collects every immature direct and transitive in one pass; the install command's `handleResolutionPolicyViolations` checkpoint decides what to do with the set. Interactive (TTY) prompts the user once with the full list (default = No) and asks whether to add them all to `minimumReleaseAgeExclude` and proceed. Approve → install continues, persisted at the end. Decline → resolution aborts before the lockfile, package.json, or modules dir is touched. Non-interactive (CI) keeps `ERR_PNPM_NO_MATURE_MATCHING_VERSION` as the exit code but lists every offending entry instead of just the first one the resolver happened to hit. 4. **The lockfile verifier now also covers `trustPolicy: 'no-downgrade'`.** The same post-resolution gate that re-checks `minimumReleaseAge` on lockfile entries now re-runs `failIfTrustDowngraded` for every npm-registry entry whose name isn't on `trustPolicyExclude`. The two checks share a single full-metadata fetch per package, so the extra coverage doesn't cost an extra round trip when both policies are active. Resolver-time trust checks still run as before — this just closes the gap when an entry bypasses resolution (peek path, `--frozen-lockfile`, restored CI cache). The steady-state flows: - **Loose mode, `pnpm add foo@immature`**: lockfile clean, verifier no-op, resolver picks via lowest-version fallback, `foo@immature` lands in `minimumReleaseAgeExclude`, install succeeds. Subsequent `pnpm install --frozen-lockfile` in CI verifies against the populated list and succeeds. - **Strict mode (interactive), security bump to `next@15.5.9`**: resolver collects `next@15.5.9` AND every immature `@next/swc-*@15.5.9` shim. pnpm prompts once with the full list. User approves → install completes, all entries persisted in `pnpm-workspace.yaml`. CI then runs the populated config cleanly. - **Strict mode (non-interactive / CI)**: aborts with `ERR_PNPM_NO_MATURE_MATCHING_VERSION` listing every immature entry's `name@version` and publish time — no more discover-by-loop dance. - **Teammate commits a poisoned lockfile**: single-policy batches reject with `ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION` (or `ERR_PNPM_TRUST_DOWNGRADE`); a batch that trips both policies escalates to the generic `ERR_PNPM_LOCKFILE_RESOLUTION_VERIFICATION` and lists each entry's per-policy code in the breakdown. ### Implementation - The npm resolver always falls back to the lowest matching version when no mature version satisfies the range, and flags the result with `ResolveResult.policyViolation` instead of throwing `NO_MATURE_MATCHING_VERSION`. `deferImmatureDecision` and `strictPublishedByCheck` are gone — every caller (install, dlx, outdated, self-update) inspects the violation and decides what to do. - `policyViolation` flows from `ResolveResult` → `PackageResponse.body.policyViolation` → a shared accumulator in `ResolutionContext` → the `resolutionPolicyViolations` field on `resolveDependencyTree`'s return → out through `mutateModules` / `addDependenciesToPackage` to the install command. - The violation type lives in `@pnpm/resolving.resolver-base` as `ResolutionPolicyViolation`; the npm resolver exports the two built-in codes (`MINIMUM_RELEASE_AGE_VIOLATION_CODE`, `TRUST_DOWNGRADE_VIOLATION_CODE`) as constants so consumers reference one source of truth. - `handleResolutionPolicyViolations` runs between `resolveDependencyTree` and `resolvePeers` — the resolver-agnostic checkpoint where the install command's plan prompts (TTY) or aborts (no-TTY) with the full violation list. - `setupPolicyHandlers` (in `installing/commands/src/policyHandlers.ts`) composes per-policy handlers behind a uniform plan interface: each handler has its own `handleResolutionPolicyViolations` (filter by code, decide what to do) and `pickManifestUpdates` (return a typed `WorkspaceManifestPolicyUpdates` patch the install command spreads into `updateWorkspaceManifest`). Today the only registered handler is `createMinimumReleaseAgeHandler` — strict + TTY prompts via `enquirer`, strict no-TTY throws `ERR_PNPM_NO_MATURE_MATCHING_VERSION` with every entry listed, loose mode auto-persists at the tail. Strict + `--no-save` is rejected up-front via `ERR_PNPM_STRICT_MIN_RELEASE_AGE_REQUIRES_SAVE`. Future policies plug in via a sibling factory + push into the handlers list, with no changes to `installDeps.ts` / `recursive.ts`. - `installDeps` / `recursive` drain `pickManifestUpdates` after install and spread the patch into `updateWorkspaceManifest`. Plain `pnpm install` (no `--update`, no params) now still updates the workspace manifest when any handler contributes a patch. The `install` command's CLI schema gained `save: Boolean` so `--no-save` actually flows through to `opts.save = false` instead of being silently dropped by nopt. - `makeResolutionStrict` (in `installing/client`) wraps a `ResolveFunction` and rethrows any `policyViolation` as a `PnpmError`. Used by `dlx` and `self-update` under strict `minimumReleaseAge` OR `trustPolicy: 'no-downgrade'`, since one-shot callers have nowhere to defer a violation to. Violation-code → error-code mapping lives in one place so future violation kinds get consistent UX. - `createNpmResolutionVerifier` extends its check to `trustPolicy: 'no-downgrade'` — same per-entry fan-out, same cache key, sharing the full-metadata fetch with the maturity check. Trust-fetch errors now propagate up so the violation reason carries the underlying message (network code, 404 detail) instead of a generic "metadata is unavailable". - `verifyLockfileResolutions`'s aggregate throw uses the per-policy code when every violation in the batch shares it, and escalates to a generic `LOCKFILE_RESOLUTION_VERIFICATION` (with per-entry codes in the breakdown) for mixed batches. - The pnpm agent path refuses installs under `trustPolicy: 'no-downgrade'` (`ERR_PNPM_TRUST_POLICY_INCOMPATIBLE_WITH_AGENT`) — the agent has no server-side counterpart to that check yet, so silently allowing it would land a lockfile the local verifier would later reject. `minimumReleaseAge` is forwarded to the agent and enforced server-side, so that combination is fine. ### Pacquet parity Pacquet only carries a stub reference to `minimumReleaseAgeExclude` (see `pacquet/crates/package-manager/src/version_policy.rs`); the broader `minimumReleaseAge` and `trustPolicy` policies aren't ported yet, so this feature is outside pacquet's current surface area. It'll come along when pacquet ports the policies. ### Closes - Closes #10488 (resolves the discover-by-loop dance for security bumps without needing `withTransitives`). |
||
|
Zoltan Kochan
|
5dc8be8a42 |
fix(graph-hasher): resolve GVS engine per-snapshot for runtime-pinned deps (#11693)
Closes #11690. A dependency that declares `engines.runtime` in its manifest carries the desugared `dependencies.node: 'runtime:<version>'` pin in the lockfile, and pnpm's bin linker spawns that dep's lifecycle scripts through the pinned Node downloaded into `<pkgDir>/node_modules/node/`. The GVS hash and the side-effects-cache key prefix were still anchored to the install-wide runtime — so the pinning snapshot's slot encoded the wrong Node major, and a reinstall on the same host could read the cached side-effects under a key whose `<platform>;<arch>;node<major>` triple disagreed with the Node the build actually ran on. Per-snapshot resolution now matches what `bins/linker` already does on a per-package basis: a snapshot's own pin wins; the install-wide value (from #11689's `findRuntimeNodeVersion`) is the fallback. ### TypeScript - `deps/graph-hasher/src/index.ts:72-77` — adds `readSnapshotRuntimePin(children)`: pulls the bare Node version from a graph node's `children.node` entry when that points at a `node@runtime:<version>` snapshot. Factors out a small `extractRuntimeNodeVersion(snapshotKey)` parser shared with `findRuntimeNodeVersion`. - `deps/graph-hasher/src/index.ts:115-116,245-246` — `calcDepState` and `calcGraphNodeHash` consult `readSnapshotRuntimePin(graph[depPath].children)` first and only fall back to the install-wide `nodeVersion` parameter when the snapshot doesn't pin its own Node. No caller changes required — install-wide fallback continues to be computed via `findRuntimeNodeVersion(Object.keys(graph))` at each call site. - **Refactor (separate commit):** `findRuntimeNodeVersion` moved from `@pnpm/engine.runtime.system-node-version` to `@pnpm/deps.graph-hasher` (along with the new `readSnapshotRuntimePin`). `system-node-version` is about probing the *host* Node — `getSystemNodeVersion`, `engineName`. The lockfile-shape parsers fit better next to the package that actually composes the engine string. Every caller already depended on graph-hasher, so no new deps; six packages drop the now-unused dependency on `system-node-version`. ### Pacquet - `pacquet/crates/package-manager/src/install_frozen_lockfile.rs:1309-1345` — new `find_own_runtime_node_major(snapshot)` reads a snapshot's `dependencies` for a `node` entry with `Prefix::Runtime`, returning the bare major. - `pacquet/crates/package-manager/src/virtual_store_layout.rs:178-205` — `VirtualStoreLayout::new` resolves engine per-snapshot inside the hash loop via `engine_name(own_major, None, None)` when the snapshot pins, otherwise inherits the install-wide `engine` argument. ### Migration Snapshots of dependencies that declare their own `engines.runtime` re-hash under that dep's pinned Node instead of the install-wide value. Old slots become prune-eligible on next install. |