* fix(installing.commands): clarify "loose mode" wording in minimumReleaseAge log
The log line printed when pnpm auto-adds entries to
`minimumReleaseAgeExclude` referred to internal "loose mode" terminology,
which doesn't appear in the docs and isn't discoverable. Point users at
the actual setting name they need to flip.
Closes#11747
* Update installing/commands/src/policyHandlers.ts
Co-authored-by: Zoltan Kochan <z@kochan.io>
* fix(installing.commands): name the value in minimumReleaseAgeStrict log hint
Change "set minimumReleaseAgeStrict to gate these updates with a prompt"
to "set minimumReleaseAgeStrict to true to ..." so the value is explicit.
---------
Co-authored-by: shiminshen <16914659+shiminshen@users.noreply.github.com>
Co-authored-by: Zoltan Kochan <z@kochan.io>
* fix(deploy): skip configDependencies in the nested install
The deploy directory never installs configDependencies, so the install
engine they designate (e.g. pacquet) isn't on disk to invoke. Without
this override, `pnpm deploy` crashes with `ENOENT: ... lstat
'<deployDir>/node_modules'` when the workspace declares pacquet under
`configDependencies`.
* test(deploy): cover deploy with pacquet in configDependencies
Reproduces the ENOENT crash that happens when `deployFromSharedLockfile`
forwards the workspace's `configDependencies` (e.g. pacquet) into its
nested install and the install engine tries to spawn from
`<deployDir>/node_modules/.pnpm-config/`.
* test(deploy): clarify the public-registry comment in the pacquet deploy test
* fix(pacquet/modules-yaml): write GVS-aware virtualStoreDir to match pnpm
Under `enableGlobalVirtualStore: true`, upstream pnpm mutates
`virtualStoreDir` in place at `extendInstallOptions.ts:419-422` so
every consumer that reads `ctx.virtualStoreDir` — including
`writeModulesManifest` and the `pnpm:context` debug log — sees the
GVS-derived `<storeDir>/v11/links` path.
Pacquet kept `Config::virtual_store_dir` at its project-local value
(deliberately, see `apply_global_virtual_store_derivation`'s rationale)
and wrote that field straight into `.modules.yaml` and the context
log. With `pnpm install` delegating to pacquet via `configDependencies`,
every run came back through pnpm's `checkCompatibility`, the recorded
project-local `virtualStoreDir` didn't match the GVS-mutated value
pnpm computed, and per-importer purges fired the
"modules directories will be reinstalled from scratch" prompt on
every install.
Route both externally-visible consumers through a new
`Config::effective_virtual_store_dir` helper that returns
`global_virtual_store_dir` when GVS is on (which already encodes
"user pinned or fall back to `<storeDir>/links`" via
`apply_global_virtual_store_derivation`) and the project-local
`virtual_store_dir` otherwise. Pacquet's internal layout consumers
still read the field directly — the divergence the helper bridges
is only at the parity boundary.
Test pins both halves: `.modules.yaml` round-trips to
`<storeDir>/v11/links` under GVS, and the `pnpm:context` event
reports the same path.
* fix(pacquet/store-dir): build modules_yaml expected path with Path::join so the test passes on Windows
`modules_yaml_serialized_store_dir_carries_store_version` (added in
3209c2510c) hardcoded `/tmp/.pnpm-store/v11` on the right-hand side
while the left-hand side flows through `StoreDir::from`'s
`PathBuf::join`. On Windows that join emits `\v11`, so the test
panicked with `\v11` vs `/v11` — and has been failing in the
post-merge `Pacquet CI` run for #11891 since it landed.
Pnpm itself uses Node's `path.join` (via `getStorePath` →
`path.join(storePath, STORE_VERSION)`), which is also
backslash-joined on Windows. Building the expected value through
`Path::join` here mirrors that path and keeps the test asserting the
real parity contract on every platform.
* feat(publish): add preserve-manifest-fields option
* fix(publish): omit pnpm field when preserveManifestFields is enabled
The preserve-manifest-fields option was deep-cloning the entire manifest,
which leaked the pnpm-specific `pnpm` field into packed/published manifests.
The PR description explicitly calls for this field to remain stripped;
align the implementation, tests, help text, and changeset accordingly.
* refactor(publish): rename preserve-manifest-fields to skip-manifest-obfuscation
The original name implied the flag preserves *all* manifest fields, which
isn't true — the pnpm-specific `pnpm` field is still stripped, and
`publishConfig` / workspace-protocol / catalog rewriting still happen. The
flag is really an escape hatch from pnpm's manifest mangling, so name it
that way. Help text and changeset updated to match.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
`pnpm add --config <pkg>` (via `resolveConfigDeps`) wrote the env
lockfile without pruning, so optional subdependencies from the
previously resolved version remained as orphans. Mirror the prune
call from `resolveAndInstallConfigDeps`.
* fix(store-dir): append STORE_VERSION to storeDir so pnpm and pacquet stay interchangeable
pnpm's `getStorePath` appends `STORE_VERSION` (`"v11"`) to whatever the
user configured, so the `.modules.yaml` it writes records the v11-suffixed
path. pacquet stored the suffix only as an internal sub-path accessor
(`StoreDir::v11`), which meant `config.store_dir.display()` — the value
pacquet writes to `.modules.yaml`, prints from `pacquet store path`, and
emits in the NDJSON `context` log — yielded the un-suffixed parent.
Switching between the two CLIs in the same project tripped pnpm's
`checkCompatibility` with `ERR_PNPM_UNEXPECTED_STORE`.
Fix is centralised in `From<PathBuf> for StoreDir`, mirroring pnpm's
`if (endsWith(v11)) return; else append(v11)` branch at
store/path/src/index.ts:39-42. Every consumer reading from `StoreDir`
(`display()`, `root()`, `files()`, `tmp()`, `links()`, `projects()`)
now sees the v11-suffixed path through one source of truth, so the
on-disk layout is unchanged and the externally-reported `storeDir`
matches pnpm's exactly.
Ref: https://github.com/pnpm/pnpm/blob/29a42efc3b/store/path/src/index.ts#L39-L42
* fix(store-dir): satisfy Perfectionist macro-trailing-comma on remaining multi-line assert
* fix(store-dir): enforce STORE_VERSION suffix on deserialize via #[serde(from)]
CodeRabbit flagged that the previous `#[serde(transparent)]` derive on
`StoreDir` deserialised straight into `StoreDir::root`, bypassing the
auto-append in `impl From<PathBuf> for StoreDir`. A persisted unsuffixed
path would therefore violate the [`STORE_VERSION`] invariant on the live
struct until the next reconstruction. Pacquet doesn't currently
deserialize `StoreDir` from any disk shape, but the type-level guarantee
is part of the public contract — future serialised state must round-trip
through the suffix logic.
Route both directions through `PathBuf` with
`#[serde(from = "PathBuf", into = "PathBuf")]`. Deserialize now flows
through `From<PathBuf>` (which applies the suffix); serialize converts
to `PathBuf` and back to the same wire shape `transparent` produced, so
no on-disk format change. `Clone` is required by `into` and was added.
Also fix CodeRabbit's doc-comment nit at
project_registry::register_skips_when_store_is_inside_project — the
comment referenced `StoreDir::from` while the test calls `StoreDir::new`;
clarified that `new` routes through `From<PathBuf>`.
Added round-trip tests in `store_dir::tests`:
- `deserialize_applies_store_version_to_unsuffixed_path`
- `deserialize_preserves_already_suffixed_path`
PR #11711 switched updateProjectManifest and the catalog-update loop in
resolveDependencies to look up wantedDependencies by alias, but
parseWantedDependency returns `{ alias: undefined, bareSpecifier }` for
inputs like `pnpm/foo#sha` or tarball URLs whose alias is only known
after fetching the package's package.json. Those entries collided under
the `undefined` Map key, so the alias-keyed lookup of the resolved dep
returned undefined, the filter dropped them from specsToUpsert, and they
silently disappeared from the manifest update and pendingBuilds.
This restored the index-based pairing the code used before #11711.
catalog: preservation isn't affected: it's driven by
rdd.catalogLookup.userSpecifiedBareSpecifier in the spec object, not by
how wantedDep is looked up.
The premise in the removed comment ("linked deps like workspace:* are
excluded from directDependencies") was also wrong — linked deps stay in
directDependencies with isLinkedDependency: true, they're not dropped.
Restores building/commands/test/build/index.ts: rebuilds dependencies,
rebuilds specific dependencies, rebuild with pending option.
* fix: preserve catalog: protocol references on upgrade (issue #11658)
* refactor: address review feedback on catalog: preservation fix
- Fix typo in 3 test assertions (`@pnpm.e2e.foo` → `@pnpm.e2e/foo`)
that made `.toBeFalsy()` pass vacuously
- Use `Map` for alias→wantedDependency lookup in `updateProjectManifest`
to match the pattern in `index.ts`
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
* feat: implement native 'pnpm repo' command
* fix(deps.inspection.commands): preserve repository.directory in fetchPackageInfo
`fetchPackageInfo` flattened `repository` to its URL string, dropping
`directory`. `pnpm repo <pkg>` therefore couldn't append the monorepo
subdirectory for registry packages even though `pickRepoUrl` supported
it. Keep the original repository value so the URL builder receives both
`url` and `directory`.
Also add the missing changeset for the `pnpm repo` command.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
* feat: add native set-script command with ss alias
* refactor(pkg-manifest): host set-script and wire it into the CLI
- Move set-script into @pnpm/pkg-manifest.commands (drops the orphan
@pnpm/pkg.commands package; pkg/* is not in the workspace).
- Use readProjectManifest from @pnpm/cli.utils so package.json5 and
package.yaml are updated in place instead of growing a stray
package.json.
- Remove set-script from notImplemented and register the command in
pnpm/src/cmd/index.ts.
- Cover the ss alias and the multi-word command path in tests.
* refactor(set-script): share the pkg-set primitive
Replace direct manifest.scripts mutation with
setObjectValueByPropertyPath - the same primitive pkg-set uses. Reuses
the prototype-pollution rejection for free and keeps the two commands
on the same write path. Avoids the pkg-set string-CLI's first-equals
key/value split, so script names containing '=' work too.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Implements `pnpm pkg` natively with `get`, `set`, `delete`, and `fix` subcommands.
Workspace usage follows pnpm conventions: use `-r` / `--recursive` for all selected workspace projects, and `--filter` to narrow the selected project graph. This does not add npm-style `--workspace` or `--workspaces` flags.
The PR also extends `@pnpm/object.property-path` with safe set/delete helpers used by the command.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
`getBinName` reads the installed package's `package.json` out of the
GVS slot to discover the bin name. On CI this read has been failing
intermittently for `node@runtime:24.6.0` with
`ERR_PNPM_NO_IMPORTER_MANIFEST_FOUND` — the dlx install reports
`added 1, done`, but the slot the symlink points at has no
`package.json`. The bin link itself is fine (pnpm creates it from the
resolution's `bin` info, not from the slot's manifest), so the only
casualty is `getBinName`.
The slot can end up without `package.json` when something populated it
without going through pnpm's `appendManifest` synthesis (or pacquet's
runtime-manifest synthesis equivalent) — runtime archives don't ship
their own `package.json`, so the synthesized one is the only way it
gets there. Pacquet's `import_indexed_dir` short-circuits on existing
slots without checking which files are present, so a slot populated
by an older code path stays incomplete.
Catch the manifest-not-found error and fall back to the scopeless
package name. For single-bin packages that match `manifest.bin` (the
common case for `pnpm dlx <pkg>`, including every `runtime:` spec),
this gives the same answer the manifest would. Multi-bin packages
already require `--package=<spec> <bin>` to disambiguate, which
short-circuits `getBinName` upstream and never enters this branch.
* feat: add npm stage command
* fix: correct stage command edge cases
* fix: handle stage error paths
* refactor: address stage command review feedback
- Type stageId via OtpPublishResponse so publishPackedPkg no longer needs a cast.
- Hoist fetchFromRegistry + auth header into a per-subcommand StageContext.
- Send npm-auth-type: web on all stage requests, not just approve/reject.
- Consolidate stageRequest / stageRequestWithOtp / stageJsonRequest into (context, params) form.
* fix: trigger stage OTP flow on web-auth challenges
The registry responds to stage approve/reject with 401 and a body of
`{ authUrl, doneUrl }` when the user must complete a browser-based
authentication, but `www-authenticate` does not contain "otp" in that
case. The previous check missed this and surfaced the response as a
generic STAGE_REGISTRY_ERROR. Detect web-auth responses by body shape so
withOtpHandling can drive the polling flow.
* test: cover stage approve web-auth detection paths
Add tests that lock in the OTP-trigger detection on the stage command:
- 401 with `{authUrl, doneUrl}` enters the web-auth flow, exercised here
via the full polling-completion path (registry returns a token, the
retry request carries it as npm-otp).
- 401 with web-auth body but no TTY surfaces as OTP_NON_INTERACTIVE.
- 401 without any OTP signals stays a STAGE_REGISTRY_ERROR so we don't
over-trigger the OTP flow on unrelated unauthorized responses.
* fix: keep web-auth and OTP wrapper on stage publish
Calling `context.publish()` directly for staged publishes bypassed
`publishWithOtpHandling`, so users without a preconfigured token had no
path through the browser-based authentication flow on `pnpm stage
publish`. Route the staged publish through the same wrapper as the
regular publish; `OtpPublishResponse.stageId` carries the registry's
identifier when set.
* refactor: split stage into per-subcommand files and lift tarball helpers
The 631-line `stage.ts` carried six subcommands, tarball parsing,
auth/request plumbing, error class, OTP detection, and rendering helpers
in one file. Reorganized to match the existing `publish/` folder layout:
- `stage/{index,help,publish,list,view,approve,reject,download,context,
request,parsing,rendering,errors,types}.ts` — one concept per file.
- `tarball/{publishSummary,summarizeTarball}.ts` — shared between
`publish` and `stage` instead of duplicated. `PublishSummary` and
`extractBundledDependencies` now live with the tarball helper rather
than inside the publish subfolder, so other commands can reuse them
without reaching into `publish/`.
Behavior unchanged. Also dropped `StageRegistryError`'s redundant
`statusCode` field (was identical to `status`) to bring it in line with
`FailedToPublishError`.
* chore: add TOTP and unparseable to cspell dictionary
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
* fix: cap lockfile verification memory and add trustLockfile opt-out
Verifying a multi-thousand-entry lockfile against `minimumReleaseAge`
or `trustPolicy: no-downgrade` retained every fetched packument in a
per-install cache for the entire install. On large workspaces this
OOM'd CI runners with a 2GB heap cap. Project both caches down to just
the fields each check reads (per-version trust evidence + the `time`
map for trust; package-level `modified` + version-name set for the
abbreviated shortcut) so the bulk packument is GC'd as soon as the
fetch returns.
Also adds a `trustLockfile` setting (default `false`) that skips the
verification pass entirely for environments where the lockfile is
already part of the trusted base. Mirrored in pacquet. Closes#11860.
* perf: share resolver packument cache with the lockfile verifier
The verifier kept its own per-install dedup Maps and re-fetched every
packument the resolver had already pulled during the same install.
Plumb the resolver's per-install `PackageMetaCache` through to the
verifier (via `createNpmResolutionVerifier` / `build_resolution_verifiers`)
so a name already in the resolver's LRU short-circuits the verifier's
disk/network round-trip — fast path only, the cached document is
projected for the trust check so the verifier's memory footprint stays
bounded.
In pnpm, `installing/client` now constructs one LRU and hands it to
both `createResolver` and `createResolutionVerifiers`. In pacquet, the
`InMemoryPackageMetaCache` is lifted to `Install::dispatch` and passed
to both `build_resolution_verifiers` and `InstallWithFreshLockfile`.
- Add `--start-point-clone-thresholds` to the non-main upload arms
so PR/feature branches inherit thresholds configured on main; pair
it with `--err` so a sample over the upper boundary fails the job.
- Add `checks: write` to the three workflows that call `bencher run`.
On main pushes (no `--ci-number`, not a PR event) Bencher falls back
to creating a GitHub Check on the commit; without the permission it
exits 1 with "Failed to create GitHub Check".
* ci(bencher): record benchmark results to Bencher
Tracks both stacks in one Bencher project (`pnpm`), under separate
testbeds (`pnpm`, `pacquet`).
- `benchmarks/bench.sh` emits a hyperfine-shaped `bencher-results.json`
combining the six pnpm scenarios.
- `benchmark.yml` adds `push: branches: [main]` so each merge updates
the `pnpm` baseline, then uploads via the Bencher CLI.
- `pacquet-integrated-benchmark.yml` adds the same main-push baseline
for `pacquet`, combines the four scenario JSONs into a Bencher
report, and stages it into the existing artifact.
- `pacquet-integrated-benchmark-comment.yml` uploads PR results from
the trusted `workflow_run` context so fork PRs are covered too.
Requires `BENCHER_API_TOKEN` in repo secrets; workflows no-op with a
`::notice::` if it's missing.
* ci(bencher): allow workflow_dispatch to upload pacquet results
The inline Bencher upload was gated to `event_name == 'push'`, which
meant manual dispatch from a feature branch ran the bench but skipped
the upload. Both push and workflow_dispatch execute in the base-repo
privilege context, so it's safe to upload from both — the fork-safety
gate only needs to keep `pull_request` runs out.
On dispatch from a non-main branch we record into the ref name with
`--start-point main --start-point-reset`, matching the pnpm bench's
branch policy.
* ci(bencher): treat workflow_dispatch on main as the main baseline
Two small fixes from PR review:
- `benchmark.yml`: manual dispatch from `main` was falling into the
non-main branch arm, recording `--branch main --start-point main
--start-point-reset` — equivalent to forking main from itself.
Treat it like a `push` event so a manual run on main updates the
baseline directly. Matches the pacquet workflow's branch policy.
- `bench.sh`: emit a stderr warning when `jq` is missing instead of
silently skipping `bencher-results.json`. Keeps behaviour optional
for local users but makes the skip discoverable.
* bench: rename scenarios with explicit state axes
Replaces the hyperfine-leaking names (`clean-install`, `frozen-lockfile`,
`peek`, `gvs-warm`, …) with a consistent grid that spells out every
state the benchmark depends on:
- "Fresh" — node_modules wiped at start (future variants will start
with a populated node_modules).
- "Install" vs "Restore" vs "Add new dep" — the work being measured.
- "hot/cold cache + hot/cold store" — both pnpm directories,
spelled out separately because they're distinct on disk.
- "isolated linker" — nodeLinker mode (future variants will cover
`hoisted` and `pnp`).
The slugs map directly from the clap-derived kebab-case names, so
`--scenario=fresh-restore-cold-cache-cold-store-isolated` is the new
CLI surface. Updates land across the Rust orchestrator
(`BenchmarkScenario`), `benchmarks/bench.sh`, the pacquet workflow,
`benchmarks/README.md`, and `pacquet/CONTRIBUTING.md` so the names
agree end-to-end.
Adds a justified `#[allow(clippy::enum_variant_names)]` on the enum
because every variant currently shares the `Fresh` prefix; the lint
will stop firing once `Filled*`/`Resynced*` counterparts land.
Bencher's stored history for the old benchmark names will become
orphaned and can be archived in the UI.
* bench: linker-first slug shape with dot-separated axes
Reshapes the scenario identifiers so the linker mode is the leading
group: `<linker>.<action>.<cache state>.<store state>`. Dots separate
the four axes the bench varies, and `isolated-linker.*` /
`gvs-linker.*` sort together in any dashboard that groups by prefix.
Future buckets (`hoisted-linker.*`, `pnp-linker.*`) will slot in
without disturbing the existing names.
GVS is its own top-level bucket rather than a sub-variant of
isolated — its perf profile differs enough to chart separately.
Renames:
- `clean-install` → `isolated-linker.fresh-install.cold-cache.cold-store`
- `full-resolution` → `isolated-linker.fresh-install.hot-cache.hot-store`
- `frozen-lockfile` → `isolated-linker.fresh-restore.cold-cache.cold-store`
- `frozen-lockfile-hot-cache` → `isolated-linker.fresh-restore.hot-cache.hot-store`
- `peek` → `isolated-linker.fresh-add-dep.hot-cache.hot-store`
- `gvs-warm` → `gvs-linker.fresh-restore.hot-cache.hot-store`
Each Rust variant now carries `#[value(name = "…")]` so clap accepts
the dotted CLI form (`--scenario=isolated-linker.fresh-install.cold-cache.cold-store`).
Display labels follow the slug structure: `Isolated linker: fresh
install, cold cache + cold store` and `GVS linker: fresh restore,
hot cache + hot store`.
The `#[allow(clippy::enum_variant_names)]` is renewed; 5 of 6 variants
share the `Isolated` prefix today. Once `Hoisted*` / `Pnp*` buckets
land the lint will stop firing on its own.
* style: apply rustfmt after scenario rename
The longer match-arm pattern produced by the linker-first rename
exceeded the rustfmt width budget. Auto-format breaks the
`&["install", "--frozen-lockfile"]` body onto its own line so the
arm stays within the limit.
Aligns `dependencies_graph_to_lockfile` with pnpm's
`addDirectDependenciesToLockfile`, which only writes manifest-declared
aliases into `importers["."].dependencies` / `importers["."].specifiers`.
The earlier synthetic-specifier branch (added in #11867 to keep the
top-level symlinks the old recursive walker happened to produce) wrote
the resolver's transitive auto-installed peers into the importer block
with the resolved version as their specifier, so `pnpm-lock.yaml`
ended up listing entries the on-disk `package.json` never declared.
`satisfies_package_manifest` then read those entries on the next install,
flagged 17 "removed" specifiers against the alotta-files fixture, and
the auto-frozen fast path (`preferFrozenLockfile: true`) fell through
to the full fresh-resolve pipeline — turning the bench's
`withWarmCacheAndLockfile` / `withWarmModulesAndLockfile` / `repeatInstall`
scenarios from sub-second no-ops into ~4.5 s re-resolutions in
pacquet@0.2.5.
Verified against pnpm itself (`pnpm install react-dom@18.2.0` →
lockfile lists only `react-dom` in `importers["."].dependencies`,
no top-level `react` symlink). Pacquet now produces the same shape:
transitive auto-installed peers live in `snapshots:` / `packages:`
only, consumers reach them through their parent's slot's
`node_modules`, and the freshness gate stays clean across installs.
Local reproduction on the alotta-files fixture:
- importer.dependencies count: 126 → 109 (matches the manifest exactly)
- `pacquet install --frozen-lockfile`: was failing with
"17 dependencies were removed", now succeeds
- `repeatInstall`: ~22 s → ~0.5 s
Test updates:
- `auto_install_peers_hoists_missing_peers_at_importer` keeps the
`.pnpm/` virtual-store assertions (the actual hoist-loop contract)
and drops the top-level `node_modules/<peer>` symlink loop, which
was asserting behavior pnpm itself doesn't produce.
- `should_install_dependencies` snapshot drops the three
`node_modules/@pnpm/{x,y,z}` entries for the same reason — those
are `@pnpm/xyz`'s transitive peers and never reach the importer's
`node_modules` under pnpm.
Refs #11867.
---
Written by an agent (Claude Code, claude-opus-4-7).
* perf(pacquet): cache per-wanted resolves in resolve_dependency_tree
Memoise `resolver.resolve()` and `extract_children()` on the tree-walk
context. The first visit for a given `(alias, bare_specifier, optional)`
wanted dep runs the resolver chain; later visits clone the cached
`Arc<ResolveResult>` and skip `pick_package`'s cache-key formatting,
HashMap probing, and semver matching. The first visit for a given
`pkgIdWithPatchHash` walks the manifest for child specs; later visits
clone the cached `Arc<Vec<ChildSpec>>` and skip the JSON traversal.
Per-occurrence `NodeId`s and the cycle-break gate are unchanged, so
peer-suffix variants for non-leaf packages stay intact. Mirrors pnpm's
`resolveDependencies.ts:1584` `isNew` gate at the wanted-dep edge,
which is the layer pacquet's recursion shape can short-circuit
without restructuring the tree builder.
Refs #11869.
* docs(pacquet): unlink cross-crate pick_package references
`pick_package` lives in `resolving-npm-resolver`, not in the
deps-resolver crate, so the `[`pick_package`]` intra-doc links broke
`cargo doc --document-private-items` under `-D warnings`. Drop the
link form and keep the bare backticks.
* chore(lint): more clippy rules
* fix: address clippy needless_collect warnings
* fix: silence clippy or_fun_call warnings
* fix: remove unnecessary clone in link_bins tests
* chore(lint): reorder clippy warnings for clarity
* fix: address clippy needless_collect warning in run_workers function
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix cargo fmt
* fix: address review feedback on needless_collect lint suppressions
Switch `#[allow]` to `#[expect]` and revert the `.any()` rewrite in
`resolve_importer/tests.rs` back to the more readable `Vec<&str>` +
`.contains()` form, suppressing the lint locally with a reason.
---
Written by an agent (Claude Code, claude-opus-4-7).
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Zoltan Kochan <z@kochan.io>
Replace `install_with_fresh_lockfile`'s recursive per-package
`install_subtree` walk with the phased warm/cold-batch pipeline
`install_frozen_lockfile` already uses (`CreateVirtualStore` +
`SymlinkDirectDependencies` + `LinkVirtualStoreBins`). The recursive
shape pinned one tokio worker per in-flight package on its own
rayon `par_iter` for the per-package link step; replacing it with a
single phased `par_iter` over every snapshot closes most of the
gap to pnpm on the no-lockfile install paths.
5-run hyperfine against the verdaccio mock, 10-core M-series Mac:
| scenario | before | after | pnpm | who wins |
|---------------------------|------:|------:|------:|----------|
| clean-install, cold cache | 25.5s | 13.7s | 24.6s | **pacquet 1.79×** |
| full-resolution, warm | 22.1s | ~10s | 9.4s | pnpm 1.07× (was 1.94×) |
| frozen, cold cache | 21.1s | 19.6s | 21.7s | pacquet (unchanged) |
| frozen, warm cache | 7.5s | 7.6s | 9.6s | pacquet (unchanged) |
The `install_with_fresh_lockfile` path already builds the v9
lockfile shape from the resolved graph (`built_lockfile`) and
constructs `VirtualStoreLayout::new` — `CreateVirtualStore` consumes
exactly that shape, so the refactor is mostly plumbing. `install_subtree`
and `InstallCtx` go away with it.
`dependencies_graph_to_lockfile` is also updated so auto-installed
peers (resolved under `autoInstallPeers: true` but not present in the
manifest) make it into the importer's `dependencies` map with a
synthetic specifier matching the resolved version. The recursive
walker surfaced these as top-level symlinks by reading
`direct_dependencies_by_alias` directly; `SymlinkDirectDependencies`
reads `built_lockfile.importers["."]`, so without recording the
hoisted peers there, the
`auto_install_peers_hoists_missing_peers_at_importer` regression
test fails. The `specifiers:` map skips the synthetic entry — only
manifest-authored specifiers belong there.
Refs #11866, #11857.
Cuts pacquet's clean-install and full-resolution wallclock by roughly 3-4× on Linux (and roughly 2× on macOS), and brings user-CPU below pnpm's, primarily by:
- pipelining tarball fetches with resolution (re-application of a prior draft);
- caching disk-loaded packuments in `pick_package`'s in-memory `meta_cache` so repeated resolves of the same `(registry, name)` don't re-pay `spawn_blocking` + multi-MB `serde_json::from_str` (biggest single win);
- de-duplicating the resolve-time tarball prefetch via an atomic `DashSet` claim (one spawn per unique URL, not per `(parent, child)` edge), routing the prefetch download through `SilentReporter` so it doesn't fire `pnpm:progress` ahead of `resolved`, and dropping a write-lock that was used purely to read a `CacheValue` variant;
- shaving redundant pre-flight `stat` syscalls in `link_file` (~130k saved per alotta-files install) and `symlink_package` (~3-5k saved), and simplifying the EEXIST recovery to a single `return Ok(())` matching pnpm's `linkOrCopy`.
## Bench (alotta-files fixture, verdaccio mock)
Linux CI numbers from the auto-bench on this PR:
| Scenario | pacquet@main | pacquet@HEAD | pnpm | vs pnpm before → after |
|--------------------------------|-------------:|-------------:|--------:|-----------------------:|
| Clean Install | 19.7 s | **5.91 s** | 4.60 s | 4.29× → **1.29×** slower |
| Full Resolution | 20.4 s | **5.12 s** | 3.34 s | 6.09× → **1.53×** slower |
| Frozen Lockfile | 2.02 s | 2.17 s | 3.57 s | already 1.77× **faster** (unchanged) |
| Frozen Lockfile (Hot Cache) | 756 ms | 752 ms | 2125 ms | already 2.83× **faster** (unchanged) |
User CPU on clean-install drops from 23.9 s (pacquet@main) to 6.19 s (pacquet@HEAD), comparable to pnpm's 6.48 s — meaning the resolver no longer over-uses CPU. The frozen-lockfile path doesn't touch the resolver, so it's unchanged (the small shift is within the stddev bands).
Local macOS sees a smaller win (Clean Install ~1.5× pnpm) — `sys` time is dominated by APFS metadata-journal contention that bites pacquet harder than pnpm because of pacquet's default `2 × available_parallelism` rayon pool. Tracked separately as #11851 / #11857.
## Commits
1. `perf(pacquet): pipeline tarball fetches with resolution` — wraps the resolver chain in a `PrefetchingResolver` that `tokio::spawn`s a `DownloadTarballToStore::run_with_mem_cache` after every successful resolve. Mirrors pnpm's `packageRequester.requestPackage` shape (the `fetching` promise is already running by the time the resolver returns). Re-applies an earlier draft (commit `f375c9161e` on the `pacquet-perf` branch). Also tightens the bench harness's git-repo check to accept worktree `.git` files via a `gitdir:` content check.
2. `perf(pacquet/resolving-npm-resolver): cache disk-loaded packuments in pick_package` — the version-spec disk fast path and the `publishedBy` mtime shortcut loaded the packument mirror from disk but never populated `ctx.meta_cache`, so every later resolve of the same `(registry, name)` re-ran `spawn_blocking(load_meta)` + a multi-MB `serde_json::from_str`. Promote the disk-loaded packument into the install-scoped cache. **Biggest single win** — resolve phase 25 s → 3 s, user CPU 4× lower.
3. `perf(pacquet/tarball): dedup prefetch spawns and read-lock cache state` — multiple improvements to the prefetcher's tarball download path:
- Atomic per-URL dedup via a `DashSet<String>` on the `PrefetchingResolver`. The previous `mem_cache.contains_key()` gate was a TOCTOU pair (two concurrent resolvers could both pass it and both `tokio::spawn`); `DashSet::insert` is the atomic claim and only the winner spawns. The `MemCache` still backstops correctness.
- Route the prefetch's `DownloadTarballToStore::run_with_mem_cache` through `SilentReporter` so `pnpm:progress fetched` / `found_in_store` don't fire ahead of the install pass's `resolved`. The install pass's later call to `run_with_mem_cache` now emits `found_in_store` on the `Available` short-circuit, so consumers still see the documented `resolved → fetched|found_in_store → imported` triple once per URL.
- Switch the waiter's `cache_lock.write().await` to `read().await` — the variant inspection is read-only, and the owner branch is the only writer.
4. `perf(pacquet/package-manager): drop pre-flight stats in link_file` — first half of the link_file simplification: the original two `fs::metadata` + `fs::symlink_metadata` calls per file collapsed to a single `fs::metadata` short-circuit, deferring dangling-symlink detection to the EEXIST recovery path.
5. `perf(pacquet/package-manager): drop redundant create_dir_all in symlink_package` — `force_symlink_dir` already handles a missing parent via its own `NotFound` → `create_dir_all` retry; the explicit pre-create was paying ~3-5k unneeded `stat`s on a typical install.
6. `fix(pacquet/package-manager): keep one stat in link_file, simplify EEXIST` — second half of the link_file simplification. Keeps the single `fs::metadata` pre-check (needed because `fs::copy` overwrites silently rather than surfacing EEXIST, so `Copy` callers need the no-op short-circuit) and simplifies the EEXIST recovery to a single `return Ok(())` matching pnpm's `linkOrCopy`. Drops the dangling-symlink scrub for the same reason — pnpm doesn't scrub either, and CAFS contents are content-addressed so a "stale" dangling symlink from a crashed prior install is a corruption case the install layer is not the right place to repair. Test `link_file::tests::dangling_symlink_is_replaced` → `dangling_symlink_is_preserved` locks in the new (pnpm-aligned) contract.
## Where the remaining gap lives
pacquet@HEAD user-CPU on clean-install is now **lower** than pnpm's (6.19 s vs 6.48 s on the Linux CI). The remaining ~1.3-1.5× wallclock gap is `sys` time — file-system syscalls during the link phase (verify-files-cache stats, per-file `linkat`/`clonefile`/`copy_file_range`, store-index lookups). The macOS-only multiplier on top of that is APFS metadata-journal contention from pacquet's over-subscribed rayon pool; see #11851 and the follow-up tracker #11857.
* ci(benchmark): run all six integrated-benchmark scenarios
Wires `clean-install`, `full-resolution`, `peek`, and `gvs-warm` into
`pacquet-integrated-benchmark.yml` so per-PR runs cover the same scenario
set the manual `benchmark.yml` workflow already exercises via
`benchmarks/bench.sh`. Requested for #11837, where the perf delta affects
the resolution-bound scenarios (`firstInstall`, `withWarmCache`,
`withWarmModules`, `updatedDependencies`) that the prior two-scenario set
did not measure.
Each scenario gets its own step with a 10 min hyperfine timeout (same
rationale as the existing steps) and writes per-scenario report copies
that the summary step concatenates into `SUMMARY.md`.
* ci(benchmark): drop peek and gvs-warm scenarios
Keep only the two new no-lockfile scenarios (`clean-install`,
`full-resolution`) on top of the existing `frozen-lockfile` and
`frozen-lockfile-hot-cache`. #11837's perf change is in the
fresh-lockfile install path, which only runs when resolution runs — i.e.,
exactly the no-lockfile scenarios. `peek` mutates an existing lockfile
and `gvs-warm` is a frozen-lockfile variant; neither exercises the
affected path, and including them only costs per-PR CI wall time.
* fix(bench): pin packages: ['.'] in synthesized pnpm-workspace.yaml
The integrated-benchmark clones each pacquet revision's source tree into
`<bench_dir>/pacquet/`, which on the pnpm/pnpm monorepo includes upstream
test fixtures like
`workspace/project-manifest-reader/__fixtures__/invalid-package-json/package.json`
— intentionally malformed JSON used to exercise pnpm's manifest reader.
Without a `packages:` field, both pnpm's `findPackages.ts:28` and
pacquet's `crates/workspace/src/projects.rs:128` default to `[".", "**"]`,
so the fresh-resolve install path's `find_workspace_projects` walk
descends into the cloned source tree and trips on the bad fixture:
Error: pacquet_package_manifest::serialization_error
× installing dependencies
╰─▶ expected `,` or `}` at line 3 column 3
The walk only runs on the fresh-lockfile branch (`install.rs:628-630`),
which is why frozen-lockfile and frozen-lockfile-hot-cache stay green
while clean-install and full-resolution fail every time.
Pin `packages: ['.']` in the synthesized manifest so enumeration stays
at the workspace root. The benchmark's installs are single-project,
so this doesn't narrow anything the install actually needed to see.
Fixtures supplied via `--fixture-dir` that already declare `packages:`
keep their own value.
* ci(benchmark): bump no-lockfile scenarios to 20 min
Clean-install and full-resolution go through pacquet's fresh-resolve
install path, which is currently ~3-5x slower than pnpm on the
`alotta-files` fixture (pnpm/pnpm#11832). Hyperfine's default 1 warmup
+ 10 timed runs across three benchmark targets (pacquet@HEAD,
pacquet@main, system pnpm) projects to ~13 min wallclock for these
two scenarios, putting the previous 10 min cap right on the edge.
Doubling to 20 min keeps the per-step timeout meaningful as a stuck-
install detector without losing CI time when the bench is healthy.
The frozen-lockfile steps stay at 10 min — they don't traverse the
slower fresh-resolve path.
* fix(bench): drop --no-frozen-lockfile from full-resolution scenario
Pacquet doesn't expose `--no-frozen-lockfile` (only `--frozen-lockfile`,
`--prefer-frozen-lockfile`, and `--no-prefer-frozen-lockfile`). Passing
it makes clap reject the install:
error: unexpected argument '--no-frozen-lockfile' found
tip: a similar argument exists: '--frozen-lockfile'
The flag was redundant for this scenario anyway: full-resolution starts
every iteration with no lockfile on disk (init() skips the lockfile when
`lockfile_enabled()` is false; cleanup removes it; `lockfile=false` in
the synthesized npmrc/workspace prevents writing one). With no lockfile
present the frozen path is unreachable regardless of the flag, so both
tools take fresh resolution by definition. Fold full-resolution into
clean-install's bare `install` arm.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Mirrors pnpm's `pkgIsLeaf` reuse: a package with no `dependencies`,
`optionalDependencies`, `peerDependencies`, or `peerDependenciesMeta`
collapses every parent reference onto one tree node keyed by the
package id, instead of allocating a fresh `NodeId` per occurrence.
Peer resolution and every downstream `HashMap<NodeId, _>` see the
reduced node set.
Closes#11844.
* fix(installing.deps-resolver): deterministically order cyclic peer suffixes (#8155)
`resolveDependencies` was pushing onto `pkgAddresses`, `postponedResolutionsQueue`,
and `postponedPeersResolutionQueue` from inside `Promise.all`-spawned callbacks,
so the order of items in those arrays reflected completion timing rather than
the order of `extendedWantedDeps`. That ordering then flowed downstream into
`resolvePeers` and the cyclic-peer suffix assignment, so two packages with
transitive peer dependencies on each other (e.g. `@aws-sdk/client-sts` and
`@aws-sdk/client-sso-oidc`) flipped between two equally-valid lockfile forms
across consecutive installs.
The fix awaits `Promise.all` to a temporary array and drains it with `for…of`
so the per-edge results land in input order. This matches the existing pattern
200 lines earlier in `resolveDependenciesOfImporters`.
End-to-end repro from the issue (`pnpm add @aws-sdk/client-s3@3.588.0` then
loop `pnpm dedupe --check`): 33/50 failures without the fix → 0/100 with it.
---
Written by an agent (Claude Code, claude-opus-4-7).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(installing.deps-installer): replace all slashes in mock metadata path
Addresses CodeQL incomplete-string-escaping finding: `replace('/', '%2F')`
only swaps the first occurrence. Scoped names in this test only have one
slash so the behavior is unchanged, but switching to `replaceAll` clears
the warning and is more defensible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(installing.deps-installer): assert raw snapshot key order
Removed the .sort() applied to the lockfile snapshot keys in the cyclic
peer determinism test so the comparison reflects the actual order
emitted by the lockfile writer. The deterministic ordering guaranteed
by 7577d47 makes the sorted view and the raw view identical today;
dropping the sort lets the test fail on any future regression that
keeps the key set stable but shuffles the order.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(installing.deps-installer): drop stray manifest from MutatedProject literal
MutatedProject does not carry a manifest field; it is conveyed via
allProjects in MutateModulesOptions. Passing it inside the install
project literal triggered TS2353 against the InstallDepsMutation shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(installing.deps-installer): rename metas to metaByName
Clearer name for the map keyed by package name, and avoids tripping
cspell on the abbreviation "metas".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: retrigger CI
Ubuntu Node.js 22.13.0 hit a transient 404 from Verdaccio's
proxy-to-npm while resolving a transitive peer of
@medusajs/medusa-js@6.1.7 in the pre-existing
"install should not hang on circular peer dependencies"
test (installing/deps-installer/test/install/misc.ts:1247).
Ubuntu Node 24 and Node 26 ran the same code green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes#11832.
On the `alotta-files` benchmark (1362 nodes, warm cache, GVS on), pacquet was 3-5× behind the TypeScript pnpm CLI whenever resolution ran (`firstInstall`, `withWarmCache`, `withWarmModules`, `updatedDependencies`). Wall-clock dropped from ~11.83s to ~5.03s on this branch; pnpm sits at ~4.16s, and the remaining gap is concentrated in the resolver's per-node `pick_package` walk (3.1s of the 5.03s — see #11843 for the `peekManifestFromStore` follow-up that would close it).
The branch is a series of small wins rather than one big rewrite. The original `PrefetchingResolver` (commit f375c916) was replaced by a batched store-index prefetch (461a4c02) — same throughput, far less plumbing.
## What's in this PR
### Resolve-phase
- **Packument fetch dedup** (386a90b5) — `PackumentFetchLocker` (per-cache-key `DashMap<String, Arc<Semaphore>>`) so concurrent picks of the same `(registry, name)` coalesce into one HTTP GET. Mirrors pnpm's `runLimited(pkgMirror, …)` in `pickPackage.ts`. Pacquet was firing N parallel GETs for the same packument per cluster of cross-referencing deps; now it's one.
- **Conditional GET on upgrade fetch** (58f49c90) — forward `etag` / `modified` so the registry can answer `304 Not Modified` on the abbreviated-to-full re-fetch path.
- **Off-tokio mirror disk reads** (6cb50b4f) — the packument cache's mirror read moves to `spawn_blocking` instead of running on the tokio worker.
- **Picked-manifest serialisation dedup** (387b8721) — `PickedManifestCache` `Arc<DashMap<String, Arc<Value>>>` so duplicate picks of the same `name@version` reuse the already-serialised `Arc<Value>` instead of re-running `serde_json::to_value`.
- **Arc-shared resolver outputs** (743c718f, 53e3cde6, 5d6a4207) — `Package`, `ResolveResult.manifest`, and `ResolveResult` itself are now shared via `Arc` so the tree walk's per-occurrence clones become refcount bumps.
- **`std::sync::Mutex` on `TreeCtx`** (a7c94a90) — the per-package dedupe gate is a short `HashMap` insert with no `await` inside, so a sync mutex is the right tool. Tokio's async mutex was paying per-acquire overhead once per visit per ctx field on the resolve hot path.
### Install-phase
- **Batched store-index prefetch** (461a4c02) — one `SELECT … WHERE key IN (…)` against `index.db` at install start, rayon-parallel verify, drops the SQLite mutex before any fs work. Replaces the per-snapshot `spawn_blocking` fan-out that was serialising on `Arc<Mutex<StoreIndex>>` and queueing in tokio's blocking pool.
- **Single `pnpm:progress` per URL** (e54208e1) — `run_with_mem_cache` was emitting `fetched` twice when the in-memory cache hit; mirrors pnpm's `packageRequester` shape where the emit fires exactly once.
- **Retain prefetched manifests for bin linking** (b0bf5970) — the fresh-install bin linker now drives `LinkVirtualStoreBins` with the prefetched bundled-manifests map (and built lockfile snapshots), so per-child `package.json` disk reads on warm hits are gone. Skips the `read_dir` enumeration too. Updated snapshots reflect the now-present `<slot>/node_modules/<pkg>/node_modules/.bin/<pkg>` self-shim the lockfile-driven path writes per pnpm's `linkBinsOfDependencies`.
- **Skip per-snapshot deep clone on warm prefetch** (3f9c1bb5) — `run_with_mem_cache` returns the prefetched `Arc<HashMap>` straight through instead of going via `run_without_mem_cache`'s deep-clone path + redundant `Arc::new`. At 1k+ snapshots that's one per-file map allocation and one `Arc::new` saved per snapshot.
### Correctness
- **Registry-scoped `PickedManifestCache` key** (57c3094e) — the shared cache key was `{name}@{version}` only; two registries (default + JSR + named-registry) serving different artifacts under the same `name@version` (private + public collisions, forks) would hand one resolver the other's manifest. Now keyed `{registry}\x00{name}@{version}`, matching `PackageMetaCache`'s shape. Regression test included.
### Diagnostics
- **Per-phase timing logs** (57864d2c) — `pacquet::install::phase` `tracing` events with `elapsed_ms` for `resolve_importer`, `prefetch_cas_paths`, `build_fresh_lockfile`, `virtual_store_layout_new`, `install_subtree`. Made the profiling for this branch tractable and stays in for future work (the same per-phase trace is what motivated #11843).
### Review cleanup
- **Refactor + comment fixes** (e7b3e6ca) — `build_resolve_result` now takes a `BuildResolveResult` struct instead of 9 positional args (`#[allow(clippy::too_many_arguments)]` gone); resolver-side `Arc` bindings are dropped explicitly before the install pass so the packument cache actually frees; the `resolve_dependency_tree` doc comment was wrong about "skipping the recursion" on dedupe hits.
### Doc + Dylint fixes
- **CI compliance** (0343a472) — broken doc links resolved; single-letter generics, `Arc.clone()` direct, unicode ellipsis in doc comments fixed for Dylint.
## Scope
- **Fresh-lockfile install path only.** The frozen-lockfile path already had the batched store-index prefetch; nothing else here changes its behaviour.
- **No user-visible behavior change** — lockfile format, error codes, CLI surface, `MemCache` semantics unchanged. The cache-key bug fix doesn't change the on-disk lockfile; it only prevents an in-memory mix-up that would otherwise produce a wrong `ResolveResult.manifest` field.
## Follow-ups
- **#11843** — port pnpm's `peekManifestFromStore` fast path. The store-index row carries the bundled `package.json` (name, version, deps, bin, engines, etc.) but no publish-time, so the fast path is safe only when no `published_by` / `minimumReleaseAge` policy is in effect, no `--update`, and the wanted lockfile pins a tarball+integrity. With ~95% of nodes short-circuiting, the `resolve_importer` phase (currently 3.1s on warm cache) drops dramatically — this is the single biggest unimplemented win and the most likely path to parity with pnpm.
## Benchmark
Wall-clock progression on the `alotta-files` warm-cache + GVS-on scenario (this branch vs `main`):
| Stage | Wall |
| ----------------------------------- | ------- |
| `main` | ~11.83s |
| + packument fetch dedup | ~8.21s |
| + batched store-index prefetch | ~6.39s |
| + Arc-shared resolver outputs | ~5.67s |
| + std Mutex + manifest serial dedup | ~5.03s |
| pnpm CLI baseline | ~4.16s |
The remaining ~0.87s gap is concentrated in `resolve_importer` and is what #11843 targets.
Historical npm registry entries sometimes carry object-valued
`dependencies` / `devDependencies` / `peerDependencies` (e.g.
`deep-diff@0.1.0`'s nested `devDependencies: { vows: { version, ... } }`).
pnpm's JS path silently ignores these via later `typeof spec === 'string'`
checks; pacquet's strict serde failed the whole packument with
`invalid type: map, expected a string`, surfacing as a spurious
`ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION` for an unrelated version pinned
in the lockfile.
Custom deserializer drops non-string entries while keeping the string
ones, matching pnpm's tolerance. Issue #11829.
Unify the two install-benchmark stacks (`benchmarks/bench.sh` and `pacquet/tasks/integrated-benchmark/`) into one shared Rust orchestrator. Scenario, fixture, workspace-manifest, install-script, and report generation live in one place so changes propagate to both stacks.
Scenario sets per workflow are unchanged: `benchmark.yml` keeps measuring the same 6 scenarios; `pacquet-integrated-benchmark.yml` keeps measuring the same 2. Both verdaccio and live-npm registry modes are preserved; neither is removed in favor of the other. All six scenarios accept both `pacquet@<rev>` and `pnpm@<rev>` targets.
The verifier was racing in-flight writers: while `ensure_file` held
`cas_write_lock(path)` and was partway through `write_all`, a
sibling snapshot's `check_pkg_files_integrity` would stat the same
CAS path (no lock), see a partial size, and call
`remove_stale_cafs_entry(path)`. The writer's `write_all` then
continued against an orphan inode, the writer's `cas_paths` was
populated with the now-deleted path, and `link_file` later hit
ENOENT — the CI failure shape on #11816 / 0.2.2-7.
Fix (Option C): keep `verify_file`'s lock-free fast path (the
common case: file unchanged since prior install, `is_modified`
false), but acquire `cas_write_lock(path)` before any branch that
could call `remove_stale_cafs_entry`. Re-`check_file` under the
lock so a writer's full `write_all` lands before we evaluate.
Performance: the fast path adds zero overhead. The slow path —
files whose mtime is > 100 ms past the recorded `checked_at` —
takes one uncontended Mutex acquire per file, sub-microsecond
on uncontended locks. Contention only fires when a writer + a
verifier hit the same blob simultaneously; the wait is bounded
by one `write_all` and trades a millisecond-scale wait for
avoiding a network re-fetch.
The new integration test in `pacquet-store-dir/tests/` is a
deterministic reproducer: it acquires `cas_write_lock` from the
test thread (standing in for an in-flight writer), pre-seeds a
partial CAS file at the matching path, and runs the verifier in
the background. Pre-fix, the verifier unlinks the file while the
"writer" is still simulated as in-progress; post-fix, the
verifier blocks on the lock until released.
To make `cas_write_lock` reachable from the store-dir crate the
function was promoted from `fn` to `pub fn` in pacquet-fs.
Adds the three cross-process scenarios upstream pnpm covers in
store/cafs/test/writeBufferToCafs.test.ts but pacquet only covered
intra-process (one 32-thread test): N workers racing on the same
target with a corrupt pre-seed, with a truncated pre-seed, and on a
clean target.
Pacquet's `cas_write_lock` is process-local (`OnceLock<DashMap<...>>`)
just like upstream's `locker: Map<string, number>`, so the cross-
process safety contract lives entirely in `O_CREAT | O_EXCL` +
atomic-rename. The existing 32-thread test in `ensure_file::tests`
exercises the lock; the new suite exercises the unprotected
filesystem-only path so a regression in the `verify_or_rewrite` +
`write_atomic` recovery would surface as a test failure instead of a
production install failing to import a CAS file at link time.
Approach:
- New `[[bin]]` `cafs_stress_worker` under `pacquet-fs/src/bin/`.
Reads a content fixture from argv[1] and a target path from
argv[2], calls `ensure_file`, exits 0/1. Tiny and test-only;
`pacquet-fs` is `publish = false` so an extra bin target is free.
- New integration test `pacquet-fs/tests/ensure_file_stress.rs`.
Uses `env!("CARGO_BIN_EXE_cafs_stress_worker")` to find the bin
Cargo builds alongside the test, then spawns 8 instances per
scenario via `std::process::Command`.
- Each scenario then asserts every worker exited 0 and the final
on-disk content sha-512-matches the expected payload.
Each recovery test was verified to catch a regression: temporarily
bypassing `verify_or_rewrite` flips both recovery tests red while
the clean-target test (which doesn't pre-seed anything) stays
green, matching upstream's coverage shape.
`pacquet install` (no flag) didn't consult `preferFrozenLockfile`. A
fresh lockfile got re-resolved from the registry instead of taking the
cheap frozen path, and a stale lockfile was silently overwritten
without seeding the resolver from the existing pins. Closespnpm/pnpm#11815.
The install dispatch now has four ordered states:
1. `--frozen-lockfile` flag → frozen path (lockfile required, freshness
check fatal).
2. No flag + lockfile present + effective `preferFrozenLockfile == true`
+ freshness check passes → frozen path (same code as state 1).
3. No flag + lockfile present + opt-out or stale → fresh-resolve, seeded
from the existing lockfile's snapshots so unrelated pins survive the
rewrite (mirrors upstream's `update: false` resolver mode).
4. No lockfile → fresh-resolve with no seed.
`check_lockfile_freshness` is the shared helper: it runs
`pnpm.overrides` parsing, `check_lockfile_settings`, the overrides-aware
manifest re-apply, and `satisfies_package_manifest`. State 1 surfaces
its `Err` as `InstallError`; state 2 treats a stale-lockfile `Err` as
fall-through and surfaces `InvalidOverrides` as fatal.
CLI exposes `--prefer-frozen-lockfile` / `--no-prefer-frozen-lockfile`
mirroring pnpm so users can override per invocation; `pacquet add` opts
out of the fast path explicitly since the manifest is necessarily
stale by the time the install dispatch runs.
`pacquet install` (no flag, fresh project) was hardcoded to
`VirtualStoreLayout::legacy`, so it materialized packages under the
project-local `node_modules/.pnpm/` even when `enableGlobalVirtualStore:
true` was configured. Closespnpm/pnpm#11814.
A new `build_lockfile_view_from_resolver_graph` adapter converts the
resolver's `DependenciesGraph` into the `snapshots:` / `packages:`
shape `VirtualStoreLayout::new` already consumes, so all hashing,
slot-dir, and bin-linker code is shared with the frozen-lockfile path.
The without-lockfile flow now also calls `register_project` against
the shared store when GVS is on, mirroring the frozen-lockfile branch.
Side effect: aligns the without-lockfile path's per-package save
directory with the peer-suffixed slot the children-recursion was
already using, so peer-suffixed snapshots no longer split between two
unreachable slot directories.
* feat(pacquet): write pnpm-lock.yaml on fresh install
Adds a `dependencies_graph_to_lockfile` adapter that converts the
resolver's `DependenciesGraph` into a v9 `Lockfile`, hooks it into
the install path so a fresh `pacquet install` produces a wanted
lockfile, renames `InstallWithoutLockfile` to `InstallWithFreshLockfile`
to match the new behavior, drops the `UnsupportedLockfileMode` branch
from the dispatch, and flips the `config.lockfile` default from `false`
to `true` to match pnpm.
Closes pnpm/pnpm#11813.
* chore(pacquet): fix rustdoc + rustfmt CI failures
- Drop the broken `[ResolvedPackage.optional]` intra-doc link (no such
item in scope; the reference is upstream's, and the prose is enough
without the link).
- Disambiguate two `dependencies_graph_to_lockfile` intra-doc links to
the function form so rustdoc doesn't error on the function/module
collision.
- Flatten an `assert!(prod.contains_key(...))` so rustfmt + dylint
agree on the body (rustfmt wanted the call wrapped, dylint wanted a
trailing comma; extracting the key into a local resolves both).
* feat(pacquet): write <virtual_store_dir>/lock.yaml in the fresh path
After a fresh install, also persist the current-lockfile alongside
`pnpm-lock.yaml` so the next install can diff each snapshot against it
and skip the unchanged slots — the same optimization the frozen path
already enables.
`InstallWithFreshLockfile::run` now returns an
`InstallWithFreshLockfileResult` that surfaces the freshly-built
`Lockfile` to the caller. `install.rs` saves it as
`<virtual_store_dir>/lock.yaml` after `.modules.yaml` succeeds, mirroring
the frozen path's safety property: a manifest-write failure can't leave
a current-lockfile pointing at incomplete install state.
The wanted lockfile and the current lockfile describe the same resolved
graph here — the resolver only walked what the install requested, so
no `filter_lockfile_for_current` step is needed. Both writes are gated
on `config.lockfile`, matching upstream pnpm's `useLockfile` opt-out.
* feat(pacquet): propagate optional flag from ResolvedPackage to SnapshotEntry
`SnapshotEntry.optional` was hard-coded to `false`, so every snapshot
looked "non-optional" and `BuildModules` would treat any build failure
as fatal even for packages reachable only via `optionalDependencies`.
Port upstream pnpm's `ResolvedPackage.optional` propagation:
1. `ResolvedPackage` (the dedup envelope) gains an `optional: bool`
field. The walker seeds it from `wanted.optional || parent.optional`
on the first visit and AND-folds it with `current_is_optional` on
every subsequent visit, so a single non-optional path flips it back
to `false` and keeps it there. Mirrors
resolveDependencies.ts:1625-1630.
2. `extract_children` now emits a per-child `is_optional` flag — `true`
when the entry came from the package's `optionalDependencies` map.
3. `resolve_dependency_tree` and `resolve_importer` look up the
importer's `optionalDependencies` set and tag each direct dep, then
propagate the flag down the recursion.
4. `DependenciesGraphNode` carries the resolved package's `optional`
field so peer-variants of the same `pkgIdWithPatchHash` share it.
5. The lockfile adapter writes `SnapshotEntry.optional = node.optional`
instead of hard-coding `false`.
Tests:
- 4 unit tests on the resolver (direct optional seed, transitive
inheritance, AND-fold when a non-optional path exists, transitive
via a parent's `optionalDependencies` edge).
- 1 unit test on the adapter (`SnapshotEntry.optional` round-trips).
- 1 integration test through `Install` asserting a top-level
`optionalDependencies` entry produces `optional: true` in the
written `pnpm-lock.yaml`.
Each test was verified to catch a regression by temporarily breaking
the implementation before re-checking it green.
* fix(pacquet): fail fast when fresh install gets node_linker: hoisted or --no-runtime
The fresh-lockfile dispatch silently dropped `skip_runtimes` and
`node_linker` — neither was forwarded into `InstallWithFreshLockfile`,
so a fresh `pacquet install` with `--node-linker=hoisted` produced an
isolated `node_modules` layout, and `--no-runtime` materialized runtime
archives anyway.
Pacquet's hoist pass and runtime-snapshot filter both run only against
a loaded lockfile (frozen-lockfile path); honoring the flags on a
fresh install needs upstream's per-snapshot filter and a hoist pass
over the freshly-built graph. Both ports are out of scope for this
PR.
Refuse the unsupported combinations up front instead, before any
reporter event fires or any state file is written, so a follow-up
retry under `--frozen-lockfile` against an existing lockfile lands on
the supported path. Adds `UnsupportedFreshInstallNodeLinker` and
`UnsupportedFreshInstallSkipRuntimes` error variants plus integration
tests asserting both that the error fires and that no `pnpm-lock.yaml`,
`lock.yaml`, or `.modules.yaml` ends up on disk.
* chore(pacquet): broaden fresh-install-flag error messages to non-frozen installs
The `UnsupportedFreshInstall*` errors fire on any `!frozen_lockfile`
run, not just first installs — pacquet doesn't honor an existing
lockfile without `--frozen-lockfile` yet (stale-lockfile rewrite is a
follow-up). So a re-install with a pinned `pnpm-lock.yaml` but no
`--frozen-lockfile` would hit the same gate, and the "on a fresh
install yet" wording pointed users at the wrong condition.
Reword the display strings to "without --frozen-lockfile yet" so the
prompt matches the actual gate. Variant names and diagnostic codes
stay put — the next round will rename them along with the
fresh-install-path semantics.
Surfaced by coderabbitai review on #11816.
The frozen-lockfile freshness check compared the lockfile's overrides
map (with `catalog:` already expanded by pnpm) against the raw config
map (still containing `catalog:` strings), so every catalog-backed
override surfaced as `ERR_PNPM_OUTDATED_LOCKFILE` on every install.
Mirror pnpm's `parseOverrides(overrides, catalogs)` →
`createOverridesMapFromParsed` pipeline: thread `&Catalogs` through
`parse_overrides[_iter]`, resolve each value via `resolve_from_catalog`,
and flatten the resolved entries into the map handed to
`check_lockfile_settings`.
The `pacquet --version` string is a hardcoded clap attribute in
`cli_args.rs`. It didn't get bumped for the 0.2.2 release, so the
published binary still reports 0.2.1. Bump the literal to 0.2.2 and
add a release-workflow step that rewrites the attribute from
`inputs.version` before `cross build`, so future releases stay
correct automatically. A trailing `grep -F` fails the job loudly if
the regex stops matching after a future refactor of the attribute.
* fix(installing.commands): forward `pnpm install` flags to pacquet
When the install engine is delegated to pacquet via configDependencies,
pnpm hard-coded the args to `install --frozen-lockfile --reporter=ndjson`
and silently dropped the user's other CLI flags. `pnpm install --no-runtime`
therefore still installed the workspace's runtime devDependency, clobbering
the Node version the surrounding tooling had set up — visible as the
`Verify Node version` failure on PR #11765 where setup-pnpm provisions
Node 24.0.0 but pacquet then materializes node 24.6.0.
Pacquet's `install` subcommand already mirrors pnpm's surface for the
common flags (`--no-runtime`, `--prod`, `--dev`, `--no-optional`,
`--node-linker`, `--offline`, `--prefer-offline`, `--cpu`/`--os`/`--libc`).
Forward the user's argv verbatim when the command is `install`/`i`;
`add`/`update`/`dedupe` still don't forward — their flag surfaces don't
line up with pacquet's `install`.
* fix(installing.commands): pass --ignore-manifest-check to pacquet
`pnpm up` / `add` / `remove` were aborting with
`pacquet_package_manager::outdated_lockfile` whenever pacquet was
declared in `configDependencies`. After resolving and writing the
updated lockfile, pnpm hands materialization off to pacquet but
hasn't yet written the post-mutation `package.json` — that write
happens after `mutateModules` returns. Pacquet's frozen-lockfile
freshness gate then saw the new lockfile paired with the
pre-mutation manifest and refused to install.
Pass pacquet's new `--ignore-manifest-check` flag (pacquet PR #11811)
on every delegation. The flag is narrow: it only skips
`satisfies_package_manifest`. Settings drift like `overrides` is
still enforced, and pnpm already re-validated the lockfile before
delegating, so re-checking the manifest here was redundant work that
only ever fired false positives on the mutate-then-materialize path.
Requires a pacquet release that ships the flag; bump
`PACQUET_VERSION` in `pnpm/test/install/pacquet.ts` once it does, or
the existing e2e tests will fail against pacquet 0.2.2-9 (which
doesn't recognize the flag and clap would reject).
Closes#11797.
---
Written by an agent (Claude Code, claude-opus-4-7).
* fix: update pacquet in tests
* fix(installing.commands): strip positionals + always-injected flags when forwarding to pacquet
`collectForwardedFlags` checked `argv[0] === 'install'` to find the
command token to strip. Any global flag the user typed before `install`
(e.g. `--config.registry=...` in the e2e test) shifted the token out
of position, so the function returned the full argv and pacquet saw
`install` twice — `error: unexpected argument 'install' found`.
Use the parsed argv that `@pnpm/cli.parse-cli-args` already produced:
`remain` lists positionals (the `install`/`i` token and nothing else
on this code path, since `isInstallCommand` is only true when no
package params are present), and `original` preserves the user's
exact tokens. Drop positionals + the flags we always inject
(`--reporter=ndjson`, `--frozen-lockfile`, `--ignore-manifest-check`)
so clap doesn't reject duplicates either.
`original` over `cooked` deliberately: nopt's `cooked` splits
`--key=value` into two tokens, which would break pacquet's
`--config.<key>=<value>` parser (it requires the `=` form).
* fix(installing.commands): make argv.cooked/remain optional on InstallCommandOptions
Widening these to required broke test fixtures elsewhere (publish/pack/
deprecate/dist-tag/deploy) that construct minimal `argv: { original }`
options for code paths that never reach pacquet. Only the pacquet
delegation actually reads `remain`, so make the two new fields optional
on the shared options type and supply a default at the runPacquet call
site. The runtime path through main.ts already populates all three.
* fix(installing.commands): strip any user-supplied --reporter when forwarding to pacquet
Pacquet's `--reporter` is a clap value option with last-value-wins
semantics, so `pnpm install --reporter=silent` (or
`--reporter silent` two-token form) reached pacquet and overrode
the `--reporter=ndjson` pnpm injects, breaking the NDJSON-to-
streamParser plumbing the default reporter depends on. The previous
filter only matched the exact `--reporter=ndjson` token.
Walk argv with a lookahead so both `--reporter=<value>` and
`--reporter <value>` are dropped without consuming an adjacent flag.
* fix(installing.commands): drop negated/value forms of always-injected flags
`collectForwardedFlags` only matched the exact positive tokens
`--frozen-lockfile` and `--ignore-manifest-check`, so a user typing
`pnpm install --no-frozen-lockfile` (or `--frozen-lockfile=false`)
forwarded the negation to pacquet, which then saw both our injected
`--frozen-lockfile` and the user's `--no-frozen-lockfile` and crashed
clap with "unexpected argument".
Match every shape the user can write the same flag in: positive,
`--no-` negated, and any `=value` form. Can't blindly strip `--no-`
either way — pacquet has flags whose literal name starts with `no-`
(`--no-runtime`, `--no-optional`); those must still forward.
The user's `--no-frozen-lockfile` intent is honored upstream — pnpm
did a fresh resolve before delegating; pacquet's role here is just
lockfile-driven materialization, which is always frozen.
* fix(installing.commands): match positionals by index, hide reporter from dropped-flags warning
`collectForwardedFlags` matched positionals via `new Set(argv.remain)`,
which strips by value: a flag value that happened to equal a
positional token (e.g. `pnpm install --node-linker install`) was
wrongly dropped from the forwarded list, costing pacquet the value
of `--node-linker`. Walk `argv.original` with a subsequence pointer
into `argv.remain` so only the actual positional indexes get skipped.
`collectDroppedFlags` still surfaced `--reporter foo` / `--reporter=foo`
in the "may not be honored" warning on `add`/`update`/`dedupe`, but
pnpm honors reporter selection itself before delegation — so the
warning was misleading. Route both helpers through the same
`isAlwaysInjected` check and consume `--reporter` and its value the
same way `collectForwardedFlags` already does.
Surfaces a narrow CLI flag on `pacquet install` that gates only
`satisfies_package_manifest`. Settings-drift checks (`overrides`,
`ignoredOptionalDependencies`, …) still fire, and the broader
`--ignore-package-manifest` name is reserved for a future port of
pnpm's `pnpm fetch` semantics (which skip linking / hoisting /
pruning too).
Intended for the pnpm CLI's `configDependencies` delegation path
(issue #11797): pnpm resolves and writes the lockfile, then hands
materialization to pacquet but hasn't yet written the post-mutation
`package.json`. With the flag set, the freshness gate skips the
per-importer manifest check that would otherwise reject every
`pnpm up` / `add` / `remove` with `ERR_PNPM_OUTDATED_LOCKFILE`. The
matching pnpm-side change to forward the flag lands separately.
Refs #11797.
---
Written by an agent (Claude Code, claude-opus-4-7).
