mirror of
https://github.com/pnpm/pnpm.git
synced 2026-10-09 14:52:53 -04:00
0cefccf15839bd69dc7a3db2c8adaa85bb32c87e
11642
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zoltan Kochan
|
0cefccf158 |
feat(registry): persist pnpr users and tokens to disk (#11977)
* feat(registry): persist pnpr users and tokens to disk
Backs UserStore with a verdaccio-shaped htpasswd file (bcrypt $2y$
hashes, atomically rewritten on every adduser) and TokenStore with a
SQLite database that stores SHA-256 token hashes plus the per-record
fields the upcoming /-/npm/v1/tokens surface will need (created_at,
last_used_at, readonly, cidr_whitelist).
Configuration mirrors verdaccio's auth.htpasswd.{file,max_users}
under the existing YAML schema; tokens default to a tokens.db
sibling of htpasswd, overridable via auth.tokens.file. max_users=-1
disables registration end-to-end. Both files are written via
tmp+rename and loaded eagerly on startup so a malformed htpasswd
fails fast rather than booting with a silent empty user list.
Closes #11974.
* fix(registry): use OS CSPRNG, satisfy dylint + rustdoc
- TokenStore's per-process secret now comes from getrandom (OS-backed
CSPRNG) instead of time/pid/stack address. Tokens are derived from
this secret + a per-issue nonce, so weak entropy was making mint
outputs guessable to an attacker who could bound those inputs.
- Reorder derives on AuthConfig / HtpasswdConfig / TokensConfig /
MaxUsers to satisfy perfectionist::derive-ordering (prefix-then-
alphabetical: Debug, Default first, then the rest).
- Re-export auth::identify so the rustdoc link from the now-public
UserStore::verify resolves; rustdoc::private-intra-doc-links no
longer fails the workspace doc build.
- Drop the inaccurate "+inf" mention from MaxUsers' doc — serde-saphyr
treats +inf as a float and can't deserialize it into i64, so the
only way to get Unlimited is to omit max_users.
|
||
|
Zoltan Kochan
|
a33c4bfcb0 |
perf: skip resolution when only pnpm-lock.yaml is missing (pnpm + pacquet) (#12004)
* perf: skip resolution when only pnpm-lock.yaml is missing
When pnpm-lock.yaml is absent but node_modules/.pnpm/lock.yaml exists and still
satisfies the manifest, reuse the materialized snapshot to regenerate the
wanted lockfile instead of walking the registry to rebuild it. Closes the
cache+node_modules variation gap in the vlt.sh benchmarks for the pnpm CLI
side; the pacquet port is tracked separately at #11993.
`--frozen-lockfile` still fails when pnpm-lock.yaml is absent: the regenerated
file must be committed, so failing loudly is the correct behavior for CI.
* perf(pacquet): port the cache+node_modules shortcut
When `pnpm-lock.yaml` is absent but `node_modules/.pnpm/lock.yaml` exists
and still satisfies the manifest, synthesize the wanted lockfile from the
materialized snapshot and take the frozen-install path. The install skips
resolution and regenerates `pnpm-lock.yaml` from the synthesized object.
Mirrors the pnpm-side change at
|
||
|
Philip Roberts
|
c94b4f89c7 |
fix: publish with default access (#11991)
* fix: preserve default publish access * chore(publish): add changeset |
||
|
Zoltan Kochan
|
72d997cc34 | chore(release): 11.4.0 (#11989) v11.4.0 | ||
|
Zoltan Kochan
|
b73908f088 | chore: update pnpm-lock.yaml (#11897) | ||
|
Zoltan Kochan
|
aa6149df65 |
fix: fail by default when a tarball does not match the locked integrity (#11968)
`pnpm install` (non-frozen) used to react to `ERR_PNPM_TARBALL_INTEGRITY` by logging the error, silently re-resolving from the registry, and overwriting the locked integrity. The lockfile's integrity was effectively advisory by default — a compromised registry, proxy, or republished version could substitute attacker-controlled content on a clean machine even though the project shipped a committed `pnpm-lock.yaml`. Integrity mismatches against the lockfile now fail by default. The **only** opt-in is **`pnpm install --update-checksums`** — a new flag, narrowly scoped to refreshing the locked integrity values. Mirrors yarn's flag of the same name. A warning still prints when the bypass takes effect so the rewrite stays auditable. `--force` and `pnpm update` deliberately do **not** bypass the integrity check. They are routine refresh operations; silently overwriting a locked integrity in those flows would erase the protection a committed lockfile is supposed to provide. `--frozen-lockfile` behavior is unchanged. `--fix-lockfile` keeps its documented purpose (filling in missing lockfile entries) and is also not a bypass. Combining `--frozen-lockfile` with `--update-checksums` errors out — frozen mode refuses to rewrite the lockfile, which is exactly what `--update-checksums` is for. `--update-checksums` also bypasses the resolver's on-disk metadata cache fast path (`pickPackage.ts:271`, `pick_package.rs:531`). Without that, a stale on-disk packument that already contained the pinned version would short-circuit the registry entirely and the flag would silently no-op on dev machines. With the gate, every first-encounter goes through a conditional GET; the in-memory cache is left alone so second-and-onward references within the same install still hit cached fresh data (one network round-trip per *unique* package, not per reference). ## Reported by Reported privately via the security channel. The reproduction: 1. Publish `example-package@1.0.0` with content `v1` and install with pnpm; lockfile records the `v1` integrity. 2. Replace the registry's tarball+metadata for the same `1.0.0` with content `v2`. 3. On a clean store/cache, run `pnpm install`. Before this fix, pnpm logged `ERR_PNPM_TARBALL_INTEGRITY` but exited 0 with `v2` installed and the lockfile rewritten to the new integrity. After this fix, the same install exits non-zero. ## Prior art - **npm** ([sebhastian](https://sebhastian.com/npm-err-code-eintegrity/)): hard-fails with `EINTEGRITY`. No dedicated override flag — recovery is `npm cache clean --force`, manually editing the lockfile, or deleting it. - **yarn** ([Sean C Davis](https://www.seancdavis.com/posts/fix-yarn-integrity-check-failed/)): hard-fails with "Integrity check failed". Has a dedicated **`yarn install --update-checksums`** flag — pnpm now adopts the same name. ## Pacquet parity Pacquet was already fail-hard on integrity mismatch by default (no auto-repair path to remove). This PR brings the rest of the surface into line so `pnpm install --update-checksums` keeps working when pacquet is the materialization target, and `pacquet install --update-checksums` behaves identically standalone: - New `--update-checksums` flag on `pacquet install` (`crates/cli/src/cli_args/install.rs`), plumbed through `Install` and `InstallWithFreshLockfile` into the resolver. - When the flag is set, pacquet skips the frozen-lockfile fast path and routes through the fresh-resolve path so locked integrity values get rewritten from the registry. - `--frozen-lockfile + --update-checksums` errors with `pacquet_package_manager::frozen_lockfile_with_outdated_lockfile`, mirroring pnpm's `ERR_PNPM_FROZEN_LOCKFILE_WITH_OUTDATED_LOCKFILE`. - `pacquet_tarball::verify_checksum_error` now carries a help hint pointing at `--update-checksums` and calling out the supply-chain implication, matching the updated pnpm `TarballIntegrityError`. - The disk fast-path gate is mirrored in `crates/resolving-npm-resolver/src/pick_package.rs:531`, with the flag threaded from `ResolveOptions` → `PickPackageOptions`. |
||
|
Zoltan Kochan
|
c12681f68d |
docs(registry): flesh out @pnpm/pnpr README (#11972)
* docs(registry): flesh out @pnpm/pnpr README Document install, default behavior, CLI flags, RUST_LOG, and a minimal verdaccio-shaped YAML config example. * docs(registry): reword pnpr intro Drop the "runs locally / verdaccio-like" framing — pnpr is hostable and not scoped to a local-dev role. * chore(cspell): add packuments, refetched |
||
|
|
f03dc2d15d |
refactor(registry): adopt verdaccio-shaped YAML config (#11970)
Reshape pnpm-registry's Config to match verdaccio's `config.yaml` schema (storage, uplinks, packages) so the same file can drive either server. The previous Config exposed a single `upstream: Option<String>` resolved at startup; this replaces it with named uplinks plus per-package `proxy:` rules walked in declared order — same semantics as verdaccio, minus the surface pnpm-registry does not implement (auth, web, plugins, middlewares, logs routing, secret), which are accepted and ignored.
Highlights:
* `Config { listen, public_url, storage, uplinks, packages, packument_ttl }` with `UplinkConfig { url }` and `PackageAccess { access, publish, unpublish, proxy }`. `packages` is an `IndexMap` walked in declared order, first-match-wins: the first pattern matching a request is the rule that applies, and if that rule has no `proxy:` the package is storage-only (resolution returns `None` instead of falling through to a later catch-all). That makes the bundled `@private/*` / `@pnpm.e2e/needs-auth` / unscoped-fixture rules behave the way the YAML says they should.
* `Config::from_yaml(path, ...)` loads via `serde-saphyr` and resolves a relative `storage:` against the config file's parent. The verdaccio-only sections in the YAML are skipped silently so `registry-mock`'s upstream `config.yaml` parses untouched.
* `DEFAULT_CONFIG_YAML` — the bundled file mirrored from `@pnpm/registry-mock` — is `include_str!`-ed and re-exported from `lib.rs` so other crates (tests, benchmarks, future embedders) can use the same defaults without reading from disk. `Config::from_default_yaml(base_dir, ...)` parses it.
* CLI: `-c` / `--config <path>` overrides the bundled default. `--storage` survives as a runtime override (handy for tests without a custom YAML); `--upstream` and `--static` are gone because the YAML now drives both. `--packument-ttl-secs` is optional — the loaded config's value wins when the flag is not supplied. The mock orchestrator at `pacquet/tasks/registry-mock` drops its `--upstream` flag — that command spawns the locally-built binary so it tracks this PR's source directly. The jest harness at `__utils__/jest-config/with-registry/globalSetup.js` keeps `--upstream` for now because CI installs `@pnpm/pnpr` from npm; the flag will be dropped in the same PR that bumps `@pnpm/pnpr` to a build that lacks it.
* `server.rs` pre-builds one `Upstream` per declared uplink at router construction (keyed by name, in an `IndexMap`) and resolves the right client per request via `Config::resolve_uplink`. No per-request `ThrottledClient` allocations.
Existing tests are kept working by retaining the `Config::proxy` / `Config::static_serve` constructors and switching the test helper from `config.upstream = ...` to mutating
`config.uplinks["npmjs"].url`. All 107 tests in `pnpm-registry` pass (55 unit + 26 + 9 + 17 integration).
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **New Features**
* YAML-based registry configuration with package-level routing, multiple uplinks, auth and audit middleware
* New --config option to load custom registry configs; optional storage override and packument TTL setting
* **Chores**
* Default registry now uses the bundled configuration (web UI disabled by default)
* Configuration refactor to support Verdaccio-style routing patterns and per-package access/publish rules
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Zoltan Kochan <z@kochan.io>
|
||
|
Zoltan Kochan
|
e55f4b5efd |
fix: require integrity for tarball-shaped lockfile resolutions (#11966)
* fix(lockfile.utils): require integrity for tarball-shaped lockfile resolutions A tampered lockfile that strips the `integrity` field from a tarball resolution let the worker download the URL contents and mint a fresh integrity from the unverified bytes, so an attacker who could also serve content at the referenced URL would install a tampered package without any error — including under `--frozen-lockfile`. pnpm now rejects such entries at lockfile-read time with `ERR_PNPM_MISSING_TARBALL_INTEGRITY`, matching pacquet's existing `pacquet_package_manager::missing_tarball_integrity` guard. * test(lockfile.utils): drop redundant integrity-less snapshot that fails strict typecheck * test(pacquet/package-manager): cover MissingTarballIntegrity rejection in snapshot_cache_key Match the upstream guard landed alongside pnpm/pnpm#11966 (`lockfile/utils/src/pkgSnapshotToResolution.ts`) with a test on the pacquet side: a `LockfileResolution::Tarball` with `integrity: None` — what a tampered lockfile looks like — must short-circuit the warm-batch cache-key derivation by surfacing `InstallPackageBySnapshotError::MissingTarballIntegrity`. The structural guard already existed but had no negative test. * fix(lockfile.utils): exempt git-hosted and file: tarballs from the integrity guard The strict guard added in the parent commit broke pnpm's own `with-git-protocol-dep` and `with-non-package-dep` fixtures: the install pipeline writes git-hosted tarball entries (codeload.github.com URLs) to the lockfile without an `integrity:` line, because the commit SHA in the URL is the integrity anchor — git's content-addressed model binds the bytes to the commit, so a separate hash adds nothing. Exempt git-hosted tarballs (detected either via the `gitHosted: true` flag or a URL on the known git hosts, matching the URL fallback in `toLockfileResolution`) and `file:` tarballs (local paths the user already controls). The strict check still fires for any other remote tarball — which is where the AutoFyn-reported vector actually manifests. Also export `isGitHostedTarballUrl` from `toLockfileResolution.ts` so the URL fallback can be shared rather than duplicated. * test(pacquet/package-manager): trim doc comment to the contract-level intent Per the repo convention that tests are documentation, the test name and body already cover what's being asserted; the prior comment duplicated that. Keep only the non-obvious why: why this guard exists at the cache-key site at all (warm-batch short-circuit) when the install-side check also rejects the same input. |
||
|
Zoltan Kochan
|
90d1ce6b60 |
fix(git-fetcher): reject non-SHA commit values before invoking git (#11967)
`fetching/git-fetcher/src/index.ts` passed the lockfile-controlled `resolution.commit` value straight to `git fetch --depth 1 origin <commit>` and `git checkout <commit>` with no `--` separator and no format validation. A malicious `pnpm-lock.yaml` could put a value such as `--upload-pack=touch /tmp/pwned` in `resolution.commit`; `git` parses anything starting with `-` as an option, and on SSH or local-file transports `--upload-pack` runs the supplied command as the user running `pnpm install`. HTTPS ignores `--upload-pack`, but the SSH/file paths are enough to reach code execution.
The fix validates `resolution.commit` against `/^[0-9a-f]{40}$/i` at the entry of the fetcher and throws `INVALID_GIT_COMMIT` otherwise. This is strictly stronger than adding a `--` separator — a validated value cannot start with `-` or contain shell-significant characters at all.
Pacquet's `pacquet-git-fetcher` crate shells out to `git` along the same code path (`pacquet/crates/git-fetcher/src/fetcher.rs`) and had the identical issue. Ported the same check there, with a new `GitFetcherError::InvalidCommit` variant carrying the `INVALID_GIT_COMMIT` diagnostic code.
Reported by [AutoFyn](https://github.com/SignalPilot-Labs/AutoFyn).
|
||
|
Zoltan Kochan
|
89c2b52728 |
ci(pacquet): pin cargo-dylint to 6.0.0 to unblock CI (#11969)
trailofbits/dylint 6.0.1 (published 2026-05-26 17:51 UTC) ships
prebuilt cargo-binstall artifacts that bake in a path from the
dylint repo's own CI workspace:
error: failed to get `dylint_driver` as a dependency of package
`dylint_driver-nightly-2026-04-16-x86_64-unknown-linux-gnu`
Caused by:
failed to read `/home/runner/work/dylint/dylint/driver/Cargo.toml`
Caused by:
No such file or directory (os error 2)
Downstream runners don't have that workspace, so the driver bootstrap
fails before any lint runs and the Dylint job goes red on every PR.
6.0.0 (the version main was passing with 90 minutes earlier) is
unaffected. Pin both binaries until upstream cuts 6.0.2.
---
Written by an agent (Claude Code, claude-opus-4-7).
|
||
|
Zoltan Kochan
|
f107b1e13a |
ci(test): install @pnpm/pnpr from npm instead of building locally (#11964)
The e2e/integration test harness spawns `pnpm-registry` as a faster verdaccio replacement. CI used to install Rust and build the crate from source on every test job — adding several minutes per platform. `@pnpm/pnpr` now publishes the prebuilt binary to npm, and `pnpm install` already pulls in the matching `@pnpm/pnpr.<platform>-<arch>` package via optionalDependencies. The Jest globalSetup resolves that binary through `@pnpm/pnpr/bin/pnpr`'s own module path (the wrapper carries the platform packages as siblings in its `node_modules`, not on the parent chain of this file). - Add `@pnpm/pnpr` to `pnpm-workspace.yaml` catalog and depend on it from `@pnpm/jest-config`. - Replace `resolvePnpmRegistryBin`'s `$CARGO_TARGET_DIR` lookup with `require.resolve` through the npm-installed wrapper. The `PNPM_REGISTRY_BIN` env var is still honored as an escape hatch for contributors who want to point at a locally-built Rust binary. - Remove the "Install Rust toolchain" + "Build pnpm-registry" steps from `.github/workflows/test.yml`. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
0f5f338a89 | chore: add pnpr to cspell | ||
|
KhảiandClaude
|
f578281d2a |
feat(pacquet/config): --recursive, --workspace-concurrency (#11959)
* feat(pacquet/config): port --recursive and --workspace-concurrency settings Mirror pnpm's `workspaceConcurrency` (a `.npmrc` / `pnpm-workspace.yaml` / `PNPM_CONFIG_WORKSPACE_CONCURRENCY` config-file key, default `getDefaultWorkspaceConcurrency()`, resolved through `getWorkspaceConcurrency`) and CLI-only `recursive` (`-r`) boolean. - Add `Config::workspace_concurrency` (resolved via the existing `resolve_child_concurrency` port of `getWorkspaceConcurrency`) and `Config::recursive`, plus `default_workspace_concurrency`. - Read `workspaceConcurrency` from workspace yaml, global config.yaml, and the `PNPM_CONFIG_*` env overlay; resolve negative offsets the same way `childConcurrency` does. - Add the `--workspace-concurrency` install flag (overrides the config-resolved value) and the global `-r` / `--recursive` flag (sets `Config::recursive`, matching pnpm's CLI-only nature). `workspaceConcurrency` is parsed and stored for config-surface parity; pacquet's frozen install materializes the whole workspace in one shared pass, so there is no per-project parallel loop for it to throttle yet (same posture as `preferOffline`). `recursive` is likewise a surface flag on install today, since install already spans the workspace. * docs(pacquet/config): drop private intra-doc link to fix Doc CI `default_workspace_concurrency`'s public doc linked the crate-private `default_child_concurrency`, which `rustdoc -D rustdoc::private-intra-doc-links` (implied by `-D warnings`) rejects. Use plain backticks instead. * style(pacquet/cli): drop redundant comment on workspace_concurrency destructure bind The sibling `offline: _` / `prefer_offline: _` binds carry no comment; match them for consistency. * test(pacquet/cli): cover the --workspace-concurrency override resolution The inline `if let Some(value) = args.workspace_concurrency` override body was only reachable when install ran *with* the flag, so the flag-absent integration runs left it uncovered. Extract the resolution into `InstallArgs::resolve_workspace_concurrency` and apply it unconditionally at the dispatch (matching upstream's final `getWorkspaceConcurrency` pass), so the substantive logic is covered by fast unit tests (absent / positive / negative) and the call site is an unconditional assignment the existing install tests already exercise. * style(pacquet/cli): pipe-trait the workspace-concurrency test parse calls Per review: rewrite the `InstallArgsHarness::try_parse_from([...])` calls in the new workspace-concurrency tests as `[...].pipe(InstallArgsHarness::try_parse_from)` so they read left-to-right, matching the codebase's pipe-trait convention. * docs(pacquet/config): drop inaccurate .npmrc source from workspaceConcurrency docs workspace_concurrency is populated only from pnpm-workspace.yaml, global config.yaml, and the PNPM_CONFIG_WORKSPACE_CONCURRENCY env overlay. Config::current reads .npmrc but applies only the auth/network subset, so a workspace-concurrency= entry in .npmrc never reaches the field. Align the doc comments and PR summary with actual behavior, matching the sibling childConcurrency docs. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
c42f69ffaf |
ci(pnpr): add manual release workflow that publishes @pnpm/pnpr to npm (#11963)
* ci(pnpm-registry): add manual release workflow that publishes to npm Mirror pacquet's release pipeline for the pnpm-registry crate: - New `Release pnpm-registry` GitHub workflow (manual `workflow_dispatch` with a version input) builds the `pnpm-registry` binary for six `<os>-<cpu>` matrix legs via `cross`, attests provenance, and uploads artifacts. - The publish job patches `registry/npm/pnpm-registry/package.json`'s `version` field with the input, downloads the artifacts, runs `generate-packages.mjs` to produce per-platform packages, then publishes everything via `pnpm publish --provenance` using OIDC. - The wrapper package is `pnpm-registry`; the per-platform binary packages are scoped `@pnpm/registry.<os>-<cpu>` and resolved by a Node shim under `bin/pnpm-registry`. The CLI's `--version` output comes from `CARGO_PKG_VERSION` via clap's derive `version` attribute, so the build leg patches the crate's `Cargo.toml` `version` field rather than a hardcoded clap string. --- Written by an agent (Claude Code, claude-opus-4-7). * ci(pnpr): rename package to @pnpm/pnpr with @pnpm/pnpr.<os>-<cpu> binaries Shorter, scoped name that reads naturally on npm: `@pnpm/pnpr` for the wrapper, `@pnpm/pnpr.<os>-<cpu>` for the six per-platform binary packages (mirroring `@pacquet/<os>-<cpu>` under a different scope). - Move `registry/npm/pnpm-registry/` to `registry/npm/pnpr/` and rename the JS shim and `bin` entry to `pnpr`. - Update `generate-packages.mjs` so `BIN_NAME = pnpr` (used for both the binary file inside each platform package and the autogenerated package dir) and the scope/prefix produce `@pnpm/pnpr.<os>-<cpu>`. - Rename the workflow file to `pnpr-release-to-npm.yml` and rename the built `pnpm-registry` binary to `pnpr-<code-target>` at archive time, so `generate-packages.mjs` finds it at `REPO_ROOT/pnpr-<plat>-<arch>`. - The Rust crate name stays `pnpm-registry` (still what `cross build -p` targets); only the npm-publishing layer is renamed. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(pnpr): propagate signal exits through the Node shim `spawnSync` returns `result.status === null` when the child terminates via a signal. Assigning that to `process.exitCode` makes the parent exit 0 and masks the signal — bad for a long-running server where the operator is most likely to kill it with SIGINT/SIGTERM. Re-raise the signal so the parent terminates the same way, falling back to a non-zero exit code if for some reason we can't. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
ad84fffd46 |
fix: reject path-traversal segments in dependency aliases (#11954)
* fix: reject path-traversal segments in dependency aliases A transitive registry package can use a dependency-alias key like `@x/../../../../../.git/hooks` to make `pnpm install` create a symlink outside the intended `node_modules` directory, since pnpm passes the alias straight into `path.join(modulesDir, alias)` without checking that the joined path stays inside `modulesDir`. Reject aliases that aren't a single `name` or `@scope/name` shape at manifest-read time (both the importer's manifest and every transitive package manifest) and re-check at the symlink layer as defense in depth. Mirror the fix in pacquet's deps-resolver. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(pacquet): use raw strings in alias validator tests for dylint Perfectionist's `prefer-raw-string` lint rejects the two backslash-escaped test inputs. --- Written by an agent (Claude Code, claude-opus-4-7). * refactor: tighten dependency-alias validator to validate-npm-package-name An alias is the directory name pnpm creates inside `node_modules`, so the only valid shapes are a single `name` or `@scope/name` consisting of URL-friendly characters with no leading `.` / `_`, and not equal to reserved names such as `node_modules`. That's the same `validForOldPackages` rule `parseWantedDependency` already applies to CLI-given names — the manifest-read path should match. Route both stacks through it so `.bin`, `.pnpm`, `node_modules`, `favicon.ico`, whitespace, and non-URL-friendly characters are all rejected alongside the path-traversal shapes the narrow validator caught. --- Written by an agent (Claude Code, claude-opus-4-7). * refactor: collapse symlink-layer assertion + path.join into safeJoinModulesDir The two-step pattern of "assert the alias stays in the dir" then "join the dir and the alias" left it possible for a caller to use the join without the assertion. Fold them into a single `safeJoinModulesDir` that returns the joined path and throws on escape, so the check is unmissable. --- Written by an agent (Claude Code, claude-opus-4-7). * test(symlink-dependency): cover the path-equals-dir guard branch The earlier tests only exercised the `!startsWith` branch with `'../sibling'` and `'@x/../../../etc'`. Add `''` and `'.'` as alias cases — both resolve to the modules dir itself and hit the `resolvedLink === resolvedDir` branch of `safeJoinModulesDir`. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
a23956e3ab |
fix(config/reader): pin unscoped per-registry settings to their source's registry at load time (#11953)
* fix(config/reader): drop user-level default auth when workspace overrides registry
When a workspace `.npmrc` overrides `registry=` to a different value than the
user's `~/.npmrc` or `~/.config/pnpm/auth.ini` would have set, do not bind
unscoped/default credentials (`_authToken`, `_auth`, `username`/`_password`)
from the user-level config to the workspace-selected registry. The previous
behavior leaked user-trusted credentials to whatever registry an untrusted
workspace `.npmrc` pointed at. Reported by JUNYI LIU.
* chore(cspell): allow JUNYI in changeset and tests
* fix(config/reader): also defend when pnpm-workspace.yaml overrides registry
Move the rebind defense to after all config layers (CLI, env vars,
pnpm-workspace.yaml, .npmrc) have settled. Compare the final resolved
default registry against what the user-level config alone would produce,
and skip the check entirely if the user requested a registry via CLI/env
themselves.
* feat(config/reader): deprecate unscoped authentication credentials
Emit a per-file warning whenever an .npmrc or auth.ini contains an
unscoped auth value (_authToken, _auth, username, _password,
tokenHelper). URL-scoped tokens have been npm's recommended pattern
since npm@9, and unscoped credentials are slated for removal in a
future major. The warning fires independently of whether the rebind
defense rejects the credentials, so users see the deprecation even when
their setup happens to be safe today.
* refactor(config/reader): rescope unscoped credentials at load time instead of detecting rebinds post-merge
Each .npmrc / auth.ini / CLI source's unscoped credential keys
(_authToken, _auth, username, _password, tokenHelper) are rewritten to
their URL-scoped equivalent during load, using the same source's
registry= value (or the npmjs default if it declares none). A later
layer overriding registry= can no longer rebind a credential to its own
registry — the credential is already pinned to the URL its author
intended.
This removes the post-merge source-tracking defense and replaces it
with the simpler per-source normalization. Each rescope emits a
deprecation warning so users migrate to writing the URL-scoped form
directly.
* refactor(network/auth-header): drop empty-string default-registry slot
After load-time rescoping, no source can populate configByUri[''] —
every credential is either URL-scoped from the start or rewritten to
the URL-scoped form during the .npmrc / auth.ini / CLI parse. The
runtime fallback that re-keyed configByUri[''] onto the merged default
registry, and the publish-side fallback that read it, are both dead
code.
Removed:
- empty-string handling in getAuthHeadersFromCreds, including its
defaultRegistry parameter
- defaultRegistry parameter from createGetAuthHeaderByURI
- the corresponding dedicated unit test
- the configByUri['']?.creds fallback in publishPackedPkg.ts
- empty-key assertions in config/reader tests
Updated all ~16 call sites of createGetAuthHeaderByURI to drop the now
unused second argument.
* feat(config/reader): extend per-source rescoping to client TLS cert/key
The same trust-boundary issue that affected unscoped credentials applies
to client TLS settings: an unscoped cert=/key= would be presented to
whatever registry the merged config settles on, even if a later layer
(workspace .npmrc, pnpm-workspace.yaml, CLI flag) overrode it. The
existing rescope helper now also rewrites unscoped `cert` and `key`
to their URL-scoped form, pinning them to the registry their author
named in the same source.
`ca`/`cafile` are intentionally left unscoped: they're trust anchors,
not credentials, and corporate MITM-proxy setups depend on them
applying to every HTTPS request. The default-registry override can't
weaponize an unscoped CA — the attacker would need a cert signed by it.
`certfile`/`keyfile` (file-path variants) are not rescoped either:
`certfile` isn't read unscoped by pnpm today (asymmetric vs. `keyfile`
in NPM_AUTH_SETTINGS), and supporting only one of them would be
confusing. Users wanting the path form can write it URL-scoped
directly.
* chore(config/reader): remove dead unscoped `keyfile` allowlist entry
`keyfile` was listed in NPM_AUTH_SETTINGS so unscoped `keyfile=<path>`
passed the .npmrc filter and ended up in authConfig — but nothing in
the codebase ever read it from there. The dispatcher uses `opts.key`
(inline PEM) and `configByUri[host].tls.key` (URL-scoped path/inline
content), neither of which is populated from unscoped `keyfile=`.
`certfile` was already absent from the allowlist for the same reason,
so this also removes the asymmetry between the two file-path variants.
URL-scoped `//host/:certfile=...` and `//host/:keyfile=...` continue
to work via `tryParseSslKey` and are unaffected.
* test(network/auth-header): drop test for removed default-registry slot
This test exercised the configByUri[''] re-keying path that was
removed in the rescope-at-load refactor. With createGetAuthHeaderByURI
no longer accepting a defaultRegistry parameter and unscoped
credentials no longer reaching the merged config, the scenario the
test described is structurally unreachable.
* fix(config/reader): handle empty/invalid registry value in rescope
Two CI fixes:
1. When a source's `registry=` resolves to an empty string (e.g. an
unresolved `${ENV_VAR}` placeholder), `new URL(...)` inside
`nerfDart` throws. Guard the call with try/catch: drop the
unscoped per-registry keys (a bare token has nowhere safe to bind)
and emit a warning naming the offending source.
2. Update `.npmrc does not load pnpm settings` to expect the rescoped
form of unscoped `_authToken`/`username` in `authConfig` — they
now appear as `//registry.npmjs.org/:_authToken` etc. since the
test's .npmrc declares no `registry=` of its own.
* chore(cspell): allow "rescoping"
* test(installing/deps-installer): drop "legacy way" auth test
This test passed credentials via the configByUri[''] empty-string slot,
which the auth-header layer re-keyed to the merged default registry at
request time. That slot was removed in the rescope-at-load refactor —
credentials are now always URL-scoped before they reach configByUri,
so the empty-key entry is unreachable from any code path.
The scenario the test covered (basicAuth via username/password) is
already exercised by the existing "installing a package that need
authentication, using password" test using the URL-scoped form.
|
||
|
Zoltan Kochan
|
0c5b66f1ea |
fix(pacquet/integrated-benchmark): use --ignore-failure singular hyperfine flag (#11960)
Hyperfine's flag is `--ignore-failure` (singular). The Rust harness was passing `--ignore-failures` (plural), so the benchmarks CI job aborted with `unexpected argument '--ignore-failures'` once a hyperfine version that validates unknown args strictly was on the runner. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
a662de44dd |
fix: pnpm runtime set defaults to devEngines (#11951)
* fix: pnpm runtime set defaults to devEngines Previously `pnpm runtime set <name> <version>` wrote to `engines.runtime` because it ran `pnpm add` with the default `--save-prod`. Default to `--save-dev` so the runtime lands in `devEngines.runtime`; pass `--save-prod` (or `-P`) to opt back into `engines.runtime`. Closes #11948 * fix: honor --save-dev precedence in pnpm runtime set When both `--save-dev` and `--save-prod` are passed, prefer `--save-dev` to match `getSaveType`'s precedence elsewhere in pnpm. Also makes the explicit `--save-dev` flag actually consulted instead of relying solely on the default branch. * ci: trigger |
||
|
Zoltan Kochan
|
26a7d633bf |
fix(patching/apply-patch): reject patch paths that escape the patched directory (#11952)
* fix(patching/apply-patch): reject patch paths that escape the patched directory A malicious .patch file with `diff --git a/../../X` headers could otherwise write, delete, or rename files outside the patched package as the user running `pnpm install`. * refactor(patching/apply-patch): narrow caught errors via util.types.isNativeError Drops the `any`-typed catch + eslint-disable in favor of the cross-realm-safe narrowing pattern documented in CLAUDE.md. * refactor(patching/apply-patch): replace error helper with PatchPathEscapesError class * chore(patching/apply-patch): reword comment to satisfy cspell |
||
|
|
35d235542e |
fix: validate devEngines runtime onFail (#11822)
Fixes #11818 ## Summary `devEngines.runtime` / `engines.runtime` entries with `onFail: error` or `warn` silently did nothing — only `onFail: download` had any effect. This PR wires up validation for all three supported runtimes (node, deno, bun). - Add `getSystemDenoVersion` / `getSystemBunVersion` and a generic `getSystemRuntimeVersion(name)` dispatcher in the runtime-version helper package. - Walk each runtime entry in the manifest during pnpm startup, compare to the live system runtime, and throw `ERR_PNPM_BAD_RUNTIME_VERSION` (or warn) on a mismatch. Invalid ranges (e.g. `"invalid range"`) are reported instead of crashing `semver.minVersion`. Missing runtimes ("no Node.js on the system") get the same error path. - The shell-out for deno/bun only runs when the manifest configures them AND `onFail` is `error`/`warn`. `download`/`ignore` short-circuit, and projects with no runtime pin pay nothing. Memoized per runtime. - `pnpm --version`, `pnpm --help`, and `pnpm <cmd> --global` are exempt from the check. - Rename `@pnpm/engine.runtime.system-node-version` → `@pnpm/engine.runtime.system-version` to match its broader scope; hoist `RuntimeName` / `RUNTIME_NAMES` / `isRuntimeAlias` to `@pnpm/types` so callers don't need to depend on `pkg-manifest.utils` just for the alias check. ## Tests - `pnpm --filter pnpm run compile` - `pnpm --filter pnpm exec jest packageManagerCheck.test` — 42 passing. New coverage: node/deno/bun version mismatch, invalid range, missing range, multi-entry runtime arrays, `engines.runtime` path (not just `devEngines.runtime`), and the `pnpm --version` exemption. - `pnpm --filter @pnpm/engine.runtime.system-version test` — 10 passing, 100% statement coverage; unit tests for each helper and the dispatcher. - Manual end-to-end smoke tests against the rebuilt bundle for deno and bun version mismatch. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added runtime version validation for Node.js, Deno, and Bun. The system now enforces `devEngines.runtime` and `engines.runtime` declarations with configurable failure behavior (`error`, `warn`, or `ignore`). * Enhanced error messages for runtime version mismatches with helpful suggestions for overrides. * **Improvements** * Improved system runtime detection and version checking across multiple runtime environments. --------- Co-authored-by: Puneet Dixit <236133619+puneetdixit200@users.noreply.github.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
a1f6f32996 |
fix(pacquet/package-manager): build workspace state from project list, not lockfile (#11946)
`build_projects_map` derived workspace projects from `lockfile.importers.keys()`. On the fresh-install path the wanted lockfile is `None` (no `pnpm-lock.yaml` on disk yet), so the function fell into its no-lockfile arm and recorded **only the root importer** — even for an 87-project workspace like babylon. That broke the [`optimistic_repeat_install`](https://github.com/pnpm/pnpm/blob/6b3ba4d337/pacquet/crates/package-manager/src/optimistic_repeat_install.rs) fast path on the *next* install: `project_structure_matches` compared the recorded project list (`len = 1`) against the workspace projects the resolver discovered (`len = 87`), returned `false`, and the install fell through to the full resolve + verify path even though the on-disk state was already a no-op. ## Fix Upstream pnpm's [`createWorkspaceState`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/workspace/state/src/createWorkspaceState.ts#L14-L27) takes `allProjects: ProjectsList` as input and emits one entry per project — the lockfile isn't involved. Pacquet already builds the matching `project_manifests: &[(PathBuf, &PackageManifest)]` at the top of `Install::run` (for the optimistic-repeat fast path); thread it through to `build_workspace_state` instead of re-deriving from the lockfile. The new `build_projects_map` is half the size of the old one and can't fall into a "lockfile missing" arm — the project list always comes from the same scan the rest of the install uses. ## Impact Re-bench against the vlt `babylon` fixture (87-project monorepo) shows every `*+node_modules` cell collapsing from "very slow" to "faster than pnpm": | Variation | Before | After | |---|---:|---:| | `node_modules` | 37.87× | **0.09×** (11× faster than pnpm) | | `lockfile+node_modules` | 25.52× | **0.07×** (14× faster) | | `cache+node_modules` | 11.55× | **0.15×** (6.7× faster) | | `cache+lockfile+node_modules` | 11.35× | **0.17×** (5.9× faster) | These were the four worst cells in the vlt benchmark chart for babylon (all flagged DNF before #11944 fixed the underlying panic). After #11944 babylon stopped DNF'ing but stayed 11-37× slower because of this workspace-state writing bug. The fast path only failed for workspace installs whose wanted lockfile was absent on the first install of the iteration — i.e. exactly the vlt benchmark's `node_modules` and `*+node_modules` shape. Single-project installs always recorded the root correctly because the no-lockfile fallback already emitted the root entry. Found while validating #11944's claim that the babylon DNF cells would collapse on the next pacquet release (#11902). |
||
|
Zoltan Kochan
|
198c661b99 |
fix(pacquet): require pnpm-lock.yaml for single-project optimisticRepeatInstall fast path (#11945)
* fix(pacquet): require pnpm-lock.yaml for single-project optimisticRepeatInstall fast path The port of pnpm's `optimisticRepeatInstall` short-circuit in #11943 applied the workspace branch's mtime-only exit (`checkDepsStatus.ts:263-271`) to every install, including single-project ones. Pnpm's single-project branch (`checkDepsStatus.ts:387-462`) additionally throws `RUN_CHECK_DEPS_LOCKFILE_NOT_FOUND` when `pnpm-lock.yaml` is absent, which the outer `try` converts into `upToDate: false`. Without that gate, pacquet treated a single-project install with `node_modules` present but no lockfile as "Already up to date" — the pnpm.io `node_modules`-only and `cache+node_modules` benchmark cells finished in ~35 ms instead of running the install (pnpm ~5–7 s on the same fixtures). Add an `is_workspace_install: bool` parameter; in single-project mode, require `<workspace_root>/pnpm-lock.yaml` to exist before declaring the install up to date. Workspace installs continue to skip the lockfile probe — pnpm's workspace branch's only lockfile check (`findConflictedLockfileDir`) silently `continue`s on ENOENT (`checkDepsStatus.ts:593-596`). Tests: - `returns_skipped_when_lockfile_missing_in_single_project_mode` - `returns_up_to_date_in_workspace_mode_without_lockfile` - `optimistic_repeat_install_does_not_short_circuit_when_lockfile_missing` (install-level integration test) - Existing happy-path tests now seed `pnpm-lock.yaml` via a new `write_empty_lockfile` helper in `setup_fresh_install`. * test(pacquet): prove single-project optimisticRepeatInstall round-trips end-to-end Add `optimistic_repeat_install_round_trips_on_single_project_install`: two real `Install::run` calls back-to-back on a non-workspace project (no `pnpm-workspace.yaml`). The first install resolves through the registry mock and writes `pnpm-lock.yaml` + `.pnpm-workspace-state-v1.json` to disk. The second install must hit the optimistic fast path — emit `Already up to date` and skip every install-setup event. Pairs with the negative `..._does_not_short_circuit_when_lockfile_missing` test so the gate's polarity is pinned in both directions. * style(pacquet): cargo fmt |
||
|
Zoltan Kochan
|
6b3ba4d337 |
fix(pacquet): port pnpm's workspace-link short-circuit and depPath helpers (#11944)
`pacquet install` was panicking on the babylon vlt fixture under the `node_modules` and `lockfile+node_modules` variations. The crash surfaced in [`PkgNameVerPeer::without_peer`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/pacquet/crates/lockfile/src/pkg_name_ver_peer.rs#L55-L62), but the root cause was an unported piece of pnpm's resolver: workspace `link:` nodes weren't short-circuited and flowed through peer resolution, producing depPaths of the shape `link:<rel-path>(<peers>)` that downstream code wasn't prepared for. This PR ports the missing short-circuit, fixes the panic, and closes the parity gaps the audit surfaced. ### Workspace-link short-circuit (the root fix) Ported upstream's [`isLinkedDependency`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/installing/deps-resolver/src/resolveDependencies.ts#L926-L937) arm plus the [`depth === -1`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/installing/deps-resolver/src/resolvePeers.ts#L396) short-circuit in `resolvePeers.ts`. Workspace `link:` deps now: - Skip child recursion in the tree walker (`TreeChildren::Realized(empty)`). - Carry `depth = -1` on the tree node. - Carry empty `peer_dependencies` on the `ResolvedPackage` — peer matching is the linked importer's concern. - Use a leaf [`NodeId`] so every reference to the same workspace path shares one id. `resolve_peers::resolve_node` early-returns for `depth == -1` nodes with `dep_path = pkg.id` (just `link:<rel-path>`). The link node never enters the graph, so `packages:` / `snapshots:` stay clean and the importer's `version:` cell carries `link:<rel-path>` exactly. ### `PkgNameVerPeer::without_peer` no longer panics Construct the bare key through the typed fields rather than reformatting `{prefix}{version}` and re-parsing under `.expect(...)`. The new `PkgVerPeer::without_peer` clones the existing `prefix`/`version` slots and returns a `PkgVerPeer` with an empty peer string. No round-trip, no `.expect(...)`. Defensive even after the upstream fix lands — `without_peer` is called on values from the lockfile, which can be hand-edited. ### `pkgIdWithPatchHash` is now strip-peer-only at every site Four call sites built a [`PkgIdWithPatchHash`] from the wrong baseline: - `virtual_store_layout::lockfile_to_dep_graph` stripped both segments (`PkgNameVerPeer::without_peer().to_string()`) — patched variants of the same `name@version` collided in the dep-graph hash input. - `create_virtual_store`'s two `cas_paths_by_pkg_id` inserts and `hoisted_dep_graph`'s `pkg_id_with_patch_hash` initialiser stripped nothing (raw `snapshot_key.to_string()`) — peer variants of the same patched package got separate CAS-paths entries instead of sharing one. All four now go through `pacquet_deps_path::get_pkg_id_with_patch_hash` (the balanced-paren scan upstream's [`getPkgIdWithPatchHash`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/deps/path/src/index.ts#L63-L70) uses): strip the peer-graph suffix, keep `(patch_hash=…)`. Non-patched packages are unaffected. ### New helpers in `pacquet-deps-path` - `is_runtime_dep_path` — matches `^(?:node|bun|deno)@runtime:` byte-level. Pnpm filters the runtime-only install pass with this. - `try_get_package_id` — strips peer-graph + patch-hash suffix, then drops the `<name>@` prefix on URL-shaped resolution ids while keeping `runtime:` entries intact. ### Ported `deps/path/test/index.ts` cases | Suite | Cases ported | |---|---| | `pacquet-deps-path::suffix_index` | runtime, scoped, scoped + patch-hash, scoped + peer, scoped + both, leading-slash-legacy nested-peer, scope-with-parens | | `pacquet-deps-path::is_runtime_dep_path` | pnpm's `isRuntimeDepPath` test + carve-outs | | `pacquet-deps-path::try_get_package_id` | pnpm's `tryGetPackageId` test + URL-shape, bare, runtime carve-outs | | `pacquet-lockfile::PkgVerPeer` | patch-hash + peer round-trip | | `pacquet-lockfile::PkgNameVerPeer` | file-protocol tarball, patch-hash + peer, scope-with-parens, babylon regression | | `pacquet-resolving-deps-resolver::tests` | babylon-shape: workspace dep with peers → `depth = -1`, empty children, no peer_dependencies | | `pacquet-package-manager::virtual_store_layout::tests` | patched snapshot → `full_pkg_id` retains `(patch_hash=…)` | Resolves #11939. |
||
|
Zoltan Kochan
|
c8c50caeca |
perf(pacquet): port optimisticRepeatInstall fast path for repeat installs (#11943)
Closes #11940. ## Summary Ports upstream pnpm's `optimisticRepeatInstall` + [`checkDepsStatus`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/deps/status/src/checkDepsStatus.ts) dispatch ([`installing/commands/src/installDeps.ts:179-194`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/installing/commands/src/installDeps.ts#L179-L194)). When nothing has changed since the previous successful install, `Install::run` now logs `Already up to date` and returns **before**: - loading the wanted or current lockfile, - building the lockfile-verifier list, - the `verify_lockfile_resolutions` fan-out (and the `<cache_dir>/lockfile-verified.jsonl` lookup it performs internally), - `getContext`, project registration, `validateModules`, - the no-op short-circuit that fires *after* all of the above. That's the missing earlier shortcut from the original investigation. It's what lets pnpm finish the vlt `lockfile+node_modules` cells in ~580 ms regardless of `~/.cache/pnpm` state — the verifier-cache file the bench wipes is irrelevant because pnpm never reaches the verifier on a repeat install. ## How it works The fast path keys off `<workspace_root>/node_modules/.pnpm-workspace-state-v1.json`'s `lastValidatedTimestamp` against each project's `package.json` mtime, plus a settings-drift check and a workspace-project structure check. Wire shape and field-by-field comparison match upstream's [`WorkspaceStateSettings`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/workspace/state/src/types.ts) so a previous-install state file written by pnpm is honored by pacquet and vice versa. Settings construction is shared between `build_workspace_state` (writer) and `check_optimistic_repeat_install` (reader) via `optimistic_repeat_install::current_settings`, so the two can't drift on a new field. ## Scope This PR ports the mtime-vs-`lastValidatedTimestamp` exit only — upstream's `modifiedProjects.length === 0` branch at [`checkDepsStatus.ts:263-271`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/deps/status/src/checkDepsStatus.ts#L263-L271). Branches that detect a modified project and then re-verify the lockfile (`assertWantedLockfileUpToDate`, `patchesOrHooksAreModified`) aren't ported here — when any manifest is newer than the last validation, this function returns `Skipped` and the install falls through to the regular path, which still has its own freshness guards (`check_lockfile_freshness`, the existing no-op short-circuit). Disabled under `--frozen-lockfile` so a headless install still fails loudly on missing / stale lockfiles, matching upstream not calling `checkDepsStatus` in that mode. ## Config New `optimistic_repeat_install: bool` field on `pacquet_config::Config`, default `true` — matches [`config/reader/src/index.ts:169`](https://github.com/pnpm/pnpm/blob/cc4ff817aa/config/reader/src/index.ts#L169). Wired through `pnpm-workspace.yaml` via `WorkspaceSettings.optimistic_repeat_install: Option<bool>`. Yaml `optimisticRepeatInstall: false` opts out per-project; the value also lives in the global config file's allowlist so users can opt out at the user level. |
||
|
Zoltan Kochan
|
cc4ff817aa |
fix(pacquet/registry): deserialize optionalDependencies and peerDependenciesMeta (#11934)
`PackageVersion` (the per-version registry manifest the npm resolver parses) was missing the `optionalDependencies` and `peerDependenciesMeta` fields. The resolver builds `ResolveResult.manifest` via `serde_json::to_value(picked)` and downstream walks it with [`extract_children`](https://github.com/pnpm/pnpm/blob/1fb8a2d5d8/pacquet/crates/resolving-deps-resolver/src/resolve_dependency_tree.rs#L752-L759) (reads `optionalDependencies`) and [`extract_peer_dependencies`](https://github.com/pnpm/pnpm/blob/1fb8a2d5d8/pacquet/crates/resolving-deps-resolver/src/resolve_dependency_tree.rs#L776-L824) (reads `peerDependenciesMeta`). Without those fields on the struct, both reads always saw nothing — so `optionalDependencies` edges were silently dropped, and every optional peer was treated as required, then auto-installed via the `autoInstallPeers` fallback in [`hoist_peers`](https://github.com/pnpm/pnpm/blob/1fb8a2d5d8/pacquet/crates/resolving-deps-resolver/src/hoist_peers.rs#L134-L136). ## Astro cascade On the vlt [`astro`](https://github.com/vltpkg/benchmarks/tree/main/fixtures/astro) fixture, `unstorage` (a transitive of astro) declares 19 optional peers via `peerDependenciesMeta` (`@azure/*`, `@vercel/*`, `@netlify/blobs`, `@upstash/redis`, `@deno/kv`, `ioredis`, `uploadthing`, …). Pacquet's resolver auto-installed every one of them and walked their transitive trees; astro's own `optionalDependencies` (`sharp`) went missing entirely. The supposed "5.5× astro deep-tree slowdown" tracked in #11902 was almost all wasted work, not a real perf bug. None of the candidate hypotheses listed there (`async_recursion` Box-pinning, per-node `lock_recoverable` mutex acquires, manifest `serde_json::to_value` cost, tarball extraction) were the bottleneck. ## Before / after on vlt astro | Metric | Before | After | pnpm 11.3.0 | |---|---:|---:|---:| | `pacquet install` wall time (warm store) | 39.6 s | 8.5 s | 7.0 s | | Lockfile lines | 13,364 | 3,037 | 3,444 | | `resolution:` entries | 1,535 | 377 | 377 | | Astro root peer suffixes | 30 (`@azure/...`, `@vercel/...`, ...) | `(rollup@4.60.4)(typescript@5.9.3)` | `(rollup@4.60.4)(typescript@5.9.3)` | | `sharp` (`optionalDependencies`) refs in lockfile | 0 | 110 | 85 | Warm-cache hyperfine (3 runs, fresh `node_modules` + lockfile each time): ``` pacquet (patched): 670 ms ± 72 ms pnpm 11.3.0: 1270 ms ± 7 ms pacquet is 1.89 ± 0.20 times faster than pnpm ``` Closes the astro column in #11902. ## Implementation - Add `optional_dependencies: Option<HashMap<String, String>>` and `peer_dependencies_meta: Option<HashMap<String, PeerDependencyMeta>>` to `PackageVersion`. The existing `#[serde(rename_all = "camelCase")]` handles wire format. - Add a `PeerDependencyMeta` newtype with just the `optional` field (the only field the resolver consumes). - Fix up the four struct-literal construction sites in tests + the trust-evidence projection. - Add a regression test that deserializes a fixture with both fields populated and asserts they round-trip through `serde_json::to_value` — which is what the resolver consumes. |
||
|
Zoltan Kochan
|
7120ac0813 |
fix(pacquet/resolving-npm-resolver): singleflight verifier lookup caches (#11933)
* fix(pacquet/resolving-npm-resolver): singleflight the verifier lookup caches Convert PublishedAtLookupContext's five per-key dedup caches from Mutex<HashMap<String, T>> to Mutex<HashMap<String, Arc<OnceCell<T>>>> so two verifier tasks that race for the same key share one in-flight fetch instead of both performing the work. Mirrors upstream's `Map<string, Promise<T>>` singleflight pattern (see #11932). The outer mutex is dropped before awaiting the init future so unrelated keys stay unblocked. Add a fan-out regression test that asserts 16 concurrent verifications of the same (registry, name, version) issue exactly one abbreviated GET; without the singleflight property mockito's `.expect(1)` fails with 16 requests received. Refs #11932. * fix(pacquet/resolving-npm-resolver): rename SingleflightMap generic to satisfy dylint `perfectionist::single-letter-generic` flagged the `T` parameter on the new `SingleflightMap` type alias. Rename to `Value` — the slot contents are the values cached behind the singleflight cells. |
||
|
Zoltan Kochan
|
1fb8a2d5d8 |
perf(pacquet): unlock no-op short-circuit + port abbreviated-modified verifier shortcut (#11931)
Two fixes that together unlock pnpm-parity on the `benchmarks.vlt.sh` `lockfile+node_modules` shape — the row where pacquet was 2-12× slower than pnpm on every fixture. ### 1. `fix(modules-yaml)`: normalise joined `virtualStoreDir` `read_modules_manifest` joins a stored relative `virtualStoreDir` with `modules_dir` to recover an absolute path, mirroring upstream's `path.join(modulesDir, modules.virtualStoreDir)`. Node's `path.join` normalises interior `..` segments; Rust's `PathBuf::join` does not. Stored values like `../../../Users/.../store/v11/links` came back as `<modules_dir>/../../../Users/.../links` — never byte-matched `Config::effective_virtual_store_dir()`, so the no-op short-circuit added in #11904 silently missed every install whose store sits outside the project (the default macOS / Linux setup). The accompanying refactor lifts `lexical_normalize` (already duplicated in `cmd-shim` and `store-dir`) into `pacquet-fs` so `modules-yaml` doesn't make it a third copy. ### 2. `perf(resolving-npm-resolver)`: port the missing verifier layers The npm resolution verifier walked a 4-layer fallback chain in upstream pnpm (abbreviated-modified shortcut → on-disk full-meta mirror → npm attestation endpoint → full packument fetch); pacquet only had the last two. The module's doc-comment explicitly noted "Phase 4 stubs the abbreviated-shortcut and on-disk-mirror layers (no cached fetcher / no mirror yet); Phase 5 ports `fetchFullMetadataCached.ts`…" — this is Phase 5. Result: a cold lockfile-verification pass now pays at most one *abbreviated* GET per name (small payload, decided by package-level `modified`) instead of a full-meta GET per name (hundreds of KB each). ## Bench 5-iteration cold-cache measured pass on `vltpkg/benchmarks/fixtures/svelte` (`pnpm-lock.yaml` + `node_modules` present, `~/.cache/pnpm` and store wiped before each run), 10-core M-series Mac: | | pnpm | pacquet@main | this PR | |-------------|------:|-------------:|--------:| | wall time | 0.54 s | 2.16 s | 0.71 s | 3.0× faster on the `lockfile+node_modules` row. |
||
|
Zoltan Kochan
|
b7229f8571 |
fix(pacquet/resolving-npm-resolver): honor linkWorkspacePackages for bare-semver deps (#11930)
* fix(pacquet/resolving-npm-resolver): honor linkWorkspacePackages for bare-semver deps Pacquet's npm resolver only consulted the workspace map for `workspace:`-prefixed wanted deps; bare-semver ranges always went straight to the registry. When the workspace package isn't on npm (e.g. babylon's `@dev/build-tools`), the install errored out with 404; when a same-named package existed on the registry, pacquet silently linked the wrong copy. Mirror pnpm's three workspace branches around `pickPackage`: * registry pick succeeded + workspace shadow (exact `name@version` match, higher local version, or `preferWorkspacePackages`), * registry pick returned `null` → workspace fallback, * registry fetch errored → workspace fallback (swallow workspace errors and re-raise the registry error). Gated by `link-workspace-packages` (true / false / "deep") which is now parsed from `pnpm-workspace.yaml`, flows through `Config`, and is encoded into `ResolveOptions::always_try_workspace_packages` at the install layer. Tri-state semantics are preserved on the config side; pacquet's single-`base_opts` deps-resolver collapses `true` and `"deep"` onto the same per-call flag until depth threading lands. Closes #11929. * fix(pacquet/resolving-npm-resolver): swap unicode ellipsis for ascii triple-dot to satisfy Dylint * test(pacquet/resolving-npm-resolver): port remaining linkWorkspacePackages tests from pnpm Cover the six pnpm tests the initial port skipped: * `injected_workspace_match_emits_file_resolution` — workspace shadow branch with `injected: true` emits a `file:` resolution. * `workspace_fallback_picks_highest_version_for_latest_tag` — 404 fallback into a multi-version workspace via the Tag branch of `pick_matching_local_version_or_null`. * `workspace_fallback_picks_local_prerelease_for_latest_tag` — 404 fallback into a prerelease-only workspace, exercising `resolve_workspace_range`'s `includePrerelease` arm. * `workspace_fallback_resolves_specific_version_request` — 404 fallback against a pinned version-spec lookup. * `workspace_fallback_kicks_in_when_registry_lacks_requested_version` — `Ok(None)` fallback path (registry serves the packument but no version matches), distinct from the `Err` 404 path. * `registry_error_propagates_when_workspace_has_no_matching_version` — negative test verifying the original 404 surfaces when the workspace can't satisfy the request. * `registry_pick_wins_when_workspace_version_does_not_match` — workspace shadow no-op when the workspace carries a different version than the registry pick. * fix(pacquet/resolving-npm-resolver): drop trailing comma in single-line assert! to satisfy Dylint * test(pacquet/resolving-npm-resolver): trim redundant doc-prose on linkWorkspacePackages tests Per pacquet/CLAUDE.md "tests are documentation" — the new test docblocks restated what the test name plus body already say. Keep only the upstream-link citation (required by the porting rule) and drop the trailing narrative. Two ports keep one extra sentence because the distinction they exercise (the `Ok(None)` vs `Err` fallback split, the `includePrerelease` arm) is not recoverable from the test body alone. Also drop the inline `lockfile_dir` / "Registry returns a packument..." comments that narrate setup the helper code already encodes; keep the `latest`-back-stamp comment because it explains why a workspace-resolved result still carries that field. |
||
|
Nicolas Le Cam
|
e52e4fce63 |
feat(pacquet): port detect-libc to Rust and replace ad-hoc libc detection in graph-hasher (#11921)
* refactor(graph-hasher): replace ad-hoc libc detection with pacquet-detect-libc Extract libc detection into a new `pacquet-detect-libc` crate ported from the upstream `detect-libc` JS package, replacing the limited ad-hoc `detect_host_libc()` in graph-hasher. Detection uses a three-tier fallback (ELF header → filesystem → command) that avoids spawning processes in the common case and works in slim containers where getconf or ldd may not be present. The command step runs getconf and ldd --version as separate subprocesses to avoid stream pollution between the two, with ldd only invoked when getconf fails. * fix(detect-libc): harden ELF parser, UTF-8 decoding, test cfg, and imports - Use checked arithmetic (checked_add/checked_mul) in elf_interpreter to return None on overflow instead of panicking on malformed headers - Use from_utf8_lossy for /usr/bin/ldd content so non-UTF-8 bytes don't skip the filesystem detection path - Gate detect_integration_host test with #[cfg(target_os = "linux")] so it doesn't fail on non-Linux platforms - Replace use super::* with explicit imports in command tests * fix(detect-libc): use from_utf8_lossy for command output, fix lints and tests |
||
|
Nicolas Le Cam
|
97391bf341 |
test(pacquet/package-manager): make side-effects write test umask-agnostic (#11922)
* fix(pacquet/package-manager): make side-effects write test umask-agnostic The test hardcoded mode 0o644 in the pre-seeded PackageFilesIndex row, but fs::write() assigns mode according to the process umask (e.g., 0o664 with pam_umask usergroups, Debian's default). calculate_diff() compares both digest and mode, so a mode mismatch caused a false-positive assertion failure for unchanged index.js on systems with umask 0002. Read the actual file mode from the written fixture and use it in the pre-seeded row instead of a hardcoded value. * fix(pacquet/package-manager): factorize umask-agnostic mode into fixture Move the actual-mode reading into create_postinstall_modifies_source_fixture so both write_path_populates_side_effects_row and write_path_cache_key_includes_patch_hash share it, instead of each test duplicating the metadata inspection. |
||
|
Zoltan Kochan
|
d579e6cbb5 |
perf(pacquet): trim install-phase syscalls and allocations (#11864)
* perf(fs,package-manager): striped CAS lock + skip pre-flight stat on fresh-target imports Two install-phase syscall trims: - `cas_write_lock` swaps the per-path `DashMap<PathBuf, Arc<Mutex<()>>>` for 256 static `Mutex<()>` stripes keyed by hashed path. Every CAFS write previously paid one `PathBuf::to_path_buf` allocation, a `DashMap` shard write lock, plus an `Arc<Mutex<()>>` slot allocation even though contention was vanishingly rare. Striping keeps the writer/verifier coordination the per-path mutex provided while removing those per-call costs. With 256 stripes and ~10 rayon workers the false-sharing probability per pair is ~4%, and the guarded body (one `O_CREAT|O_EXCL` open + `write_all` of a tar entry) is microseconds long. - `import_indexed_dir::populate_dir` now calls a new `import_into_fresh_target` instead of `link_file`. `populate_dir` only ever runs against a directory it just `mkdir`'d, so the `fs::metadata` pre-flight `link_file` performs to protect the Copy-method overwrite contract is wasted — every call is `NotFound` in practice and the EEXIST surface from the import syscall is the only collision signal we need. Saves ~170k `stat` syscalls per clean install on the alotta-files fixture. `link_file` still exists with the original semantics for any caller that genuinely doesn't know whether the target is fresh. On the 3343-package alotta-files fixture against the verdaccio mock, clean-install wall time goes from ~28s to ~19-22s on the local 10-core machine — roughly closing the gap to pnpm (~20s) for that scenario. Refs #11857, #11851. * perf(store-dir): trim per-CAS-file allocations on the hot write path Two micro-optimisations in `cas_file_path`, the helper every CAFS write goes through: - `cas_file_path` no longer `format!`s the sha-512 digest into a fresh `String`. Sha-512 is always 64 bytes / 128 hex chars, so render the hex into a stack buffer and slice it into the `file_path_by_hex_str` call instead. One heap allocation per file shaved off — ~170k on the alotta-files clean install. - The repeated `self.v11().join("files")` rebuild used to walk two `PathBuf::join`s per call. Memoise the result behind a `OnceLock` on `StoreDir` (`cached_files_dir`) so `file_path_by_head_tail` borrows it without re-joining. Race-free initialisation across rayon workers, one allocation per process instead of one per file. Refs #11857. * docs(pacquet): address CodeRabbit nits - Refresh `import_indexed_dir` doc comments so they name `import_into_fresh_target()` (the actual materialization helper after the fresh-target split) instead of `link_file()`. - Add a const assertion that `NUM_CAS_LOCK_STRIPES` stays a power of two, since `cas_write_lock` uses `& (NUM_CAS_LOCK_STRIPES - 1)` as the stripe selector. * docs: forbid past-implementation history in comments - Extend AGENTS.md Comments rules: comments must describe the current contract, not what the code replaced. Phrasings like "used to", "previously", "the original X", or parentheticals naming a removed type belong in `git log`. - Apply the rule to `cas_write_lock`'s doc, which previously framed itself in terms of the removed `DashMap<PathBuf, Arc<Mutex<()>>>` shape. |
||
|
mehmet turac
|
440e15586d | fix: summarize all global update groups (#11920) | ||
|
Zoltan Kochan
|
ae2175829a |
feat(registry-access): extract dist-tag + adduser helpers, dogfood from tests (#11926)
* feat(registry-access): extract setDistTag and dogfood from tests
Add `@pnpm/registry-access.commands#setDistTag` — the low-level PUT to
`/-/package/:pkg/dist-tags/:tag`. The CLI `dist-tag add` handler now
calls it instead of issuing the fetch inline.
Tests in this monorepo now use a thin new package
`@pnpm/testing.registry-mock` (REGISTRY_MOCK_PORT + REGISTRY_MOCK_CREDENTIALS
baked in) that delegates to `setDistTag`, replacing `addDistTag` from
`@pnpm/registry-mock`. That dropped helper relied on
`anonymous-npm-registry-client` and a verdaccio-era
fetch-then-DELETE-then-PUT dance that is no longer needed against
pnpm-registry.
39 test files swapped from `@pnpm/registry-mock` to
`@pnpm/testing.registry-mock`.
* fix: move setDistTag to its own package to break tsconfig project-reference cycle
testing/registry-mock → registry-access.commands → releasing/commands
→ installing/commands → installing/deps-installer → testing/registry-mock.
Extract setDistTag into @pnpm/registry-access.set-dist-tag (only depends
on @pnpm/error, @pnpm/network.fetch, @pnpm/npm-package-arg). Both
@pnpm/registry-access.commands and @pnpm/testing.registry-mock import
from it. Cycle gone.
* feat(registry-access): extract addUser helper, dogfood from login + tests
Add @pnpm/registry-access.add-user — a small helper that PUTs to
/-/user/org.couchdb.user:<name> and returns { token }. The CLI's
classicLogin (pnpm login fallback path) now calls it, and tests
use it via @pnpm/testing.registry-mock instead of the legacy
addUser from @pnpm/registry-mock.
Swapped 3 call sites: globalSetup.js, installing/deps-installer's
auth.ts, and pnpm/test/dlx.ts. AddUserHttpError exposes status +
text + parsed-json-if-applicable + headers so the CLI can still
do its OTP detection. One webauth-OTP login test mock had to be
adjusted to provide its body via `text` (JSON-stringified) rather
than `json` only, since the helper consumes the body via `text()`.
* refactor: consolidate set-dist-tag + add-user helpers into one @pnpm/registry-access.client package
One shared package is better than splitting per endpoint. Future endpoints
(publish, deprecate, etc.) can land here without another wrapper.
No behavioral change — same setDistTag and addUser exports as before,
just under one roof. Callers updated: registry-access.commands,
auth.commands, testing.registry-mock.
* fix(registry-access): sort imports
|
||
|
Zoltan Kochan
|
ac299aa0e5 |
fix(pacquet,package-manager): walk every workspace project in fresh-resolve install (#11905)
* fix(package-manager): walk every workspace project in fresh-resolve install The fresh-resolve install path (no `--frozen-lockfile`, no usable lockfile) only resolved the workspace root manifest, so sibling workspace projects' own dependencies never landed in the lockfile or on disk. Re-run `resolve_importer` per importer with shared install caches (`meta_cache`, `fetch_locker`, `picked_manifest_cache`), merge the per-importer graphs, and emit one `importers[<id>]` entry per project. Mirrors upstream's [`resolveRootDependencies`](https://github.com/pnpm/pnpm/blob/3422cecfd3/installing/deps-resolver/src/resolveDependencies.ts#L327-L437) iteration shape — one shared resolution context, per-importer direct-deps slices. Per-importer `link_bins` so each project gets its own `node_modules/.bin`. GVS `register_project` now loops every importer key the freshly-built lockfile carries, mirroring the frozen path. `importer_dep_version` and `snapshot_dep_ref` learned a `link:` short-circuit so workspace-sibling edges emit `ImporterDepVersion::Link` / `SnapshotDepRef::Link` instead of falling through to the `name@version` parser. Cross-importer `TreeCtx` sharing (full upstream parity: one resolution context with per-importer hoist loops) is deferred — each `resolve_importer` call still has its own context. Network-side caches still amortize packument fetches and JSON parsing across importers; only per-resolve semver matching duplicates. Closes #11901. * fix(workspace): drop trailing comma on single-line assert_eq! for Perfectionist lint * fix(package-manager): register only the workspace root with the store, matching pnpm Pacquet was looping `register_project` over every importer in both the frozen-lockfile and fresh-lockfile branches, but upstream pnpm calls `registerProject(opts.storeDir, opts.lockfileDir)` exactly once per install against the workspace root — store prune walks the workspace's `node_modules/.pnpm/` to find every installed package, so one registry entry per workspace is enough. Consolidate to a single call near the start of `Install::run`, matching pnpm's `getContext` ordering at <https://github.com/pnpm/pnpm/blob/d8a79a9c30/installing/context/src/index.ts#L128>. Also port two upstream-derived tests that the multi-importer rewrite of `compute_corrected_optional` and the per-importer link rendering were previously missing direct coverage for: - `multi_importer_pruner_marks_shared_dep_non_optional_when_any_importer_reaches_via_prod` ports the spirit of pnpm's [`pruneSharedLockfile`](https://github.com/pnpm/pnpm/blob/d8a79a9c30/lockfile/pruner/src/index.ts#L17) cross-importer pooling: a depPath reached via a non-optional path from any importer ends up `optional: false` even when another importer reaches it only via an optional path. - `workspace_sibling_link_renders_per_importer_with_link_ref` exercises the multi-importer `workspace:`-link case — importer A depends on importer B via a `link:`-resolved depPath, both render their own `importers[<id>]` entries, and the link node stays out of `packages:` / `snapshots:`. * fix(package-manager): skip undeclared aliases from pruner BFS seeds Addresses CodeRabbit's review on PR #11905. Pacquet's resolver hoists auto-installed peers into `direct_dependencies_by_alias` even when they aren't in the importer's manifest (see `resolve_importer::direct.extend(...)` after each `hoist_peers` call). `build_importer` correctly excludes those undeclared aliases from the importer's lockfile entry, but `compute_corrected_optional` was seeding the pruner BFS from the full `direct_dependencies_by_alias` and defaulting unknown aliases to `DependencyGroup::Prod`. That diverges from upstream's [`pruneSharedLockfile`](https://github.com/pnpm/pnpm/blob/d8a79a9c30/lockfile/pruner/src/index.ts#L27-L29), which seeds purely from `lockfile.importers[*].{dev,optional,}dependencies` — i.e., from the same set `build_importer` writes. The mismatch forced auto-peers reachable only via an optional parent's chain to `optional: false`, leaking them into non-optional installs. Skip aliases not in the manifest when seeding. The new test `auto_installed_peer_not_declared_in_manifest_is_skipped_from_pruner_seeds` pins the corrected behavior — `peer-x` (auto-installed for an `optionalDependencies` parent) stays `optional: true`, matching pnpm. Verified the test fails against the pre-fix code. Also tightens the multi-importer integration test's lockfile assertion: scope the `hello-world-js-bin-parent` check to the `packages/a:` importer section instead of a global substring match, so the test proves the direct-dep entry — not just any mention in `packages:`. * fix(package-manager,store-dir): ensure store root exists before registering project CI failure: `fresh_install_honors_enable_global_virtual_store` started failing after the previous register_project consolidation. Two compounding bugs: 1. `register_project` now runs early in `Install::run`, before any install phase has materialized the store. With the test's relative `storeDir: ../pacquet-store` in `pnpm-workspace.yaml`, `config.store_dir.root()` ends up shaped like `<workspace>/../pacquet-store/v11` — a path that doesn't yet exist on disk. 2. `path_contains`'s "lexical fallback" wasn't actually lexical — it called `dunce::canonicalize`, and on failure (path doesn't exist) it kept the path verbatim and ran `starts_with`. So `<workspace>/../pacquet-store/v11`.starts_with(`<workspace>`) returned true, the early-return guard fired, and the call silently skipped without writing the registry entry. Two-part fix matching upstream: - `Install::run` now calls `fs::create_dir_all(store_dir.root())` before `register_project`, mirroring pnpm's [`fs.mkdir(opts.storeDir, { recursive: true })`](https://github.com/pnpm/pnpm/blob/d8a79a9c30/installing/context/src/index.ts#L125) call right before `registerProject`. Once the store exists, `canonicalize` succeeds and `path_contains` resolves both sides correctly. - `path_contains` now lexically normalizes `.` / `..` components when canonicalize fails. Matches upstream's `is-subdir` semantics (which uses `path.relative`, purely lexical). New test `path_contains_resolves_parent_components_when_paths_do_not_exist` pins the behavior; verified it fails against the pre-fix code. * style: cargo fmt * fix(package-manager,store-dir): satisfy Perfectionist lint and harden lexical_normalize Two issues: 1. The multi-line `assert!` in `multi_importer_pruner_marks_shared_dep_non_optional_when_any_importer_reaches_via_prod` was missing its trailing comma after `cargo fmt` reformatted it from one-line to multi-line. Perfectionist's `macro-trailing-comma` rule (which CI enforces via Dylint) flagged it. Added the comma. 2. CodeRabbit pointed out that `lexical_normalize` silently dropped leading `..` components because `PathBuf::pop()` is a no-op when the path is empty. For the current `path_contains` callers (both inputs are absolute paths) this doesn't matter, but the helper is now a general-purpose utility and the bug would bite any future caller passing a relative path. Replaced the naive `out.pop()` with a match on the trailing component: - `Component::Normal(_)` → pop (real segment collapses with `..`) - `Component::RootDir | Prefix(_)` → drop the `..` (`/..` is `/` per POSIX) - else → push `..` (preserve leading `..` chain in relative paths) Matches Go's `path.Clean` semantics. New test `lexical_normalize_handles_parent_dir_corner_cases` pins all four corner cases. |
||
|
mehmet turacandZoltan Kochan
|
0721d64188 |
fix: require provenance for trusted publisher evidence (#11911)
* fix: require provenance for trusted publisher evidence
* test: align provenance fixtures with registry types
* chore: include pnpm CLI in changeset
The repo guideline requires every changeset that touches a published
package to list the pnpm CLI explicitly so the fix appears in the CLI's
release notes.
* fix(resolving-npm-resolver): require provenance for trusted publisher evidence
Ports pnpm's
|
||
|
mehmet turac
|
e8b3ae132e | fix: clarify non-root resolutions warning (#11912) | ||
|
Zoltan Kochan
|
494cdcaa01 |
chore: drop verdaccio from the repo (#11925)
The TS test harness (`__utils__/jest-config/with-registry/globalSetup.js`) already launches `pnpm-registry`, and pacquet's `RegistryMode::Verdaccio` spawns `pnpm-registry` too (the enum variant is a misnamed leftover). The verdaccio dependency was only there to satisfy `@pnpm/registry-mock`'s peerDependency declaration — nothing in this repo invokes verdaccio at runtime. Remove the catalog entry, the stranded `verdaccio.yaml` config, and the `@verdaccio/auth` packageExtensions block. Mark `@pnpm/registry-mock`'s verdaccio peer optional so pnpm doesn't auto-install it (and the entire `@verdaccio/*` tree) across the workspace. Lockfile drops ~1100 lines. Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
d8a79a9c30 |
feat(registry): add auth/dist-tag/publish endpoints + wire TS tests onto pnpm-registry (#11914)
Lands the pieces of the npm registry protocol that pnpm-registry was missing, and switches the TypeScript test harness off verdaccio onto pnpm-registry. `@pnpm/registry-mock` (the npm package) is untouched. ### Server-side additions (`registry/crates/pnpm-registry`) - `PUT /-/user/org.couchdb.user:<name>` — adduser / login, returns a Bearer token. In-memory user + token stores. - `PUT /:pkg` — publish (scoped + unscoped). Base64-decodes `_attachments`, merges into the existing packument, writes manifest + tarball atomically. 100 MiB body limit. - `GET /-/package/:pkg/dist-tags` + `PUT/DELETE /-/package/:pkg/dist-tags/:tag` — rewrites the on-disk packument so tag changes survive a restart. - `Authorization: Bearer` and `Authorization: Basic` both identify the caller. - Per-package access policy (wax glob patterns). Defaults mirror `@pnpm/registry-mock`'s `config.yaml`: `@private/*` and `@pnpm.e2e/needs-auth` require auth; everything else is anonymous read, authenticated write. Enforced on every packument / version-manifest / tarball GET and every write endpoint. ### TypeScript-test migration - `__utils__/jest-config/with-registry/globalSetup.js` keeps `prepare()` from `@pnpm/registry-mock` (still needed for the tempy storage path written into the runtime-config yaml — `getIntegrity` reads it from there) but spawns `pnpm-registry` instead of verdaccio. `addUser`, `addDistTag`, `getIntegrity`, `REGISTRY_MOCK_*` from registry-mock work as-is — they're plain npm-wire-protocol HTTP calls. - Binary lookup follows pacquet's pattern: `PNPM_REGISTRY_BIN` env override, then `target/release/pnpm-registry`, then `target/debug/pnpm-registry`. - CI test job (`.github/workflows/test.yml`) installs the Rust toolchain via the existing `./.github/actions/rustup` composite action and builds `pnpm-registry --release` before tests run. Per-platform — Linux and Windows in the matrix each build their own. |
||
|
Zoltan Kochan
|
058f5f2f8b |
fix(package-manager): port pnpm's lockfile-pruner BFS to re-derive transitive optional (#11919)
The resolver's per-node AND-fold updates only the directly-revisited package, so descendants walked first via an `optionalDependencies` edge stay stuck at `optional: true` even when a later non-optional path reaches them transitively. Upstream hides this from users by re-deriving the flag in `copyDependencySubGraph`; pacquet had no equivalent pass, so a fetch or build failure on a transitively-required package was silently tolerated as if it were optional. Port the BFS into `dependencies_graph_to_lockfile`: walk from the importer's direct deps (classified by manifest dep-group), recurse through each node's children with parent-inherited optional for regular edges and forced `optional: true` for `optionalDependencies` edges, then override `SnapshotEntry.optional` to `false` for any package reached by an all-non-optional path. Refs https://github.com/pnpm/pnpm/issues/11916. |
||
|
Zoltan Kochan
|
3788a8b0e6 |
perf(pacquet): lazy children realization in dependency tree (#11915)
* perf(resolving-deps-resolver): defer per-occurrence child realization until peer resolution
Mirrors upstream pnpm's lazy `children` thunk on `DependenciesTreeNode`:
revisits of a `pkgIdWithPatchHash` no longer recurse to fan out a fresh
NodeId subtree eagerly. Instead the tree node records
`TreeChildren::Lazy { parent_ids }` and the peer resolver allocates
per-occurrence child NodeIds on first descent via `realize_children`,
applying the same `parentIdsContainSequence` cycle break upstream uses
in `buildTree`.
Pure subtrees that the peer resolver already short-circuits through
`purePkgs` (ported in #11906) now skip realization entirely — the tree
never gets walked past the first occurrence of those packages.
Bench (astro deep tree, cold store, single resolver phase):
- tree nodes: 74,940 → 4,069 (~18× smaller)
- `resolve_importer`: 11.6s → 8.2s (~1.42× faster)
Refs #11907.
* fix(resolving-deps-resolver): fix doc + dylint failures from #11915
- Re-export `TreeChildren` and `ChildEdge` from the crate root so the
intra-doc links from public docs (`resolve_peers`, `ResolvedTree`)
resolve. They were `pub` on the enum/struct but unreachable because
`resolved_tree` is a private module.
- Drop the `[Walker::realize_children]` / `[Walker::pure_pkgs]`
intra-doc references from `resolve_peers`'s public doc — `Walker`
is private, so the links failed under `--document-private-items`
with `-D rustdoc::private_intra_doc_links`. The prose still names
the items in plain backticks.
- Rename closure params `p` and let binding `v` in `realize_children`
to satisfy `perfectionist::single-letter-{closure-param,let-binding}`.
* fix(resolving-deps-resolver): persist first-walk is_leaf for lazy realization
Mirrors upstream's
[`ResolvedPackage.isLeaf`](https://github.com/pnpm/pnpm/blob/b9de85dcb6/installing/deps-resolver/src/resolveDependencies.ts#L250)
field: `pkg_is_leaf(&result)` is computed once on the first walk and
stored on `ResolvedPackage::is_leaf`; the peer-resolver's
`realize_children` reads it back instead of inferring leaf-ness from
`children_by_id.is_empty() + peer_dependencies.is_empty()`.
The inferred check was a weaker approximation — a package with a
missing manifest (e.g. a git/tarball/local resolution where `result.
manifest` is None) lands on `pkg_is_leaf == false` in the eager walk
but on `is_leaf == true` in the realize path, which would collapse
distinct per-occurrence `NodeId`s onto a shared `NodeId::leaf` and
break the peer resolver's per-call-site state.
Matches upstream's
[`buildTree` consumption](https://github.com/pnpm/pnpm/blob/b9de85dcb6/installing/deps-resolver/src/resolveDependencyTree.ts#L381):
`ctx.resolvedPkgsById[child.id].isLeaf` is the source of truth, not
recomputed per realization.
* style(resolving-deps-resolver): apply cargo fmt for is_leaf binding
* test(resolving-deps-resolver): cover lazy children edge cases
Adds two regression tests for the lazy-children mechanism introduced
in #11915 that the existing coverage didn't hit:
1. `revisit_with_no_manifest_child_keeps_per_occurrence_node_id` —
a child whose first walk produced `result.manifest == None`
(the shape git / tarball / local resolvers return) must keep the
non-leaf classification on every lazy realisation. Without the
`ResolvedPackage::is_leaf` persistence the realizer would mis-
classify it as a leaf and collapse distinct occurrences onto a
shared `NodeId::Leaf`, breaking per-call-site state.
2. `pure_revisit_leaves_lazy_children_unrealized` — a pure pkg
reached through multiple parents only realises its children for
the occurrence the peer resolver walks first. Subsequent
occurrences hit the `purePkgs` short-circuit before
`realize_children` runs, so their `TreeChildren::Lazy` stays
Lazy. Regression guard against accidentally moving the realise
call above the short-circuit.
Both tests were validated by breaking the relevant subject (swapping
`is_leaf` back to the inferred check; moving `realize_children`
above the `purePkgs` gate) and confirming they fail cleanly.
* style(resolving-deps-resolver): drop hyphenated mis- in test docs
CI's typos pass at .typos.toml flags `mis-` (suggests "miss" /
"mist"). Use "misclassify" instead of "mis-classify" — same
word, no hyphen, no typo hit.
|
||
|
Zoltan Kochan
|
b9de85dcb6 |
ci(pacquet): drop pnpm comparison and self-compare on main from integrated-benchmark (#11913)
The pnpm-CLI baseline was useful while pacquet was slower than pnpm; now that pacquet is the perf target itself, comparing against pnpm on every run is noise. Drop --with-pnpm from every scenario step. When the workflow runs on main, HEAD and main point at the same commit, so a HEAD-vs-main comparison is wasted work. Resolve the target list at job level: pacquet@HEAD on main, pacquet@HEAD pacquet@main everywhere else (PRs, workflow_dispatch from non-main). The Bencher upload already filters to pacquet@HEAD, so the single-target result still lands on the main baseline as before. |
||
|
Zoltan Kochan
|
f5d7723f3a |
perf(pacquet): port pnpm's purePkgs + peersCache for peer resolution (#11906)
* test(resolving-deps-resolver): port four peer-resolution cases from pnpm Pacquet's `mod peers` test block had five tests, all of which exercise single-occurrence happy paths. None covered the harder branches the upstream resolver is designed to handle — cycles, packages reached twice with divergent peer scope, parallel peer chains, transitive peer issues. That gap left the peer resolver under-tested even for the current algorithm, and would have made it dangerous to land the `peersCache` + `purePkgs` ports tracked in #11907 because the new cache lookup short-circuits exactly the branches no existing test exercises. Port four resolver-layer cases from [`installing/deps-resolver/test/resolvePeers.ts`](https://github.com/pnpm/pnpm/blob/c86c423bdc/installing/deps-resolver/test/resolvePeers.ts): - `cyclic_peer_dependencies_resolve_cleanly` — four-way cycle (foo ↔ bar ↔ qar ↔ zoo), every node lands in the graph without the walker panicking on cycle re-entry. Upstream `:14`. - `revisit_resolves_peer_in_one_occurrence_misses_in_other` — same package reached via two parent chains, one where the peer resolves and one where it's missing; both occurrences must surface with distinct depPaths. Upstream `:128`. - `two_peer_chains_resolve_against_their_own_sibling` — two parallel pkg-with-peer chains in the same importer; each picks its own sibling, no cross-pollination. Substitutes for upstream's `'resolve peer dependencies with npm aliases'` (`:573`) since npm-alias plumbing isn't yet wired through the test stub resolver — the TODO captures the gap so a follow-up can swap in the alias form once it lands. - `bad_peer_inside_subtree_records_resolved_from_parent` — a peer reachable through a subdependency but at the wrong version surfaces as a *bad* peer, not a missing one. Stands in for upstream's `'unmet peer dependency issue resolved from subdependency'` describe-block (`:502`); the `resolvedFrom` field upstream tracks isn't exposed on pacquet's `PeerDependencyIssue` yet, so the test asserts the bad/missing classification only. All four pass on `main`. Together they exercise the parent-context matching that future cache optimizations (#11907) need to get right — the second test in particular drives a shared-subtree shape where a naive NodeId-keyed cache returns stale depPaths. Refs #11907. * perf(resolving-deps-resolver): port pnpm's purePkgs + peersCache for peer resolution The peer resolver was rewalking every `NodeId` in the tree from scratch and recomputing the full per-package peer set on each hoist-loop iteration. On the `astro@^5` install (~1.6k unique packages, deep transitive shape) that pushed `resolve_peers` to 3.8 s of an 8.5 s `resolve_importer` phase — pacquet had already recorded `is_pure` per graph node but wasn't using it as a cache key, and `peersCache` was deferred in the original slice. Port both upstream optimizations and remove the unsafe `node_dep_paths` shortcut at the top of `resolve_node` that silently returned stale `depPath`s when the same shared `NodeId` got walked under two different parent peer contexts (an inevitable shape post-isNew-gate, and the bug `revisit_resolves_peer_in_one_occurrence_misses_in_other` catches). ## `purePkgs` fast path A `HashSet<String>` of `pkgIdWithPatchHash` values whose full subtree resolved with zero external peers and zero missing peers. Populated bottom-up: a node is added when `is_pure` is true after its own walk completes. A revisit of any pure pkg whose own `peerDependencies` is empty short-circuits with `depPath = pkgIdWithPatchHash` — no recursion, no peersCache lookup. Mirrors upstream's `purePkgs` early-return (resolvePeers.ts:398-406 at |
||
|
Zoltan Kochan
|
add6c794f1 |
feat(registry): implement pnpm-registry server and adopt it in pacquet's test mock (#11898)
Creates a working pnpm-compatible npm registry server (verdaccio analogue, in Rust) — and replaces `@pnpm/registry-mock`'s Node + Verdaccio launcher in pacquet's test setup with the new binary, against `@pnpm/registry-mock`'s shipped storage.
### What `pnpm-registry` does
- **HTTP server** (axum + tower-http) with the three endpoints pnpm/npm clients need:
- `GET /<pkg>` — packument (`/{name}` and `/{scope}/{name}`)
- `GET /<pkg>/<version-or-tag>` — single-version manifest, resolves `dist-tags` and rewrites `dist.tarball` to point at this server
- `GET /<pkg>/-/<tarball>` — tarball, streamed
- **Two modes:**
- **Proxy** — fetches missing packuments/tarballs from a configurable upstream (defaults to `https://registry.npmjs.org`), caches to disk
- **Static** (`--static`) — serves the storage directory verbatim, 404s on cache miss
- **Verdaccio-shaped on-disk storage** (`<root>/<pkg>/package.json` + flat tarballs) — drop-in compatible with the storage `@pnpm/registry-mock` publishes
- **Tarball streaming** — cache hits stream off disk; cache misses tee upstream chunks into a temp file via an mpsc channel and forward them to the client at the same time, atomically renaming on success and abandoning on upstream error or client disconnect
- **Tuned HTTP client** — wraps `pacquet_network::ThrottledClient::new_for_installs()`, inheriting pnpm's tuned defaults (`User-Agent: pnpm`, HTTP/1.1, hickory DNS, connection-pool tuning, concurrency semaphore)
- **Gateway-style status mapping** — `is_timeout()` → 504, `is_connect()` → 503, everything else (incl. upstream 5xx) → 502. No proxy-side retry (the pnpm client already has `fetch-retries`; stacking retries would only multiply latency on real failures).
### What changed in pacquet
- `pacquet/tasks/registry-mock` now spawns `pnpm-registry` against `node_modules/@pnpm/registry-mock/registry/storage-cache` (proxy mode with `npmjs.org` upstream and a 1-year packument TTL — matching `@pnpm/registry-mock`'s `'**': proxy: npmjs` verdaccio config). No more Node, no more Verdaccio, no more `launch.mjs`, no more process-tree walk to kill child verdaccios.
- `@pnpm/registry-mock` stays as a devDep — only for the storage data it ships, not the launcher.
### Tests
- **36 pnpm-registry tests** (12 unit + 7 against `@pnpm/registry-mock` storage in static mode + 17 mockito-based proxy/cache/streaming): packument rewrite, version-manifest resolution, tarball streaming (large body, cache finalize, mid-stream upstream error, client disconnect mid-stream, concurrent fetches → one cache file), gateway status mapping (504/503/502), stale-cache fallback on upstream failure, TTL refresh, invalid-package-name 400, scoped vs unscoped routing.
- **Full pacquet test suite** (2043 tests) runs green against `pnpm-registry`-backed mock.
### CI
- `pacquet-ci.yml` and `pacquet-codecov.yml` path filters now include `registry/**` (so registry-only PRs trigger the workspace CI); typos checker covers `registry` too. The workflow name stays "Pacquet CI" but a header comment explains the intentional cross-stack scope.
- `just registry-mock launch` pre-builds with `cargo nextest run --no-run` (workspace-wide) so its fingerprint matches what `just test` will later need — without this, Windows MSVC fails with `os error 5` trying to re-link the running `pnpm-registry.exe`.
### Crates.io name reservations (from the original scaffold commit)
- [`pnpm-registry`](https://crates.io/crates/pnpm-registry) — published from this repo
- [`pnpm-registry-cli`](https://crates.io/crates/pnpm-registry-cli) / [`pnpm-registry-server`](https://crates.io/crates/pnpm-registry-server) — placeholder stubs, name reservation only
|
||
|
Zoltan Kochan
|
e549cd1cf1 |
perf(pacquet): no-op short-circuit when node_modules is up to date (#11904)
* perf(pacquet): no-op short-circuit when node_modules is up to date Adds a fast-path gate in `Install::run` that mirrors upstream pnpm's `validateModules` + `allProjectsAreUpToDate` shortcut: when the frozen-lockfile dispatch is eligible, `.modules.yaml` agrees with the current config, and `<virtual_store_dir>/lock.yaml` is byte-equal to the wanted lockfile, skip materialization entirely. The install emits the `name: "pnpm" / level: "info"` "Lockfile is up to date, resolution step is skipped" log, refreshes the workspace-state timestamp so `pnpm run`'s `verifyDepsBeforeRun` doesn't fire spuriously, and returns. Closes #11899. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(pacquet): tighten short-circuit test assertions Address review feedback: - Use `r#"..."#` raw-string for the up-to-date log assertion message so dylint's `perfectionist::prefer-raw-string` lint stops flagging the escaped quotes inside it. - Loosen the "up-to-date log fires" check to `any(|e| matches!(...))` so unrelated future `LogEvent::Pnpm` emits don't make the test brittle. - Swap `Path::exists()` for `std::fs::symlink_metadata().is_err()` on the "link: dep not materialized" assertion so a dangling symlink (which `exists()` reports as `false`) wouldn't sneak past. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
mehmet turac
|
a456dc78fb | fix(list): limit manifest reads for large workspaces (#11692) | ||
|
|
572842a039 |
fix(installing.commands): clarify "loose mode" wording in minimumReleaseAge log (#11763)
* fix(installing.commands): clarify "loose mode" wording in minimumReleaseAge log The log line printed when pnpm auto-adds entries to `minimumReleaseAgeExclude` referred to internal "loose mode" terminology, which doesn't appear in the docs and isn't discoverable. Point users at the actual setting name they need to flip. Closes #11747 * Update installing/commands/src/policyHandlers.ts Co-authored-by: Zoltan Kochan <z@kochan.io> * fix(installing.commands): name the value in minimumReleaseAgeStrict log hint Change "set minimumReleaseAgeStrict to gate these updates with a prompt" to "set minimumReleaseAgeStrict to true to ..." so the value is explicit. --------- Co-authored-by: shiminshen <16914659+shiminshen@users.noreply.github.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
9a3207367d | chore: update pnpm and pacquet | ||
|
Zoltan Kochan
|
bcbc008f2d | fix: temporarily disabling pacquet for release v11.3.0 | ||
|
Zoltan Kochan
|
6316e7b275 |
fix(deploy): skip configDependencies in the nested install (#11895)
* fix(deploy): skip configDependencies in the nested install The deploy directory never installs configDependencies, so the install engine they designate (e.g. pacquet) isn't on disk to invoke. Without this override, `pnpm deploy` crashes with `ENOENT: ... lstat '<deployDir>/node_modules'` when the workspace declares pacquet under `configDependencies`. * test(deploy): cover deploy with pacquet in configDependencies Reproduces the ENOENT crash that happens when `deployFromSharedLockfile` forwards the workspace's `configDependencies` (e.g. pacquet) into its nested install and the install engine tries to spawn from `<deployDir>/node_modules/.pnpm-config/`. * test(deploy): clarify the public-registry comment in the pacquet deploy test |