mirror of
https://github.com/pnpm/pnpm.git
synced 2026-07-20 04:32:26 -04:00
node-runtime-fix
11458 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Kochan
|
2844302bd1 | ci: don't verify deps before run | ||
|
Zoltan Kochan
|
ae703b1bcd |
fix(test-ipc-server): allow tilde in shell-arg paths for Windows 8.3 short names (#11661)
`os.tmpdir()` on GitHub's Windows runners returns the 8.3 short-name form
of the user-profile directory (e.g. `C:\Users\RUNNER~1\AppData\Local\Temp`)
because `runneradmin` is longer than 8 characters. The `~` then trips the
`quoteShellArg` allowlist regex and every test that calls `sendLineScript`
or `generateSendStdinScript` throws "Unsupported character in shell argument".
The tilde is safe to allow:
- cmd.exe performs no tilde expansion at all.
- POSIX shells only expand `~` when it is unquoted at the start of a word;
inside the double-quoted `"${arg}"` wrapper produced here it is literal.
The matching CodeQL shell-injection sanitization argument is unchanged —
the allowlist is still anchored and still rejects every metacharacter.
The bug was masked until #11659 because the Windows test legs had been
silently no-op'ing since #11608.
---
Written by an agent (Claude Code, claude-opus-4-7).
v11.1.2-canora.10
v11.1.2-canora.8
v11.1.2-canora.7
|
||
|
Zoltan Kochan
|
e8fc34389a |
ci: pin Run tests step to bash so $TEST_SCRIPT expands on Windows (#11659)
Without an explicit shell, the step ran under PowerShell on windows-latest, where `$TEST_SCRIPT` is not a variable (PowerShell exposes env vars as `$env:TEST_SCRIPT`). `pn run ""` then exited 0 and just listed available scripts — the Windows test legs have been silently no-op'ing since the env-var move in #11608. The sibling `Verify Node version` and `Determine test scope` steps already pin `shell: bash`; this brings `Run tests` in line. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
a4f3c6d3b7 |
ci: enable manual releases for pacquet and pnpm (#11652)
Two related workflow changes: ### `pacquet-release-to-npm.yml`: switch to `workflow_dispatch` The trigger was "push to main touching `pacquet/npm/pacquet/package.json`" — the version came from a committed bump and the workflow auto-fired on every such commit. Switch to `workflow_dispatch` only, with a `version` input (validated as semver). The workflow patches `pacquet/npm/pacquet/package.json` before `generate-packages.mjs` runs, so the version is single-sourced from the manual trigger rather than needing a separate commit to bump the manifest first. The committed manifest now omits the `version` field entirely — it only exists at release time inside the runner. Dropped along the way: - The `check` job (EndBug/version-check against unpkg) — no longer needed when the operator types the version. - The `Create GitHub Release` step — no draft release, no `v*.*.*` git tag. The pacquet `v0.x.x` tag scheme collided with pnpm's `v11.x.x`; npm is the authoritative artifact store and provenance attestations stay attached via `--provenance` on `pnpm publish`. - `contents: write` on the publish job (no longer needs to create a tag). ### `release.yml`: add `workflow_dispatch` as a lib-only republish path Add a `workflow_dispatch:` trigger alongside the existing tag-push trigger. Tag-push behaves exactly as before. Manual dispatch becomes a fast **lib-only republish** path — useful after a version bump to one or more lib packages that doesn't warrant a full CLI release. On `workflow_dispatch` from any ref, the following are skipped (guarded with `if: startsWith(github.ref, 'refs/tags/')`): - `Publish @pnpm/exe` step — also contains the multi-minute `build-artifacts` call. - `Publish pnpm CLI` step. - `Copy Artifacts`, `Attest build provenance` (the `dist/*` attestation), `Generate release description`, `Release` (`softprops/action-gh-release`) — these are the GitHub-Release-side ceremony. Without an explicit `tag_name`, `softprops/action-gh-release@v2.5.0` defaults to `github.ref_name`, which on a manual dispatch from main would create a junk release tagged literally `main`. What still runs on `workflow_dispatch`: - `actions/checkout`, garnet scan, `pnpm/setup` - `Publish internal workspace packages (static token)` — i.e. `pn publish --filter=!pnpm --filter=!@pnpm/exe --access=public --provenance` Compilation is handled by each lib package's own `prepublishOnly: tsgo --build` hook (which `pnpm publish` runs automatically), same as the existing tag-push flow. The npm registry rejects any version already on it, so re-running on an already-released tree is a no-op — that's the safety net for accidental clicks. ## How to use **pacquet release**: Actions → Release Pacquet → Run workflow → fill in `version` (e.g. `0.2.3` or `0.2.3-rc.1`) → Run. No tag, no GitHub release. **pnpm full release**: still triggered by a `v*.*.*` tag push. Publishes @pnpm/exe + libs + CLI, attests, copies artifacts, creates a draft GitHub release. **pnpm lib-only republish**: Actions → Release → Run workflow → choose `main` → Run. Publishes just the internal workspace packages from whatever versions are currently in each `package.json`. Skips CLI, @pnpm/exe, build-artifacts, GitHub release. |
||
|
Zoltan Kochan
|
1575076d08 |
chore(pacquet): fold registry-mock into root workspace, fix npm metadata (#11643)
* chore(pacquet): fold registry-mock into root workspace, fix npm metadata
Two unrelated cleanups bundled because they touch the same publishing/
workspace plumbing:
1. **`pacquet/npm/pacquet/package.json` metadata** — the imported file
still pointed at the standalone repo: `repository.url` was
`pnpm/pacquet`, `repository.directory` was `npm/pacquet`, `homepage`
and `bugs` likewise. Repoint at `pnpm/pnpm`. Update
`generate-packages.mjs` so the per-platform packages it emits at
release time also point at `pnpm/pnpm`.
2. **Fold `pacquet/tasks/registry-mock` into the root pnpm workspace**.
Pacquet's standalone-repo nested-workspace setup pinned
`nodeLinker: hoisted` "for verdaccio CJS resolution," but pnpm's
own jest globalSetup (`__utils__/jest-config/with-registry/`) calls
the same `@pnpm/registry-mock.start()` API under the default
isolated linker without issue, and verified locally that
`node launch.mjs prepare` works after consolidation. The hoisted
constraint was scoped to standalone-pacquet's install pattern; in
the monorepo it's unnecessary.
Changes for (2):
- Add `pacquet/tasks/registry-mock` to `pnpm-workspace.yaml`.
- Rename the package `@pnpm-private/pacquet-registry-mock-launcher`
(private, matches the `@pnpm-private/*` convention used by other
internal workspace members) and switch `@pnpm/registry-mock` to
`catalog:` (the root catalog already pins it at 6.0.0).
- Delete `pacquet/tasks/registry-mock/pnpm-lock.yaml` and
`pnpm-workspace.yaml` — root install handles both now.
- Delete `pacquet/package.json` and `pacquet/pnpm-lock.yaml` — the
file only had a `cargo build` script + `devEngines: pnpm`, both
already covered by root, and nothing referenced it.
- `justfile install` is now just `pnpm install` (was
`cd pacquet/tasks/registry-mock && pnpm install --frozen-lockfile`).
- `pacquet-integrated-benchmark.yml` path filter and cache key
swap the deleted nested lockfile for the root `pnpm-lock.yaml` /
`pnpm-workspace.yaml`.
Verified: `pnpm install` resolves the workspace member, the lockfile
gains a `pacquet/tasks/registry-mock` importer entry, and
`pacquet/tasks/registry-mock/node_modules/@pnpm/registry-mock` is
linked correctly under the isolated layout.
* fix(pacquet): match meta-updater conventions in registry-mock launcher
The previous version of `pacquet/tasks/registry-mock/package.json`
omitted `version`, which crashed `pn lint:meta` (meta-updater hits
`manifest.version!.split('.')[0]` for every workspace package).
Backfill all the fields meta-updater would emit for an internal
`@pnpm-private/*` private package, matching `__typecheck__/package.json`
and friends:
- `version: 1100.0.0` (the pnpm 11.x convention for non-experimental
internal packages)
- self-devDep entry (`workspace:*`) that meta-updater would otherwise
inject
- `keywords: [pnpm, pnpm11]`
- `repository` pointing at this directory inside pnpm/pnpm
This is the same shape every other `@pnpm-private/*` private workspace
member uses; it lets `pn lint:meta --test` pass without modifying the
file.
* fix: update lockfile
* chore(pacquet): add build:pacquet script at root
Restore the `cargo build --release --bin pacquet` shortcut that lived in
the deleted `pacquet/package.json`. Naming it `build:pacquet` (rather
than `build`) matches the existing namespacing convention in this
file (`lint:ts`, `lint:meta`) and leaves room for a general `build`
script later. Invoke with `pnpm build:pacquet` or `pn build:pacquet`
from the repo root.
|
||
|
Zoltan Kochan
|
2e33bb1955 |
docs: split AGENTS.md into shared + pacquet-specific (#11640)
* docs: split AGENTS.md into shared + pacquet-specific
Before: root AGENTS.md and pacquet/AGENTS.md each maintained their own
copy of the GitHub PR workflow, agent-footer rule, "never ignore test
failures," Conventional Commits list, and code-reuse philosophy.
Drift waiting to happen.
After:
- Root AGENTS.md owns the shared conventions (PR workflow, agent
footer, conventional commits, code reuse, never-ignore-tests,
PR-conflict script) and marks TS-only sections explicitly
(setup/build, testing, linting, changesets, Standard Style, Jest
gotchas).
- pacquet/AGENTS.md opens with "Read ../AGENTS.md first" and keeps
only pacquet-specific rules (cardinal rule, branded types, just
recipes, insta snapshots, miette diagnostics, Rust style notes,
the `bench:` commit type, things-not-to-do that are Rust-flavored).
- Root adds a one-line entry for `pacquet/` in the repo structure
list so first-time readers find the cross-link.
CLAUDE.md and pacquet/{CLAUDE,GEMINI}.md are unchanged — they're
symlinks to AGENTS.md and follow automatically.
* docs(agents): require parity between pnpm and pacquet
Add a "Keep pnpm and pacquet in sync" section to root AGENTS.md spelling
out the bidirectional obligation: any user-visible change (CLI surface,
lockfile/manifest format, error codes, defaults, env-var handling, log
emissions, store layout) must land in both stacks in the same PR, or
the originating PR must spawn a tracking issue. Pure refactors / perf
wins / TS-only test cleanups don't need mirroring.
Cross-link from pacquet/AGENTS.md's "cardinal rule" so a pacquet-side
reader knows the obligation goes both ways and where the pnpm-side
version lives.
* docs(agents): restore Rust-specific dependency-level guidance
The root "Keep the dependency on the right level" bullet uses npm
vocabulary ("package," "shared package"). For a Rust reader that
required mentally translating "package" → "crate" and made the
workspace-vs-crate distinction less obvious. Restore the pacquet
phrasing alongside the existing pacquet-specific notes.
* docs(agents): hand off cross-stack porting via the same PR
Drop the "open a tracking issue" fallback — it lets one side drift
behind while the issue sits in the backlog. Instead, the PR author
opens the PR with their side and flags in the description what still
needs porting; someone else pushes the matching commits to the same
PR before it lands. Both sides land together or not at all.
* docs(agents): drop external-repo framing from the cardinal rule
pacquet now lives in the same repo as pnpm, so the cardinal rule no
longer needs the "fetch pnpm/pnpm main, compare ls-remote SHAs, watch
your local clone for drift" mechanics. The reference TypeScript code
is just a few directories over (`pnpm/`, `pkg-manager/`, `resolving/`,
`lockfile/`, `store/`, etc.), and pnpm is the source of truth by
position in the repo, not by branch tracking.
Updates:
- Root `AGENTS.md`: rephrase the cross-link line to drop the "follow
pnpm's main" framing.
- `pacquet/AGENTS.md` cardinal rule: redirect "find the equivalent
code" from `https://github.com/pnpm/pnpm` to the in-repo
TypeScript workspaces, drop the "confirm you're on the freshest
main" paragraph, and reword the source-of-truth wording.
- Permalink citation rule: generalize from "upstream pnpm" to "any
GitHub repository, including this one" — citation SHAs now usually
point at this repo's history.
* docs(agents): note pacquet's current scope is install-only
Without this caveat the parity rule reads as if every command needs
porting today. pacquet only implements `install` right now; resolution
and other commands (`update`, `add`, `remove`, `publish`, `exec`,
`run`, `dlx`, `audit`, etc.) live only in TypeScript, so changes there
don't need a pacquet-side port. The caveat also flags that the parity
rule's scope will widen as pacquet ports more commands.
|
||
|
Zoltan Kochan
|
d2b64b6689 |
ci(pacquet): fix all zizmor code-scanning findings (#11641)
* ci(pacquet): fix all zizmor code-scanning findings Resolves the 90 alerts opened by zizmor against the imported pacquet-* workflows and shared composite actions: - unpinned-uses: pin every third-party action to a SHA + version comment (matching SHAs already used elsewhere in the repo where applicable; taiki-e/install-action collapsed onto v2.78.0 with explicit `tool:` input). - artipacked: add `persist-credentials: false` to every actions/checkout. - template-injection: pass `inputs.*` and `steps.*.outputs.*` through `env:` in binstall/rustup composite actions and pacquet-release-to-npm.yml. - excessive-permissions: add top-level `permissions: contents: read` to pacquet-release-to-npm.yml; move issues/pull-requests writes from the workflow level to the benchmark-compare job in pacquet-micro-benchmark.yml. - dangerous-triggers: keep workflow_run in pacquet-integrated-benchmark- comment.yml but suppress with a documented zizmor: ignore — the trigger is the recommended pattern for posting comments back to fork PRs. - superfluous-actions: keep softprops/action-gh-release with a zizmor: ignore (matches release.yml). Verified by running `zizmor .github` locally with no remaining findings. * ci(pacquet): point SHA pins at the patch-version tag Swatinem/rust-cache and montudor/action-zip were pinned to the SHA the major-version alias (`v2`, `v1`) resolves to, but the version comments claimed `v2.9.1` / `v1.0.0`. zizmor's online `ref-version-mismatch` audit flagged the inconsistency. Repoint at the SHAs the patch-version tags actually annotate so the pin and the comment agree. |
||
|
Zoltan Kochan
|
763ddf1c99 |
chore(pacquet): wire pacquet workflows into monorepo (#11635)
* chore(pacquet): wire pacquet workflows into monorepo
Move Cargo workspace, Rust toolchain configs, justfile, composite actions,
and 7 workflow files out of `pacquet/` and up to the repo root so:
- cargo / just / taplo run from repo root, the way the rest of the
monorepo's tooling does
- GitHub Actions actually discovers the workflows (it only reads
`.github/workflows/` at the repo root)
Workflows are prefixed with `pacquet-` and renamed to "Pacquet ..." so
they don't collide with the existing pnpm CI. Path filters are scoped
to `pacquet/**` so they don't trigger on every commit. The cargo entry
from pacquet's standalone `dependabot.yml` is folded into the root one;
pacquet's `CODEOWNERS` and `pull_request_template.md` are dropped because
the root copies supersede them.
Path rewrites:
- `Cargo.toml` workspace members → `pacquet/crates/*`, `pacquet/tasks/*`
- all path-deps in `[workspace.dependencies]` → `pacquet/...`
- `justfile` recipes (`install`, `install-hooks`) point at `pacquet/...`
- `.taplo.toml` include globs → `pacquet/crates/*/*.toml`, `pacquet/tasks/*/*.toml`
- `pacquet/npm/pacquet/scripts/generate-packages.mjs` REPO_ROOT walks one
more level up
- workflow `paths:` filters, `hashFiles(...)`, and shell paths all updated
Verified: `cargo metadata` resolves the workspace, `cargo fmt --check`
clean, `taplo format --check` picks up all 26 Cargo.tomls, `actionlint`
reports no new issues (the `type:`-on-input warnings on the rustup action
predate this move).
* chore(pacquet): drop pnpm version pin from pacquet CI workflows
The monorepo's root `package.json` declares `pnpm@11.1.1` under
`packageManager`, which conflicts with the workflows' explicit
`version: 11.0.0-rc.5` and trips `pnpm/action-setup` ERR_PNPM_BAD_PM_VERSION.
The pin was a pacquet-era workaround for the v9 lockfile while pnpm 11
was still pre-release. Stable 11.x writes v9 too, so let action-setup
read the version from `packageManager` like every other workflow in
this repo does.
* chore(pacquet): use pnpm/setup matching the rest of the monorepo
Replaces `pnpm/action-setup@v6` with the same `pnpm/setup@b1cac3...`
SHA the rest of pnpm/pnpm uses (release.yml, test.yml, ci.yml,
benchmark.yml, audit.yml). Reads pnpm version from `packageManager`
in root package.json, and skips the implicit `pnpm install` since
pacquet does its own scoped install via `just install` (which only
touches `pacquet/tasks/registry-mock/`).
The release workflow now also installs Node via the same action
(`runtime: node@22`) instead of via `pnpm runtime -g set node 22`,
since pnpm/setup handles runtimes in one step.
* chore(pacquet): tighten permissions and Dependabot cooldown
Address zizmor warnings on the pacquet CI changes:
- `dependabot.yml`: the cargo entry I added in the previous commit
inherited from pacquet's standalone repo and is missing the
`cooldown: default-days: 7` the github-actions entry uses. Add it
so cargo and github-actions debounce updates consistently.
- `pacquet-ci.yml`, `pacquet-codecov.yml`, `pacquet-cargo-unused.yml`
lacked a top-level `permissions:` block, so GITHUB_TOKEN inherited
the repo default. Declare `contents: read` — every job in these
workflows only reads the repo and runs local checks.
The other four pacquet workflows already declare permissions
explicitly (integrated-benchmark/comment, micro-benchmark, release).
* chore(pacquet): add "reimagining" to cspell dictionary
cspell at the repo root scans all `**/README.md` and was rejecting
`pacquet/README.md` and `pacquet/npm/pacquet/README.md`, which describe
pacquet as "not a reimagining of pnpm." Add the word to the existing
allow-list rather than rewording two READMEs imported from a separate
repo.
* fix(pacquet): prefix workspace-relative paths with pacquet/
Two Rust source files looked up paths off the cargo workspace root
(\`cargo locate-project --workspace\`), which now resolves to the
monorepo root rather than the pacquet directory. Add the \`pacquet/\`
prefix:
- \`tasks/registry-mock/src/dirs.rs\` — \`registry_mock()\` was
pointing the node launcher at \`<repo>/tasks/registry-mock/launch.mjs\`
instead of \`<repo>/pacquet/tasks/registry-mock/launch.mjs\`, which
failed every Pacquet CI test job ("Cannot find module ...launch.mjs").
- \`tasks/micro-benchmark/src/main.rs\` — same idea for the
fixtures folder.
|
||
|
Zoltan Kochan
|
4a89b06b44 |
chore: import pacquet repository into monorepo
Imports https://github.com/pnpm/pacquet at the pacquet/ subdirectory, preserving full commit history (rewritten with git filter-repo so blame and log work for files under pacquet/). |
||
|
Zoltan Kochan
|
ceec335e85 |
fix(package-manager): link optionalDependencies siblings into slot (#526)
`create_symlink_layout` only iterated `snapshot.dependencies`, so a
package whose CPU/OS-specific siblings live entirely under
`optionalDependencies` (e.g. `@typescript/native-preview`,
`@reflink/reflink`, every `*-darwin-arm64` / `*-linux-x64` family)
ended up with a slot `node_modules/<scope>/` containing only the
parent package — no platform binary sibling. Consumers that do
`require.resolve('@typescript/native-preview-darwin-arm64')` from
inside `getExePath.js` walked parent directories and found nothing,
so `tsgo --version` (and every other tool that delegates to a
platform variant) crashed with `Unable to resolve … missing the
package on disk`.
Port upstream's `dependencies ∪ optionalDependencies` merge — the
graph builder at
https://github.com/pnpm/pnpm/blob/da65e6262/deps/graph-builder/src/lockfileToDepGraph.ts#L150-L156
unifies both maps into one `allDeps` for each node's children, and
`linkAllModules` then symlinks every child with two short-circuits:
`alias === depNode.name` (a snapshot referencing itself) and
`!pkg.installable && pkg.optional` (a non-materialized optional).
See https://github.com/pnpm/pnpm/blob/da65e6262/installing/deps-installer/src/install/link.ts#L521-L549.
`create_symlink_layout` now takes both maps and a `SkippedSnapshots`
reference, merges them, and applies both short-circuits. The skip
set is threaded through `InstallPackageBySnapshot` (cold batch) and
the warm-batch `CreateVirtualDirBySnapshot` call site in
`CreateVirtualStore::run`, so a target dropped by the installability
pass, by `--no-optional`, or by a swallowed optional fetch failure
gets no dangling symlink.
Five new unit tests in `create_symlink_layout/tests.rs` cover the
matching-optional happy path, the skipped-optional dangling-link
guard, the self-name guard for entries listed in either bucket, the
both-buckets-absent no-op, and the alias-resolve path (aliased deps
still link the alias filename while resolving the slot via the
target's name). End-to-end verification: `pacquet install
--frozen-lockfile` followed by `tsgo --version` in the pnpm v11
repo now succeeds; the matching `native-preview-darwin-arm64`
sibling shows up in the slot's `node_modules/@typescript/`.
---
Written by an agent (Claude Code, claude-opus-4-7).
|
||
|
Zoltan Kochan
|
da65e62625 |
fix: pacquet install --frozen-lockfile works against pnpm v11 repos (#525)
* fix(lockfile): skip env document in pnpm v11 combined lockfiles Pnpm v11 writes `pnpm-lock.yaml` as a stream of up to two YAML documents: an optional first document carries the package-manager bootstrap (`packageManagerDependencies` and the snapshots that back it) and the second document is the regular project lockfile. Pacquet hands the file straight to `serde_saphyr::from_str`, which rejects a multi-document stream with "multiple YAML documents detected" — so every install against a v11 repo that has a `packageManager` / `devEngines.packageManager` declaration fails before staleness checking even runs. Port upstream's `extractMainDocument` to a new `yaml_documents` module: if the file starts with `---\n`, return the slice after the next `\n---\n` separator; if there is no second separator, the file is env-only and the loader returns `Ok(None)`. `load_from_path` threads the lockfile content through the filter before deserializing, matching pnpm's `_read` call site at https://github.com/pnpm/pnpm/blob/31858c544b/lockfile/fs/src/read.ts#L103-L110. Three unit tests in `yaml_documents/tests` mirror upstream's `extractMainDocument` test cases, and two integration tests in `load_lockfile/tests.rs` cover the combined-document and env-only paths end-to-end. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(package-manifest): reify devEngines.runtime into devDependencies With #511 / #512 pacquet recognises `runtime:` specifiers in the lockfile, so a frozen install against a v11 repo gets past the YAML parse but then trips the staleness check: the lockfile lists `node@runtime:24.6.0` under the root importer's `devDependencies`, while the on-disk `package.json` only declares the runtime through `devEngines.runtime`. The flat-record diff then surfaces a spurious "node@runtime:24.6.0 was removed" mismatch. Port upstream's `convertEnginesRuntimeToDependencies` to a new free function in `crates/package-manifest`. For each of `node`, `deno`, `bun`, if `devEngines.runtime` (or `engines.runtime`) declares the runtime with `onFail: "download"` and an explicit version, and the target dependencies bucket has no explicit entry yet, insert `<name>: "runtime:<version>"`. Array and single-object runtime shapes are both accepted. Skip when `onFail` is anything other than `"download"` or when the version is absent — upstream warns on the missing-version path; pacquet skips silently and the staleness check still surfaces the gap if it matters downstream. WebContainer's "no runtime download" branch is intentionally omitted since pacquet does not run there. `PackageManifest::read_from_file` calls the function for both `(devEngines, devDependencies)` and `(engines, dependencies)`, mirroring upstream's `convertManifestAfterRead` at https://github.com/pnpm/pnpm/blob/9cad8274fd/workspace/project-manifest-reader/src/index.ts#L227-L231. Six unit tests in `tests.rs` cover the happy path, the no-version skip, the non-`download` `onFail` skip, preservation of an explicit user-declared entry, the array-of-runtimes form, and the `engines` → `dependencies` variant. A seventh test exercises the hook through `PackageManifest::from_path` end-to-end. Upstream reference: https://github.com/pnpm/pnpm/blob/9cad8274fd/pkg-manifest/utils/src/convertEnginesRuntimeToDependencies.ts#L10-L45. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
b07b152c19 |
feat(lockfile,package-manager,real-hoist): npm-alias importer dep versions (#524)
pnpm writes importer dependency `version:` fields in three shapes:
bare semver-with-peer (`4.0.0`), `link:<path>`, or — when a specifier
(typically `catalog:`) resolves to a different package name — the full
npm-alias `<name>@<version>`. The third shape is what
`refToRelative` recognises with the same leading-`@` / `@` before
`(`/`:` test that `SnapshotDepRef` already uses, and which pnpm v11
emits for entries like:
js-yaml:
specifier: 'catalog:'
version: '@zkochan/js-yaml@0.0.11'
Pacquet's `ImporterDepVersion` only modelled `Regular` and `Link`, so
deserialising a lockfile with an aliased catalog dep failed with
"Failed to parse importer dependency version".
Add an `Alias(PkgNameVerPeer)` variant and a `resolved_key` helper
that returns the correct snapshot-map key for each shape — the
importer-map key paired with the version for `Regular`, the alias's
own `(name, suffix)` for `Alias`, and `None` for `Link`. Every site
that previously built a key from `as_regular().map(|v| PkgNameVerPeer::new(name, v))`
now goes through `resolved_key`, so aliased deps reach the snapshot,
the skipped-set, the reachability BFS, the build-sequence root walk,
the runtime exclusion check, and both hoist passes correctly.
For symlink targets, an aliased dep links the importer-key name to
`<slot>/node_modules/<alias-real-name>` (the resolved package's true
name inside its slot), matching pnpm's `linkDirectDeps`. The
`pnpm:root added` event now reports `realName` as the resolved
package name for aliases, where before it always echoed the
importer-map key.
Upstream reference: `refToRelative` in
`pnpm/pnpm@8a80235c7b/deps/path/src/index.ts:96-110`.
|
||
|
Zoltan Kochan
|
8a80235c7b | chore(release): 11.1.2 v11.1.2 | ||
|
wolf-j3blair
|
18a464f5b4 |
fix(network): strip sec-fetch-* headers to fix Azure DevOps Artifacts 400 errors (#11602)
* fix(network): strip sec-fetch-* headers to fix Azure DevOps Artifacts 400 errors undici's fetch() automatically adds sec-fetch-* headers (e.g. sec-fetch-mode: cors) per the Fetch spec. Azure DevOps Artifacts interprets these as browser requests and returns HTTP 400 for uncached upstream packages. Since pnpm is a CLI tool, these headers serve no purpose. Adds a stripSecFetchHeaders interceptor applied to all dispatchers (global, proxy, and non-proxy) via undici's compose() API. Fixes #11572 * refactor: fix header types and function placement in stripSecFetchHeaders - Widen header type from Record<string, string> to Record<string, string | string[] | undefined> to match Dispatcher.DispatchOptions - Move stripSecFetchHeaders below its first use, relying on function hoisting per codebase conventions * refactor(network.fetch): handle iterable header form and tidy test `Dispatcher.dispatch` accepts headers as a Map/web-Headers iterable in addition to the flat string[] and plain object forms. The previous object branch routed iterables through Object.entries, which would silently drop every header for Map-like inputs. Detect Symbol.iterator and consume the iterator directly when present. Also drop the underscore prefix on the test's `req` parameter since it is used. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
MCMXC
|
8c06d1a2f9 |
fix: preserve named catalog group during interactive upgrade --latest (#11567)
When upgrading a dependency that uses a named catalog (e.g. "catalog:foo"), the previous specifier's catalog name now takes priority over the global saveCatalogName option. This prevents the package.json from being rewritten to "catalog:" and the updated version from landing in the default catalog instead of the named one. Closes #10115 Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Katerina Skroumpelou
|
e526f89650 |
fix(npm-resolver): minimumReleaseAge handling for cached abbreviated metadata (#11622)
* fix(npm-resolver): dont rethrow ERR_PNPM_MISSING_TIME from version-spec cache * fix(npm-resolver): upgrade cached abbreviated metadata on 304 for minimumReleaseAge * fix(npm-resolver): expand abbreviated-meta upgrade to in-memory cache and preferOffline paths * fix(npm-resolver): address Copilot review feedback on pickPackage - Extract `persistUpgradedMeta` helper and call it from all three sites (in-memory cache hit, preferOffline disk-cache hit, 304 path) so a fresh process doesn't repeat the upgrade fetch. - Forward `etag`/`modified` to the upgrade fetch in `maybeUpgradeAbbreviatedMetaForReleaseAge` so the registry can answer 304. - Extract `shouldRethrowFromFastPathCache` so the two fast-path catch sites can't drift on the MISSING_TIME-vs-strict invariant. - Document the deliberate choice to upgrade-fetch when `meta.modified` is absent or unparseable (correctness over saving a network call). - Add a companion test that exercises the catch fix with the default `ignoreMissingTimeField` so the invariant holds regardless of that flag. - Fix the existing `bareSpecifier: '3.1.0'` test setup: 3.1.0 was published 2016-01-11, after the test's `publishedBy` of 2015-08-17, so strict mode correctly rejected it. Switch to 3.0.0 (released 2015-07-10). * chore(npm-resolver): replace 'unparseable' with 'malformed' for cspell * style(npm-resolver): declare pickPackage helpers after their caller --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
fb221c626c |
feat(cmd-shim,package-manager): hoisted-bin precedence + post-build top-level bin re-link (#342) (#523)
Two related bin-linking behaviors deferred from #333. Behavior 1 — hoisted-bin precedence: - Add `BinOrigin { Direct, Hoisted }` discriminator to `PackageBinSource`. - New top tier in `pick_winner`: Direct wins outright over Hoisted regardless of ownership / lexical order. Mirrors upstream's `preferDirectCmds` partition at https://github.com/pnpm/pnpm/blob/4750fd370c/bins/linker/src/index.ts#L92. - New `link_top_level_bins(modules_dir, direct, hoisted)` helper in `pacquet-package-manager` mixes both candidate lists into one `link_bins_of_packages` call so the new tier resolves conflicts in a single pass — previously the two passes (SymlinkDirectDependencies for direct + hoist pass for publicly-hoisted) wrote shims independently and a hoisted bin could shadow a direct one when its package name was lexically smaller. Behavior 2 — lifecycle-script-created bins: - Add a post-`BuildModules` per-importer top-level bin link pass. Re-reading each direct dep's `package.json` after lifecycle scripts run picks up bins generated by `postinstall` (the `@pnpm.e2e/generated-bins` upstream fixture). Idempotent for unchanged shims via `is_shim_pointing_at`. - Mirrors upstream's `linkBinsOfImporter` pass that runs after `buildModules` at https://github.com/pnpm/pnpm/blob/4750fd370c/installing/deps-installer/src/install/index.ts#L1539. Supporting changes: - `PackageBinSource::new(location, manifest)` constructor + `with_origin` builder so existing call sites don't have to spell out the new field. - Public `direct_dep_names_for_importer` helper extracted from `SymlinkDirectDependencies` so the post-build pass uses the same filter (skipped / first-wins / link_only) as the symlink phase. - `InstallFrozenLockfileError::TopLevelBinLink` for the new failure surface. Tests: - `direct_origin_wins_over_hoisted_regardless_of_lexical` — pins the new tier overrides lexical ordering. - `hoisted_origin_loses_to_existing_direct` — pins both arms of the new tier (Direct incumbent vs Hoisted candidate). |
||
|
Benjamin Staneck
|
180aee9ba5 |
fix: handle lockfile conflicts in optimistic install (#11605)
* fix: handle lockfile conflicts in optimistic install * test: move sharedWorkspaceLockfile out of WorkspaceState.settings `sharedWorkspaceLockfile` is not in `WORKSPACE_STATE_SETTING_KEYS`, so placing it inside `WorkspaceState.settings` broke type checking. Pass it directly on the `CheckDepsStatusOptions` instead. * chore: add lockfile/fs project reference to installing/commands tsconfig `@pnpm/lockfile.fs` was added as a runtime dependency but the tsconfig project references were not updated. meta-updater enforces this in CI. * perf: restore optimistic-repeat-install fast-path for conflict-free state The first iteration of the conflict-detection fix unconditionally read pnpm-lock.yaml on every install - once in installDeps and again inside checkDepsStatus - defeating the point of optimisticRepeatInstall, which was to skip reading the lockfile entirely when nothing changed. Restore the fast path by: - Dropping the redundant lockfile read from installDeps. checkDepsStatus already returns upToDate: false when the lockfile is conflicted, so the pre-check was dead weight. - Gating the conflict check inside checkDepsStatus on the lockfile's mtime: if it hasn't been touched since the last successful install, it cannot have grown conflict markers, so the read is skipped. Conflict markers introduced after a successful install (e.g. via git pull/merge) still update the lockfile mtime, so the correctness fix is preserved. * perf: make lockfile-conflict check synchronous findConflictedLockfileDir awaits its work serially with no concurrent operations to interleave, so the async overhead (Promise.all microtasks, event-loop hops) buys nothing. Convert to a plain for-loop with fs.statSync and a new wantedLockfileHasMergeConflictsSync export. Also resolves a test-mock issue: the previous version called safeStat from deps/status, which is jest-mocked across the test file. The mocked safeStat returned undefined (or stale stats from earlier tests), causing the conflict check to silently no-op. Switching to fs.statSync bypasses the mock and gets the real mtime of the temp lockfile the regression tests write. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Peter Goldberg
|
c2c289094f | fix: time-based resolution loses publishedAt on fast path (#11618) | ||
|
Zoltan Kochan
|
1ad6ffd152 |
feat(config,package-manager): hoistingLimits + externalDependencies knobs (#438 slice 10) (#522)
Plumbs the two programmatic-only hoister knobs from
`pnpm-workspace.yaml` through to the slice 4 walker and the slice 3
hoister. Both fields already existed on `HoistOpts`; this slice wires
them end-to-end.
- `Config::hoisting_limits: BTreeMap<String, BTreeSet<String>>` —
per-importer block-list, locator-keyed (`'.@'` for the root). Reads
`hoistingLimits: { ".@": [foo, bar] }` from yaml. Mirrors upstream's
https://github.com/pnpm/pnpm/blob/94240bc046/installing/linking/real-hoist/src/index.ts#L10
programmatic-only knob, exposed as yaml for parity since the
ergonomics of the locator-keyed map don't translate to a CLI flag.
- `Config::external_dependencies: BTreeSet<String>` — name slots
reserved at the root for an external linker (the Bit CLI is the
only known consumer upstream). Reads `externalDependencies: [...]`
from yaml.
- `LockfileToHoistedDepGraphOptions` gains both fields and forwards
them to `HoistOpts` in `build_dep_graph`.
- `InstallFrozenLockfile::run` clones the two `Config` fields into the
walker opts.
Both knobs default to empty (no limits, no externals), matching
upstream's default. Neither has any effect under `nodeLinker:
isolated` — the isolated linker keeps per-importer subtrees by
construction and doesn't consult the hoister.
Tests:
- `parses_hoisting_limits_from_yaml_and_applies` — yaml round-trip +
apply_to.
- `parses_external_dependencies_from_yaml_and_applies` — same.
- `omitting_hoisting_limits_and_external_dependencies_keeps_defaults`
— pins the apply_to skip-on-None branch so a yaml without these
keys doesn't accidentally overwrite Config defaults.
- `walker_forwards_external_dependencies_to_hoister` — end-to-end:
the walker observes an empty graph for an externalised alias
because the hoister stripped it. Pins the slice 10 plumbing.
|
||
|
Zoltan Kochan
|
6db0430e27 |
feat(real-hoist,package-manager): workspace-aware hoisting (#438 slice 9) (#521)
Enables `nodeLinker: hoisted` for multi-importer (workspace) lockfiles. Previously the hoister rejected any lockfile with importers beyond `.` with `UnsupportedWorkspace`; now the whole workspace shares one hoist plan so conflicting versions across projects dedupe, and each importer's project-tree node_modules is materialized per-project. real-hoist: - Drop the upfront UnsupportedWorkspace guard in `hoist()`. The wrapper already constructed non-root importers as Workspace-kind children of the virtual `.` root; only the guard needed to go. - Add `HoistOpts::hoist_workspace_packages: bool` (default true). When false, non-root importers stay out of the shared tree (matches upstream's `hoistWorkspacePackages: false` mode for the Bit CLI). Removed the corresponding error variant. hoisted_dep_graph walker: - Replace the `workspace:`-prefix skip with a recurse-into branch: for each Workspace-kind hoister child, walk its post-hoist children under `<lockfile_dir>/<importer_id>/node_modules`. The workspace node itself is NOT added to the graph or to the parent's hierarchy — it has no contents to import. - Add per-importer `hierarchy` and `direct_dependencies_by_importer_id` entries. Per-importer direct deps are computed from the lockfile (not from the workspace node's post-hoist children) because hoisted-up siblings don't show up in the workspace node's tree. First-recorded location wins, matching upstream's `pkgLocationsByDepPath[depPath][0]` pick. - Add `LockfileToHoistedDepGraphOptions::hoist_workspace_packages` (default true) and plumb to HoistOpts. Config: - Add `Config::hoist_workspace_packages: bool` (#[default = true]). Read from `pnpm-workspace.yaml` via `WorkspaceSettings`. SymlinkDirectDependencies: - Add `link_only: bool` flag. When true, skip every Regular dep and only materialize Link entries (workspace siblings). Used by the hoisted branch in InstallFrozenLockfile::run so workspace-sibling symlinks land under each importer's `node_modules/<alias>` even though the regular deps now live as real directories from the hoisted linker. InstallFrozenLockfile::run: - Plumb `config.hoist_workspace_packages` into the walker. - After link_hoisted_modules, run SymlinkDirectDependencies with `link_only: true` so workspace siblings get their per-project symlinks. Mirrors upstream's hoisted branch at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L411-L440. Tests: - real-hoist: replace the now-removed `UnsupportedWorkspace` test with one that pins multi-importer hoister output (Workspace children with percent-encoded names + `workspace:<id>` references) and one that pins the `hoist_workspace_packages: false` opt-out. - walker: three new tests cover per-importer direct_deps emission, per-importer hierarchy entries, the `hoist_workspace_packages: false` root-only mode, and version-conflict-across-importers nesting. |
||
|
Zoltan Kochan
|
c2aa928d83 |
test(package-manager): variant-mismatch + --no-runtime install tests (#437 slice F) (#516)
* test(package-manager): variant-mismatch + --no-runtime install tests (#437 slice F) Two end-to-end install tests for the runtime-dependency pipeline, now that #512 (`PkgVerPeer::Prefix::Runtime`) unblocked fixture-lockfile parsing. - `frozen_lockfile_install_errors_when_no_variant_matches_host`: Lockfile with a `VariationsResolution` whose only variant targets `aix/ppc64` (a platform pacquet CI never runs on). Install fails with `NoMatchingPlatformVariant` from the cold-batch dispatcher. Variant selection runs before any network fetch, so the bogus archive URL is never read — the test stays hermetic. Closes the variant-mismatch checkbox. - `frozen_lockfile_install_skips_runtime_when_skip_runtimes_set`: Same lockfile, but with `skip_runtimes: true`. The `--no-runtime` filter iterates importer-direct deps, builds `node@runtime:22.0.0` from `(alias, version)`, sees the `@runtime:` substring, and adds the snapshot to the skip set — so variant selection never runs and the unmatchable variant doesn't fail the install. Asserts both the runtime slot and the direct-dep symlink are absent. Closes the `--no-runtime` checkbox. Both tests were verified to catch their respective regressions (disable the `skip_runtimes` filter → the second test fails with `NoMatchingPlatformVariant` instead of succeeding). Required a fix in `PkgNameVerPeer::without_peer`: the existing implementation dropped the `Prefix::Runtime` when stripping the peer-dependency suffix, so a runtime snapshot key like `node@runtime:22.0.0(some@peer)` would resolve to the non-existent `node@22.0.0` `packages:` entry. Now preserves the prefix through the strip. Slice F's remaining items (happy-path archive fetch, integrity mismatch, NODE_EXTRAS verified end-to-end, TEST_PORTING.md entries) need a mock HTTP server + recorded archive fixture, so defer to a follow-up — keeping this PR reviewable. * test(package-manager): use symlink_metadata for absent-entry checks Address CodeRabbit review on #516: `Path::exists()` follows symlinks, so a regression that created a *dangling* symlink at `<modules_dir>/node` would still pass the `!direct_dep.exists()` assertion — the install would have created the entry the skip set was supposed to suppress, but the test wouldn't notice. Switch both "must NOT exist" assertions to `std::fs::symlink_metadata(...).is_err()` so the entry-level check catches dangling symlinks too. |
||
|
Zoltan Kochan
|
6018359e4c |
feat(package-manager): BuildModules under hoisted node-linker (#438 slice 7) (#520)
Re-enables `BuildModules` for `nodeLinker: hoisted` installs. Slice 6 landed the hoisted install pipeline but skipped the build phase entirely because `BuildModules` walked virtual-store slot directories that don't exist under hoisted. Slice 7 routes the build phase through the slice 4 walker's per-node `dir` so postinstall scripts can run against the on-disk hoisted tree. Changes: - `BuildModules` gains two fields: `pkg_root_by_key: Option<&HashMap<PackageKey, PathBuf>>` overrides the per-snapshot pkg_root lookup with the slice 4 walker's `DependenciesGraphNode::dir` values; `gather_ancestor_bin_paths: bool` switches `extra_bin_paths` to the new `bin_dirs_in_all_parent_dirs` helper, a port of upstream's `binDirsInAllParentDirs` at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L476-L487. - `pkg_root_for_key` helper: routes between the layout-based slot computation (isolated) and the override map (hoisted). Hoisted snapshots absent from the map (walker-dropped) take the same exit as the isolated `!pkg_dir.exists()` skip. - `InstallFrozenLockfile::run` now builds a snapshot-key → first-recorded-dir map from `walker_result.graph.values()` and threads it (plus `gather_ancestor_bin_paths: true`) into `BuildModules` instead of skipping the phase. Multiple graph nodes with the same dep_path collapse to the first entry, matching upstream's `pkgRoots[0]` pick at https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/index.ts#L348. - New private `HoistedLinkerOutput` struct bundles `hoisted_locations` and `pkg_root_by_key` so the hoisted-branch return doesn't trip `clippy::type_complexity`. Side-effects-cache key shape is unchanged — it's keyed by `pkg_id_with_patch_hash` + dep-graph hash, both layout-independent (`crates/graph-hasher/src/dep_state.rs`). `MISSING_HOISTED_LOCATIONS` is intentionally deferred — pacquet has no `rebuild` command, so the install path always re-runs the walker and never reads `.modules.yaml.hoisted_locations`. Tracked as a follow-up for when `pacquet rebuild` lands. Workspace-aware hoisting (slice 9) and `hoistingLimits` / `externalDependencies` (slice 10) remain. Tests: - Three new helper tests pin `bin_dirs_in_all_parent_dirs` against top-level, conflict-nested, and scoped-package shapes. - Three new helper tests pin `pkg_root_for_key` for the isolated pass-through, the hoisted override hit, and the hoisted-missing short-circuit. |
||
|
Zoltan Kochan
|
9844cdf3a9 |
ci: integrate garnet-org/action for supply-chain monitoring (#11626)
Adds the Garnet network-monitoring action to the smoke test job, the release workflow, and the npm tag workflow. The full CI test matrix is left untouched to keep per-job overhead off the broad cross-platform runs; the smoke test still exercises a representative install/test flow. |
||
|
Zoltan Kochan
|
2c8f8ad60c |
feat(package-manager): wire nodeLinker=hoisted into install pipeline (#438 slice 6) (#518)
* feat(package-manager): wire node_linker hoisted into install pipeline (#438 slice 6) Branches `Install::run` / `InstallFrozenLockfile::run` on `config.node_linker == Hoisted`. Threads the slice 4 `lockfile_to_hoisted_dep_graph` walker output and the per-package CAS index produced by `CreateVirtualStore` (slot writes skipped under hoisted) into the slice 5 `link_hoisted_modules` linker. Persists the walker's `hoisted_locations` into `.modules.yaml` for rebuild and follow-up installs to consume. Pipeline changes under hoisted: - `CreateVirtualStore` skips `CreateVirtualDirBySnapshot` for both warm and cold batches, collects each snapshot's CAS file index keyed by `PkgIdWithPatchHash` into a new `cas_paths_by_pkg_id` output field. - `InstallPackageBySnapshot::run` returns the per-package CAS map unconditionally and skips the virtual-store-slot write when its new `node_linker` field is `Hoisted`. - `InstallFrozenLockfile::run` skips `SymlinkDirectDependencies`, `LinkVirtualStoreBins`, the isolated hoist pass, and `BuildModules` under hoisted; runs `lockfile_to_hoisted_dep_graph` + `link_hoisted_modules` in their place. Folds the walker's augmented skip set back into the install-time `SkippedSnapshots` so `.modules.yaml.skipped` reflects the union. - `Install::run`'s `build_modules_manifest` now takes the walker's `hoisted_locations` and writes it through `Modules.hoisted_locations` (only when non-empty so the isolated path doesn't produce a hoisted-only key). The build phase under hoisted (rebuild over `hoistedLocations` with ancestor-`.bin` lookup, `MISSING_HOISTED_LOCATIONS`) is slice 7's scope and is intentionally left as a no-op here. Workspace and `hoistingLimits` / `externalDependencies` knobs are slices 9-10. Mirrors upstream's hoisted branch in `headlessInstall` at https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L369-L425. --- Written by an agent (Claude Code, claude-opus-4-7). * fix(package-manager): docs intra-link disambiguation + skip-set merge fix (#438 slice 6) Docs CI was failing because `[`crate::link_hoisted_modules`]` is ambiguous between the function and the module of the same name. Disambiguate by switching to the function form `[`crate::link_hoisted_modules()`]` everywhere. Slice 6's hoisted-walker skip-set merge previously folded every entry in `walker_result.skipped` into `SkippedSnapshots::insert_installability`, which would promote pre-existing transient skips (`fetch_failed` / `optional_excluded`) into the persisted-on-disk `.modules.yaml.skipped` set. Diff against the input `walker_skipped` so only walker-discovered (genuinely-new) installability skips flow into the persisted subset. |
||
|
Zoltan Kochan
|
62900806fe |
feat(config,cli,tarball): support offline / preferOffline settings (#513)
* feat(config,cli,tarball): support `offline` / `preferOffline` settings Adds the two settings to `Config`, `pnpm-workspace.yaml` parsing, and `pacquet install` as `--offline` / `--prefer-offline` flags. CLI flags merge into Config (boolean OR) before leak, so any source (yaml, CLI) flips `true` and never the other way. Upstream pnpm gates the metadata-fetch path in [`pickPackage`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/npm-resolver/src/pickPackage.ts): when `offline: true` and no cached metadata exists, it fails with `ERR_PNPM_NO_OFFLINE_META`; `preferOffline: true` biases the resolver toward the cached copy. Pacquet's frozen-install path has **no metadata fetch** (the lockfile pins every resolution), so the upstream flag is semantically a no-op on pacquet's current flow. To make the flag actually useful for frozen installs, pacquet adds a **tarball-side gate** (no upstream equivalent — pnpm doesn't gate tarball downloads on `offline`): when both the warm prefetch and the SQLite `index.db` lookup miss, the fetcher refuses the network and errors with `ERR_PACQUET_NO_OFFLINE_TARBALL`. Same shape as `ERR_PNPM_NO_OFFLINE_META`, scoped to tarballs because that's what pacquet's frozen install needs network for. Documented as a pacquet- specific divergence in the field's rustdoc. `prefer_offline` is a no-op today — the warm prefetch + `load_cached_cas_paths` already prefer the local store. The field exists so `.npmrc` / yaml / CLI all parse cleanly; Stage 2's resolver will honour it on the metadata path. - `Config.offline` / `Config.prefer_offline`: bool, default `false`. - `WorkspaceYaml.offline` / `WorkspaceYaml.prefer_offline`: `Option<bool>`; applied via the existing `apply!` macro. - `InstallArgs.offline` / `InstallArgs.prefer_offline`: bool CLI flags; merged into Config in `cli_args.rs` between `config()` and `State::init` (the brief window when `Config::leak` returns `&'static mut Config`). - `DownloadTarballToStore.offline` / `DownloadZipArchiveToStore.offline`: bool fields. Wired through the three construction sites in `package-manager`. - `TarballError::NoOfflineTarball { package_id, url }`: new variant. `tarball_error_to_request_retry` updated for exhaustiveness; `code: ERR_PACQUET_NO_OFFLINE_TARBALL` in the retry-logger projection so a future surface that runs this error through the retry path renders the right code. - `offline_mode_skips_network_on_cache_miss` — `mockito` server with `.expect(0)`; the offline gate must fire before the network is touched. Asserts variant shape AND the diagnostic code (the code is part of the user-facing surface). - `offline_mode_still_uses_prefetched_cache` — same `.expect(0)` guard but with a prefetched cache hit; pins that the prefetch branch short-circuits *before* the offline check so warm installs under `--offline` succeed. Regression-catch verified: gating the gate behind `false &&` flipped `offline_mode_skips_network_on_cache_miss` red with the wrong-variant panic message. Reverted before commit. All bulk-edit fixups (15 `DownloadTarballToStore` / `DownloadZipArchiveToStore` literals in tests + micro-benchmark, 1 `Config` literal in package- manager tests) get `offline: false` to match the prior implicit default. * docs(cli): pin pickPackage link to commit SHA (#513 review) Per project guideline: upstream pnpm citations link to a specific commit SHA (first 10 hex chars), not a branch name. Same SHA the PR's rustdoc and commit body already use for the same file. Resolves CodeRabbit review comment on `crates/cli/README.md`: <https://github.com/pnpm/pacquet/pull/513#discussion_r3238182177>. |
||
|
Zoltan Kochan
|
888184226d |
feat(cli): --node-linker flag (#438 slice 8) (#514)
Add the `--node-linker [isolated|hoisted|pnp]` CLI flag to `pacquet install`, mirroring pnpm's flag. Overrides `Config.node_linker` for the invocation; absent flag → config's yaml/npmrc value wins. The CLI parses into a `NodeLinkerArg` mirror enum (kept in the CLI crate so `pacquet-config` stays free of `clap`), then `into_config()` converts to the canonical `pacquet_config::NodeLinker`. Threaded into the `Install` runner as an explicit `node_linker: NodeLinker` field rather than reading `config.node_linker` directly. Matches the existing pattern `supported_architectures` uses (`state.config` is `&'static`, so the override-merge happens at the CLI layer and lands here as a fully-resolved value). `build_modules_manifest` consumes it on the `.modules.yaml.nodeLinker` write so the persisted setting reflects the invocation, not just the config. `pacquet add` uses Config's value by default (`add` doesn't expose `--node-linker` per the umbrella scope; Slice 6's pipeline integration will revisit if needed). 5 new CLI tests: - `node_linker_default_is_none` — flag absent → field is None. - `node_linker_hoisted` / `node_linker_isolated` / `node_linker_pnp` — each value parses and round-trips through `into_config()`. - `node_linker_invalid_value_rejected` — clap rejects unknown values with the bad value in the error message. - `node_linker_arg_into_config_matches_every_variant` — every ValueEnum variant has a canonical mapping (compile-fails on future variants that forget the mapping). `NodeLinker` gains `Clone + Copy + Eq` so it can be threaded by value into `Install` and matched in tests. |
||
|
Zoltan Kochan
|
5483cc1661 |
feat(lockfile): support runtime: prefix on PkgVerPeer (#511) (#512)
Pnpm v11 writes runtime depPaths with a scheme prefix in front of
the semver — e.g. `node@runtime:22.0.0` in `packages:` /
`snapshots:` keys, and `runtime:22.0.0` in importer `dependencies:`
values. Pacquet's `PkgVerPeer` parser previously accepted only
bare semver and rejected the prefix, so it couldn't load any v11
lockfile containing a runtime dependency.
Extend `PkgVerPeer` to recognise a leading `runtime:` (closed enum
`Prefix::{None, Runtime}`) and preserve it through `Display` /
`serde`. The version + peer parts continue to parse the same way
— the prefix-strip happens once up front, after which the
parenthesis-based peer-suffix detection is unchanged.
Touch points:
- Added `Prefix` enum with `as_str()` and `Display`.
- `PkgVerPeer::prefix()` exposes the variant so downstream
consumers can discriminate runtime entries without
substring-searching the depPath.
- `PkgVerPeer::into_tuple()` keeps the `(Version, String)` shape
for backward compatibility — pre-runtime callers don't see the
prefix.
5 new unit tests pin: bare-semver no-prefix, runtime no-peer,
runtime with peer-suffix, runtime-substring-mid-version is NOT a
prefix (anchored at start), and serde round-trip on the runtime
form. All 100 existing `pacquet-lockfile` tests still pass.
Unblocks #437 slice F (end-to-end install fixtures) — pacquet's
lockfile loader now accepts `node@runtime:X.Y.Z`-shaped keys.
|
||
|
Zoltan Kochan
|
a603a52523 |
feat(network): switch reqwest TLS backend to rustls for PKCS#1 + EC keys (#509)
Replaces the `native-tls-vendored` reqwest feature with `rustls`, closing the PKCS#1 parity gap flagged in #499. Native-tls's `Identity::from_pkcs8_pem` accepted only `-----BEGIN PRIVATE KEY-----` (PKCS#8); rustls's `Identity::from_pem` accepts PKCS#1 (`-----BEGIN RSA PRIVATE KEY-----`), PKCS#8, and EC keys — the same surface Node's `tls.createSecureContext` exposes to pnpm via undici. Pacquet now matches pnpm bug-for-bug on the set of client-cert key formats accepted from `.npmrc`'s `key=` / `:key=` / `:keyfile=` entries. PKCS#12 (`.pfx`) stays out of scope — pnpm's `.npmrc` allow-list doesn't expose a `pfx=` option so pacquet doesn't either. ## What changed - `Cargo.toml`: drop `native-tls-vendored`, add `rustls`. Reqwest's `rustls` feature uses `aws-lc-rs` for crypto and `rustls-platform-verifier` for OS trust roots — closest behavioral match to native-tls's "consult the platform trust store" default. - `crates/network/src/lib.rs`: `apply_tls` swaps `Identity::from_pkcs8_pem(cert, key)` for `Identity::from_pem` applied to the concatenated `cert\nkey` PEM buffer. Adds a new `looks_like_pem_cert` syntactic check before `Certificate::from_pem` because rustls's `from_pem` stores the bytes verbatim and validates lazily at `Client::build()` time — a garbage CA entry would otherwise slip through silently and the install would proceed against an unknown trust root. - Updated doc comments on `apply_tls`, `TlsConfig::key`, `RegistryTls::key`, and `TlsError::InvalidClientIdentity` to describe the new surface and drop the PKCS#8-only caveat. - `deny.toml`: allow `CDLA-Permissive-2.0` for `webpki-root-certs` (pulled in by reqwest's `rustls` feature through `rustls-platform-verifier`'s fallback chain). - New fixtures at `crates/network/tests/fixtures/test-client-pkcs1.{crt,key}` loaded via `include_str!`. Regenerable with `openssl genrsa -traditional` + `openssl req -new -x509`. ## Tests `for_installs_with_pkcs1_client_key_builds` pins the contract — if a future change reverts the backend or otherwise narrows the accepted key formats, this build will fail with a clear `InvalidClientIdentity`. All 1175 workspace tests pass. ## Notes for review - **Cert store change.** `rustls-platform-verifier` reads the OS trust store on macOS / Windows / Linux. The lookup is a different syscall path from native-tls's; behavioral parity for "trust roots in the OS store" should hold, but corporate CAs that worked under native-tls and *don't* show up in `rustls-platform-verifier`'s enumeration would now silently fail. Users hitting that should add the CA explicitly via `cafile=` in `.npmrc`. - **Performance.** CI's integrated-benchmark will run on this PR; if it regresses materially on the warm-install path we'd consider falling back to the preprocessing approach (option 2 in #499). - **`hickory-dns` compatibility.** Verified by running the workspace test suite — DNS resolution is independent of the TLS backend. - **`cargo deny` posture.** One new license allowance (`CDLA-Permissive-2.0`) for `webpki-root-certs`. No new advisory surface area beyond what reqwest's `rustls` feature already pulls through `aws-lc-rs` / `rustls` / `rustls-platform-verifier`. |
||
|
Zoltan Kochan
|
38e2954f81 |
feat: ignoredOptionalDependencies config + lockfile field + outdated-setting check (#434 slice 7) (#507)
* feat: ignoredOptionalDependencies config + lockfile field + outdated-setting check (#434 slice 7) Last slice of the optional-dependencies umbrella (#434). Adds the user-facing `ignoredOptionalDependencies` setting: a list of dep-name patterns the user wants entirely excluded from resolution + install. Mirrors pnpm/pnpm@94240bc046's three surfaces: - **Hook**: `hooks/read-package-hook/src/createOptionalDependenciesRemover.ts` builds a matcher and drops matching keys from `optionalDependencies` AND `dependencies` (a package may list the same dep under both for "optional only when consumed"). - **Lockfile field**: `lockfile/types/src/index.ts:19` — `ignoredOptionalDependencies?: string[]` at the top level (sibling of `lockfileVersion`/`overrides`, NOT inside `settings`). - **Drift check**: `lockfile/settings-checker/src/getOutdatedLockfileSetting.ts:58-60` sorts both arrays and compares; mismatch triggers `needsFullResolution`. In pacquet's frozen-only flow this surfaces as `OutdatedLockfile`. ## Changes - **`pacquet-config`**: `Config::ignored_optional_dependencies: Option<Vec<String>>` + `WorkspaceSettings` field + `apply_to` wiring. - **`pacquet-lockfile`**: `Lockfile::ignored_optional_dependencies: Option<Vec<String>>` top-level field with serde round-trip; `check_lockfile_settings(lockfile, config_set)` sorts-and- compares; new `StalenessReason::IgnoredOptionalDependenciesChanged { lockfile, config }` variant. - **`pacquet-lockfile::satisfies_package_manifest`** extended with an `is_ignored_optional: &dyn Fn(&str) -> bool` parameter. Skips matching names in `flat_manifest_specs` and the per-field check so a manifest that still lists ignored entries doesn't falsely surface as drift against a lockfile the resolver correctly built without them. - **`pacquet-package-manager::install.rs`**: builds a matcher from `Config::ignored_optional_dependencies` (reuses `pacquet_config::matcher::create_matcher` — same glob engine as `hoistPattern`); calls `check_lockfile_settings` before `satisfies_package_manifest`; threads the matcher closure into the freshness check. - **`pacquet-package-manager::current_lockfile`**: preserves the lockfile's `ignored_optional_dependencies` through the slice 6 filter so the recorded set round-trips to the current lockfile. ## Tests - `pacquet_config`: yaml-parse + `apply_to` round-trip + omission baseline. - `pacquet_lockfile::freshness::check_settings_*`: both-sides- empty, sorted-match-regardless-of-order, drift in both directions. Test-the-test verified by removing the sort+compare guard. - `pacquet_lockfile::freshness::ignored_optional_filtered_*`: manifest-side filter passes when the matcher fires; polarity test confirms the unfiltered case surfaces as `SpecifiersDiffer`. - `pacquet_lockfile::freshness::ignored_optional_dependencies_round_trips_through_yaml`: serde wire-shape round-trip. Closes #503. Closes the #434 umbrella. * fix(lockfile): scope ignoredOptionalDependencies filter to prod+optional + set-based predicate CodeRabbit review on PR #507 (major): the filter wrongly applied to `devDependencies` too. Upstream's `hooks/read-package-hook/src/createOptionalDependenciesRemover.ts` iterates `manifest.optionalDependencies` and deletes matches from `optionalDependencies` AND `dependencies` only — `devDependencies` is untouched. The previous impl applied the filter to all three groups in both `flat_manifest_specs` and the per-field check, so a stale lockfile could incorrectly pass when the manifest added or removed a matching `devDependency`. Two fixes: 1. **Group gate** in `flat_manifest_specs` and the per-field check: apply the filter only when the group is `Prod` or `Optional`. `Dev` walks ignore the closure. 2. **Set-based predicate** at the call site (`Install::run`): build the "to drop" set from `manifest.optionalDependencies ∩ pattern`, not just from the pattern. A name listed only in `dependencies` (not `optionalDependencies`) that happens to match the pattern is NOT removed by upstream's hook (the hook never iterates that name). The set-based predicate captures that nuance. Both fixes together mirror the hook's exact semantics. Two new regression tests pin the dev-dependency behavior: `ignored_optional_does_not_apply_to_dev_dependencies` and `ignored_optional_dev_only_lockfile_entry_kept`. Test-the-test verified by dropping the group gate inside `flat_manifest_specs` — both tests fail. |
||
|
Zoltan Kochan
|
c7ab26a056 |
fix(package-manager): link_hoisted_modules PkgIdWithPatchHash types (#508)
The Slice 5 linker (#505) was written against `String` for `pkg_id_with_patch_hash` and merged onto a main that had already switched the underlying `DependenciesGraphNode` field to the `PkgIdWithPatchHash` newtype (#504). The merge auto-resolved without flagging the type incompatibility — `cargo check` on main now errors with: error[E0277]: the trait bound `String: Borrow<PkgIdWithPatchHash>` is not satisfied error[E0308]: mismatched types expected `String`, found `PkgIdWithPatchHash` Switch the two slice-5-introduced surfaces to match the newtype: - `CasPathsByPkgId = HashMap<PkgIdWithPatchHash, HashMap<String, PathBuf>>` — the per-package CAS index now keys on the brand the graph node carries, matching the post-#504 invariant across the workspace. - `LinkHoistedModulesError::MissingCasPaths.pkg_id_with_patch_hash: PkgIdWithPatchHash` — same type the graph node has, so the error round-trips the value without `.to_string()`. Tests updated to construct `PkgIdWithPatchHash::from(...)` keys/fields. All 8 linker tests pass on the fixed branch. This unblocks main building again. |
||
|
Zoltan Kochan
|
13a6970ae6 |
feat(package-manager): runtime bin-link + --no-runtime (#437 slice D3 + E) (#506)
* feat(package-manager): runtime bin-link + --no-runtime (#437 slice D3 + E) Closes the install-pipeline side of #437: **Slice D3 — runtime bin-link integration.** Runtime archives (`node@runtime:`, etc.) don't ship a `package.json`, so the existing bin-link step had nothing to consume off the slot. `fetch_binary_resolution_to_cas` now synthesizes one in the same shape upstream's `appendManifest` does: { "name": "<pkg.name>", "version": "<pkg.version>", "bin": <BinarySpec> } The bytes go through `StoreDir::write_cas_file` (same content- addressing as every other CAS-imported file), the resulting cas-path is inserted into the snapshot's `cas_paths` under `package.json`, and the existing `link_bins_of_packages` flow picks it up. `link_bins::build_has_bin_set` now includes `Binary` / `Variations` resolutions unconditionally — pnpm v11 doesn't emit `hasBin: true` on runtime metadata, but the synthesized manifest always carries bins, so the lookup must not be gated on the metadata flag. **Slice E — `--no-runtime` flag.** New `Config::skip_runtimes` (default false) and `InstallArgs::no_runtime` CLI flag. The CLI computes `config.skip_runtimes || --no-runtime` and threads it through `Install` → `InstallFrozenLockfile`. When set, every snapshot whose metadata resolution is `Binary` or `Variations` is added to the install-time skip set (re-using `add_optional_excluded` since the bucket count and `.modules.yaml.skipped` semantics line up with `--no-optional`). **Unit tests.** - `synthesize_runtime_manifest_emits_name_version_and_bin_single` — pins `BinarySpec::Single` → JSON string. - `synthesize_runtime_manifest_emits_name_version_and_bin_map` — pins `BinarySpec::Map` → JSON object with all bin entries. - `synthesize_runtime_manifest_preserves_scoped_name` — `@scope/name` round-trips through the `name` field. - `build_has_bin_set_includes_runtime_resolutions_even_when_has_bin_is_absent` — pins the runtime arm. Verified by removing the arm — test fails as expected. End-to-end install fixtures (slice F's remaining items — recorded-archive frozen-lockfile install, variant-mismatch error, `--no-runtime` integration, integrity-mismatch path) need a small recorded Node archive in the repo (or a mockable `pnpm-resolution-mirror`). Tracking them as a separate follow-up so this PR stays reviewable. * fix(package-manager): narrow --no-runtime to importer-direct deps Address CodeRabbit review on #506: my initial implementation iterated `packages` (every metadata entry) and added all `Binary` / `Variations` snapshots to the skip set, which widened the behavior beyond pnpm's `skipRuntimes`. Per the cardinal rule (CLAUDE.md: "Port behavior faithfully. ... Do not invent behavior that pnpm does not have."), match upstream exactly. Upstream's filter at [`installing/deps-installer/src/install/index.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/index.ts#L1374-L1387) iterates `dependenciesByProjectId` (per-importer direct deps) and adds `depPath` to `ctx.skipped` when `depPath.includes('@runtime:')`. Transitive runtime entries — unusual but possible — stay in the install. Pacquet now mirrors that exactly: walk each importer's `dependencies` / `dev_dependencies` / `optional_dependencies`, build the candidate snapshot key from `(alias, version)`, check for the `@runtime:` substring, and only add the metadata-confirmed `Binary` / `Variations` keys to `skipped`. Aliased runtime deps (unusual) fall through the same way upstream does — pnpm's lookup is by depPath, not by alias. |
||
|
Zoltan Kochan
|
be759bf13f |
feat(package-manager): linkHoistedModules linker (#438 slice 5) (#505)
* feat(package-manager): linkHoistedModules linker (#438 slice 5) New module `crates/package-manager/src/link_hoisted_modules.rs` produces the on-disk `node_modules/` tree from Slice 4's `LockfileToDepGraphResult`. Ports installing/deps-restorer/src/linkHoistedModules.ts. ## Three phases 1. **Orphan removal.** Every directory in `prev_graph` but not in `graph` is silently `rimraf`'d before any insert. `try_remove_dir` swallows all errors (NotFound + PermissionDenied + EBUSY), matching upstream's `tryRemoveDir` tolerance. 2. **Per-node import.** Hierarchy walked top-down, rayon-parallel at each level. Every node goes through `import_indexed_dir` with `force: true, keep_modules_dir: true` — the hoisted-linker call shape Slice 1 was designed for. 3. **Per-`node_modules` bin link.** After a level's children are all imported, `link_direct_dep_bins` populates `<parent>/node_modules/.bin` from the just-materialized direct children. Bin link runs only after every child's subtree is done so package.json reads always see the fully-populated package. ## API shape ```rust pub fn link_hoisted_modules<R: Reporter>( opts: &LinkHoistedModulesOpts<'_>, ) -> Result<(), LinkHoistedModulesError>; ``` `LinkHoistedModulesOpts` borrows `graph`, `prev_graph`, `hierarchy`, and a `cas_paths_by_pkg_id: HashMap<String, HashMap<String, PathBuf>>` keyed by `pkg_id_with_patch_hash`. ## Decoupling from the store Upstream's linker is async and calls `storeController.fetchPackage()` inline during the walk — pacquet's is sync and takes pre-fetched CAS paths. Slice 6 (the install pipeline) is responsible for fetching every package through pacquet's existing tarball / store-dir / git-fetcher machinery before invoking the linker. This keeps the linker focused on file-system layout and reuses the proven fetch chain that the isolated path already exercises. ## Optional-dep tolerance A graph node whose `pkg_id_with_patch_hash` is missing from `cas_paths_by_pkg_id`: - If `node.optional`: silently skipped, no directory created. Mirrors upstream's `if (depNode.optional) return` on fetch failure. - Otherwise: surfaces as `LinkHoistedModulesError::MissingCasPaths { pkg_id, dir }`. ## Tests 7 real-tempdir tests in `link_hoisted_modules/tests.rs`: - `import_pass_creates_package_directory` — single-package smoke. - `orphan_directory_is_removed` — `prev_graph` diff produces rimraf of stale directory; planted contents are gone after. - `nested_hierarchy_materializes_inner_node_modules` — version-conflict layout; loser ends up at `<outer>/node_modules/<inner>`. - `missing_cas_for_required_dep_errors` — required + missing CAS → typed `MissingCasPaths` error. - `missing_cas_for_optional_dep_skips_silently` — optional + missing CAS → no error, no directory. - `no_prev_graph_skips_orphan_pass` — fresh install (no prior lockfile) path. - `orphan_already_removed_is_tolerated` — phantom orphan in `prev_graph` not present on disk doesn't error. Each test plants synthetic CAS files in a tempdir and asserts the on-disk tree after the linker runs. * fix(package-manager): fail-fast on hierarchy/graph inconsistency (#505 review) The hierarchy walk silently skipped entries missing from `graph`, which would produce a partial install layout instead of surfacing the bug. Slice 4's walker keeps the two in sync today, but a future bug there shouldn't yield a partial tree. Add `LinkHoistedModulesError::MissingGraphNode { dir }` and return it when a hierarchy directory has no graph entry. Upstream effectively errors here too — its `graph[dir].fetching` read would throw `Cannot read properties of undefined` — pacquet just spells the failure out. Regression test `hierarchy_entry_missing_from_graph_errors` exercises the path with an empty graph and a hierarchy referencing a phantom dir. Caught by Coderabbit. |
||
|
Zoltan Kochan
|
3b9f5977c4 |
feat(lockfile): PkgIdWithPatchHash newtype, replace String in dep-graph nodes (#481) (#504)
* feat(lockfile): PkgIdWithPatchHash newtype, replace `String` in dep-graph nodes (#481) Port upstream's [`PkgIdWithPatchHash`](https://github.com/pnpm/pnpm/blob/94240bc046/core/types/src/misc.ts) branded string (`string & { __brand: 'PkgIdWithPatchHash' }`) as a non-validating newtype in `pacquet-lockfile`, matching `CLAUDE.md`'s rule 3 for non-validating brands: `From<String>` / `From<&str>` via `derive_more`, `#[serde(transparent)]` for wire-format identity with `String`, no validating constructor. Modeled on `pacquet_modules_yaml::DepPath` — the sibling brand from the same upstream `misc.ts` file under the same rules. Replaces the plain `String` field/parameter in the two pacquet consumers CodeRabbit flagged on #478: - `DependenciesGraphNode.pkg_id_with_patch_hash` in `package-manager/src/hoisted_dep_graph.rs`. - `create_full_pkg_id`'s first parameter and the `lockfile_to_dep_graph` local in `package-manager/src/virtual_store_layout.rs`. A workspace audit (`grep "pkg_id_with_patch_hash"`) turns up no other slots typed as plain `String`. Two new tests in `pkg_id_with_patch_hash.rs` pin the contract: a serde round-trip (`#[serde(transparent)]` keeps the on-disk shape a raw string) and the non-validating-construction property (empty string and `From<&str>`/`From<String>` both work). * fix(lockfile): replace intra-doc links to pacquet-modules-yaml with plain text CI \`Doc\` job runs with \`RUSTDOCFLAGS=-D warnings\`, so the two \`[\`pacquet_modules_yaml::DepPath\`]\` intra-doc links in \`pkg_id_with_patch_hash.rs\` failed \`rustdoc::broken-intra-doc-links\`: \`pacquet-lockfile\` doesn't depend on \`pacquet-modules-yaml\` (the dependency would go the wrong way), so the rustdoc resolver can't follow the link. Switched both references to bare \`pacquet_modules_yaml::DepPath\` prose, with a note that the bare reference is intentional. The docstring still tells a reader where to find the sibling brand; it just stops resolving as a clickable link, which costs nothing practical since the type name is searchable in any IDE. Adding \`pacquet-modules-yaml\` as a dev-dep purely for a doc link would invert the natural crate ordering (lockfile is upstream of modules-yaml in the build graph) — rejected as a cosmetic fix that introduces a real architectural smell. |
||
|
Zoltan Kochan
|
52364d4b5a |
feat(config,network,tarball,registry): per-registry TLS overrides (#502)
Closes #497.
## Summary
Adds per-registry TLS overrides keyed by nerf-darted `.npmrc` URI, the natural follow-up to #490's top-level TLS keys. Corporate environments running a private Verdaccio (or any registry with its own self-signed cert) can now pin scoped `:cafile=…` / `:cert=…` / `:key=…` per host without disabling strict-ssl globally.
Three commits, layered:
- **`feat(network)`** (eff1248e): adds `RegistryTls` + `PerRegistryTls` types in `pacquet-network` plus the lookup machinery — `pick_for_url` ports pnpm's [5-step `pickSettingByUrl`](https://github.com/pnpm/pnpm/blob/94240bc046/network/fetch/src/dispatcher.ts#L338-L375) exactly (exact > nerf-dart > no-port > shorter prefix > recursive no-port retry). `ThrottledClient::for_installs` gains a third `&PerRegistryTls` parameter and pre-builds one reqwest `Client` per non-empty override. New `acquire_for_url(url: &str)` routes per-request; `acquire()` keeps the default-client behavior for callers without a URL.
- **`feat(config)`** (4e69868a): `NpmrcAuth` parses the six per-registry TLS suffixes (`:ca`, `:cafile`, `:cert`, `:certfile`, `:key`, `:keyfile`) matching pnpm's `SSL_SUFFIX_RE` and applies onto `Config.tls_by_uri`. `*file` variants read from disk at parse time (silent on error); inline values get `\\n` → `\n` expansion. `:cert` and `:certfile` share the same `tls.cert` slot — last-write-wins inside one `.npmrc`.
- **`refactor(tarball,registry)`** (5f9cae93): three production call sites (registry metadata + version-tag fetches, plus two tarball download paths) move from `acquire()` to `acquire_for_url(url)` so the per-registry routing actually fires.
## Parity policy
Bug-for-bug with pnpm v11 ([SHA
|
||
|
Zoltan Kochan
|
13f401ab73 |
feat(package-manager): NODE_EXTRAS ignore filter for runtime archives (#437 slice D2) (#496)
Construct the per-archive ignore filter at the install dispatcher. For unscoped `node` the filter matches upstream's [`NODE_EXTRAS_IGNORE_PATTERN`](https://github.com/pnpm/pnpm/blob/94240bc046/engine/runtime/node-resolver/src/index.ts), which strips bundled `npm` / `corepack` from the Node.js runtime archive during the CAS write — pnpm (and pacquet) install pnpm itself as the package manager, so the bundled tooling is dead weight and would shadow the user's pnpm via `node_modules/.bin/`. Wiring matches upstream's [`archiveFilters: { node: NODE_EXTRAS_IGNORE_PATTERN }`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/client/src/index.ts): the per-package-name table is keyed by `pkg.name`, so `@foo/node` and other packages keep `None` and the full archive lands unfiltered. Pacquet uses a hand-coded matcher rather than the upstream regex so `pacquet-tarball` doesn't have to pull in a regex engine. The three branches mirror the regex alternation exactly: 1. `^(?:lib/)?node_modules/(?:npm|corepack)(?:/|$)` 2. `^bin/(?:npm|npx|corepack)$` 3. `^(?:npm|npx|corepack)(?:\.(?:cmd|ps1))?$` A `OnceLock` caches the `Arc<IgnoreEntryFilter>` so per-snapshot clones share one trait object. Unit tests pin every branch of the alternation plus the negative cases (e.g. `lib/node_modules/yarn/...` is *not* matched, `bin/npm.cmd` is *not* matched — the regex's `$` and arm-specific extension rules are deliberately asymmetric). Verified by temporarily collapsing the `bin/` branch to a no-op — the test fails as expected. Bin-link cmd-shims for the runtime executables and `@runtime:` substring handling in skip lists / reporter prefixes are Slice D3. End-to-end runtime install fixtures land in Slice F. |
||
|
Zoltan Kochan
|
9c67bc82c7 |
feat(package-manager): prev_graph diff from current lockfile (#438 slice 4d) (#494)
* feat(package-manager): prev_graph diff from current lockfile (#438 slice 4d) `lockfile_to_hoisted_dep_graph` now takes an optional `current_lockfile: Option<&Lockfile>` and populates `result.prev_graph` from a second walk over that lockfile. Ports upstream's wrapper at installing/deps-restorer/src/lockfileToHoistedDepGraph.ts. When the current lockfile is `Some(lf)` with a non-empty `packages:` map, the second walk runs with `force: true` and `skipped: BTreeSet::new()`. Force matters: an orphan that landed under the previous install but would now fail installability (e.g., the host changed platforms) still surfaces in `prev_graph` so Slice 5's linker can find and rimraf the stale directory. Empty skipped matters: the previous install's own filter set is unrelated to "which directories still exist on disk." API change - The public `lockfile_to_hoisted_dep_graph(lockfile, opts)` signature gains a middle `current_lockfile: Option<&Lockfile>` argument. Single existing caller (the tests) updated to pass `None`. Splits the previous body into a private `build_dep_graph` helper that the wrapper calls once or twice depending on whether a current lockfile is present. prev_graph shape - `None` when no current lockfile is supplied, or when the supplied lockfile has no `packages:` map (a brand-new install in progress). Mirrors upstream's `prevGraph = {}` fallback — pacquet uses `None` rather than an empty map, but the linker treats both the same. - `Some(graph)` otherwise, keyed by directory just like the wanted-lockfile graph. Tests - `prev_graph_none_when_current_lockfile_absent` — no current lockfile → `prev_graph` is `None`, wanted graph still produced. - `prev_graph_none_when_current_lockfile_has_no_packages` — current lockfile with `packages: None` → `prev_graph` is `None`. - `prev_graph_contains_orphan_from_current_only_lockfile` — package in current but not wanted appears in `prev_graph`, not in `graph`. - `prev_graph_includes_orphan_even_when_now_incompatible` — darwin-only orphan in the current lockfile, host is linux, wanted lockfile is empty: `prev_graph` still contains it (proves `force: true` overrides the installability check), and the wanted-walk's `skipped` stays empty (proves the prev-walk's skipped set is independent). All 4 new tests pass alongside the 15 walker tests from 4a-4c. * fix(package-manager): collapse empty current packages to None for prev_graph (#494 review) `current.packages.is_some()` matched `Some(empty_map)` too, causing 4d to do an unnecessary second walk and return `Some(empty_graph)` for a case the doc-comment described as `None`. Tighten the guard to require a non-empty `packages` map. Pacquet uses `Option<DependenciesGraph>` for `prev_graph` (upstream uses an always-present `DependenciesGraph` with `{}` for the no-current case). The no-packages and empty-packages cases both produce the same observable behavior — "no orphans to consider" — so they should share one representation rather than be inconsistent. The doc-comment's stated contract was `None`; the code now matches. Added `prev_graph_none_when_current_lockfile_has_empty_packages` to pin the empty-map case. Caught by Coderabbit on #494. * style(package-manager): rustfmt the 4d follow-up |
||
|
Zoltan Kochan
|
3fe272f050 |
feat: filter the wanted lockfile when writing <virtual_store_dir>/lock.yaml (#434 slice 6) (#495)
Pacquet wrote the raw wanted lockfile as the current lockfile. Upstream pnpm writes a filtered version with optional + skipped subtrees pruned and `include` flags applied, so the next install diffs against what was actually materialized rather than the resolver's full ambition. Without the prune, dropped snapshots (slice 1 installability, slice 4 fetch failures, slice 5 `--no-optional`) were claimed present in the current lockfile and the follow-up install would skip work that should have run. Ports <https://github.com/pnpm/pnpm/blob/94240bc046/lockfile/filtering/src/filterLockfileByImportersAndEngine.ts#L46-L94> → <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L687-L695>. Pacquet-specific simplification: instead of re-running the engine + supportedArchitectures + skipped checks at filter time, `filter_lockfile_for_current` reuses the `SkippedSnapshots` set already produced by the install pipeline. Its three subsets (`installability`, `fetch_failed`, `optional_excluded`) are the exact set upstream's filter would drop — same observable result, no duplicated walk. ## Changes - **`pacquet-lockfile`**: `Clone` derives on `Lockfile`, `LockfileSettings`, `ProjectSnapshot`, `SnapshotEntry`, `PackageMetadata`, `PeerDependencyMeta`, `ResolvedDependencySpec`. Needed to build a filtered lockfile by clone-and-mutate. - **`pacquet-package-manager/src/current_lockfile.rs`** (new): `filter_lockfile_for_current(lockfile, included, skipped) -> Lockfile`. Three-phase filter: 1. BFS the snapshot graph from filtered importer roots, skipping keys in `skipped`. Produces the reachable set. 2. Per-importer: clear dep maps whose `include` flag is false; trim `optional_dependencies` to entries whose target survived. 3. `snapshots:` / `packages:` filtered to the reachable set (packages key off `without_peer()` so peer-variant survivors keep their shared metadata row). - **`install.rs`**: replace the raw `save_current_to_virtual_store_dir` call with `filter_lockfile_for_current(...).save_current_to_virtual_store_dir(...)`. ## Tests 8 unit tests covering each filter behavior: - `skipped_snapshot_pruned_from_snapshots_and_importer_optional` - `include_optional_false_clears_importer_section` - `transitive_under_skipped_snapshot_is_pruned` - `snapshot_reachable_via_kept_path_survives` (mirrors upstream's `:712` shared-dep case at the filter level) - `packages_filtered_to_surviving_metadata_keys` (peer-variant metadata sharing) - `link_optional_entries_survive_post_filter` (workspace link: deps don't get post-filtered) - `empty_skipped_and_full_include_is_identity_for_reachables` (baseline) - `orphan_snapshots_are_pruned` (snapshots unreachable from any importer get dropped) Test-the-test verified by breaking the BFS walker — two tests fail. ## Out of scope - Hoisted-linker current-lockfile variant (`:633`) — pacquet's hoisted node-linker isn't fully wired through the lockfile-write path yet; tracked separately under #438. - `pnpm install --filter` slicing — pacquet has no `--filter` yet. Closes #493. |
||
|
Zoltan Kochan
|
d2cdb8de65 |
test(git-fetcher): end-to-end shallow-fetch argv assertion via PATH-shim (#436 §E follow-up) (#487)
Ports the last open §E test — upstream's [`fetching/git-fetcher/test/index.ts:183`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/test/index.ts#L183) `still able to shallow fetch for allowed hosts`, which jest-mocks `execa` to spy on the `git` argv. Pacquet can't intercept `Command::new("git")` without touching production code, so the test uses a `/bin/sh` shim on `PATH`: 1. Writes `<tempdir>/shim/git`: a tiny shell script that tab-logs each invocation to a file and fakes `rev-parse HEAD` so the fetcher's commit-match check passes. 2. Prepends the shim dir to `PATH` for the test body via `unsafe { std::env::set_var(...) }`. Edition 2024 requires `unsafe`; the project's `cargo nextest` runner isolates each test in its own process so no sibling can race the modified env. PATH is restored before assertions. 3. Parses the log and asserts the shallow sequence: `git init` → `git remote add origin <url>` → `git fetch --depth 1 origin <commit>`, with `git clone` absent. `fetcher_clones_when_host_not_in_shallow_list` is the mirror — same shim, empty `git_shallow_hosts`, asserts `git clone <url> <dir>` appears and `init` / `fetch` don't. Together the two tests pin the gate's polarity end-to-end. The existing `should_use_shallow_matches_known_host` unit test only covers the predicate cross-platform; the new tests add Unix end-to-end argv coverage. Unix-only via `#[cfg(unix)]`. A Windows mirror would need a `.cmd` shim and a different launcher; the predicate-level test still covers Windows. |
||
|
Zoltan Kochan
|
6d9d0ad08c |
feat(config,network,cli): TLS keys + local-address from .npmrc (#490)
Closes #482. ## Summary Ports pnpm v11's TLS + `local-address` `.npmrc` keys onto pacquet (SHA [`94240bc046`](https://github.com/pnpm/pnpm/blob/94240bc046/network/fetch/src/dispatcher.ts)), the natural pair to the proxy support that landed in #476. Three layers: - **`feat(network)`** (e9ed56c9): adds `TlsConfig` next to `ProxyConfig` in `pacquet-network`, with `ca: Vec<String>`, `cert`/`key`, `strict_ssl: Option<bool>`, and `local_address: Option<IpAddr>`. `ThrottledClient::for_installs` gains a `&TlsConfig` parameter; the unified error surface is the new `ForInstallsError` enum carrying either `ProxyError` or `TlsError`. CAs route through `Certificate::from_pem`, client identities through `Identity::from_pkcs8_pem` (the only PKCS path reqwest exposes on the native-tls backend pacquet builds with). `strict_ssl` defaults to `true` at build site, matching pnpm's per-emit-site `strictSsl ?? true` rather than a config-layer default. - **`feat(config)`** (e8dcd87e): extends `NpmrcAuth` with the six new keys (`ca`, `cafile`, `cert`, `key`, `strict-ssl`, `local-address`) and adds `apply_tls_and_local_address` to populate `Config.tls`. `cafile` reads from disk and splits on `-----END CERTIFICATE-----` to mirror pnpm's [`loadCAFile`](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/loadNpmrcFiles.ts#L249-L265). Unreadable `cafile` is silently treated as unset. Invalid `strict-ssl` and `local-address` values drop silently. - **`feat(cli)`** (1759f9d2): one-line swap in `State::init` to pass `&config.tls` instead of `TlsConfig::default()`. Folds in two CI-parity fixups: the self-signed test cert moves to a shared `.pem` fixture under `crates/network/tests/fixtures/` (typos linter false positives on base64 DER), and four reqwest intra-doc links become plain backticks. ## Parity policy Faithful to pnpm — see the [research brief](https://github.com/pnpm/pacquet/issues/482) on the issue. Highlights: - **No new error codes.** pnpm doesn't define `ERR_PNPM_INVALID_CA` etc.; invalid PEMs surface as raw `tls.connect` errors at request time upstream. Pacquet validates eagerly via `Certificate::from_pem` / `Identity::from_pkcs8_pem` (pushing the failure to per-request time would silently degrade every install behind a broken `ca`) but deliberately omits a `code(...)` attribute on `TlsError` so reviewers can see at a glance it's a pacquet-only diagnostic, not a pnpm error code. - **Silent `cafile`-not-found.** Matches pnpm's `catch {}` swallow in `loadCAFile`. - **No env-var fallback.** pnpm reads only `.npmrc`; Node's implicit `NODE_EXTRA_CA_CERTS` / `NODE_TLS_REJECT_UNAUTHORIZED` honoring doesn't apply to pacquet's reqwest stack. - **`strict-ssl: false` disables both chain-of-trust and hostname verification**, matching Node's `rejectUnauthorized=false` short-circuit (pacquet uses reqwest's `danger_accept_invalid_certs(true)` which has the same combined semantics). ## Reviewer flags - **PKCS#8-only client keys.** Reqwest's native-tls backend exposes only `Identity::from_pkcs8_pem`; legacy PKCS#1 keys (`-----BEGIN RSA PRIVATE KEY-----`) and PKCS#12 bundles are not supported by this constructor. Documented at the `apply_tls` callsite with the `openssl pkcs8 -topk8 -nocrypt` conversion command. Switching to rustls-tls would broaden the supported formats but is out of scope here. - **Per-registry TLS overrides** (`//host:cafile=`, `//host:ca=`, `//host:cert=`, `//host:key=`) are **not** included. Same shape as the existing scoped-auth handling but a sizeable feature on its own; flagged in #482 as a follow-up. |
||
|
Zoltan Kochan
|
a55957d0c0 |
feat(package-manager): dispatch Binary/Variations to runtime fetchers (#437 slice D1) (#492)
Replace the cold-path `UnsupportedResolution` arms for
`LockfileResolution::{Binary, Variations}` with actual fetcher
dispatch. Slice C wired the per-archive download
(`DownloadTarballToStore` / `DownloadZipArchiveToStore`); this
slice routes lockfile resolutions through them.
- `Binary(b)` dispatches on `b.archive` (`Tarball` / `Zip`):
- `Tarball` → `DownloadTarballToStore` (same shape as
registry / tarball entries minus the `package_unpacked_size`
hint).
- `Zip` → `DownloadZipArchiveToStore` with
`archive_prefix: b.prefix.as_deref()` so the runtime
archive's top-level wrapper (e.g. `node-vX.Y.Z-darwin-arm64/`)
is stripped before the CAS keys are written.
- `Variations(v)` runs `select_platform_variant` against a
freshly-built `PlatformSelector` (`pacquet_graph_hasher`'s
`host_platform` / `host_arch` / `host_libc`, mapping
`"unknown"` → `None` per upstream's
`process.platform === 'linux' ? family : null` convention).
The picked variant's inner resolution must be `Binary`; any
other shape (corrupt lockfile or a future shape pacquet hasn't
learned about) raises the typed
`VariantHasNonBinaryResolution` error rather than silently
routing.
- `create_virtual_store::snapshot_cache_key` returns the proper
warm-cache key for both new arms: `Binary` uses
`store_index_key(integrity, pkg_id)`; `Variations` runs the
same selector and keys off the picked variant. A miss on
variant selection or a non-`Binary` inner shape returns
`Ok(None)` and lets the cold path raise the typed error.
- Two new error variants on `InstallPackageBySnapshotError`:
- `NoMatchingPlatformVariant { package_key, host_os, host_cpu,
host_libc, available_targets }` — fires when no variant
matches the host; carries the host triple + the rendered
available target list so the user can see at a glance why.
- `VariantHasNonBinaryResolution { package_key, inner_kind }`
— defensive guard for malformed lockfiles.
The Node-runtime `NODE_EXTRAS_IGNORE_PATTERN` filter (strips
bundled `npm` / `corepack` from the archive) and bin-link
cmd-shims for runtime executables will land in Slice D2; the
filter slot stays `None` for now and `BinaryResolution::bin` is
read but not yet acted on. Slice E adds `--no-runtime` and
Slice F adds the end-to-end install fixtures.
Two new unit tests cover the helpers:
- `host_platform_selector_omits_libc_on_non_linux_hosts` —
pins the `host_libc() == "unknown"` ⇔ `selector.libc.is_none()`
relationship without needing platform-gated tests.
- `render_variant_targets_formats_each_triple_with_optional_libc`
— locks in the `os/cpu[+libc]` rendering used by the
`NoMatchingPlatformVariant` error message.
|
||
|
Zoltan Kochan
|
11f9a27aa2 |
fix(config): default enableGlobalVirtualStore to false (#489)
Pacquet's `default_enable_global_virtual_store()` returned `true` and cited upstream's [`config/reader/src/index.ts:392-394`](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/index.ts#L392-L394) as the authority. But that range lives entirely inside the [`if (cliOptions['global']) { ... }`](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/index.ts#L348-L395) block — surrounded by `CONFIG_CONFLICT_*_WITH_GLOBAL` errors and closed by `} else if (!pnpmConfig.bin) { ... }`. So in pnpm v11: - `pnpm install --global foo` → `enableGlobalVirtualStore` defaults to `true` - `pnpm install` (regular) → stays `null`/unset → effectively `false` Pacquet doesn't have a `--global` CLI flag (only `install --frozen-lockfile`), so the applicable upstream default is the `false` one. Pre-fix, every pacquet install silently wrote slots to `<store>/links/...` and registered projects, even when the user never asked for GVS — and a project alternating between `pnpm install` and `pacquet install --frozen-lockfile` would see two different on-disk layouts. The default introduced by #444 cited the same `L392-L394` range but read it as an unconditional default. Corrected here. ## Test changes - `gvs_default_writes_links_into_global_virtual_store_dir` renamed to `gvs_default_is_off_and_paths_derive_cleanly` and inverted — now asserts the default-off behaviour. The path derivation still populates both `virtual_store_dir` and `global_virtual_store_dir` cleanly so downstream code can read either field without first checking the toggle. - `gvs_user_pinned_virtual_store_routes_into_global_virtual_store_dir` and `yaml_global_virtual_store_dir_wins_over_derivation`: yaml now opts into GVS explicitly (`enableGlobalVirtualStore: true`) so the GVS-on derivation path is still exercised. - `install/tests.rs` doc + inline comments updated to reflect that GVS-on tests need to pin the flag explicitly now. ## Benchmark The bench fixture's explicit `enableGlobalVirtualStore: false` pin is kept (it's now redundant but future-proofs the bench against a default flip), with an updated comment explaining the design intent. Same for `MinimalWorkspaceManifest.enable_global_virtual_store` doc. |
||
|
Zoltan Kochan
|
ff8f458bc8 |
feat(package-manager): hoisted dep-graph installability check (#438 slice 4c) (#491)
Sub-slice 4c of #438. Wires `package_is_installable` into the hoisted-graph walker shipped in 4b (#486), so optional packages on unsupported platforms get filtered into `result.skipped` and engine-strict mismatches surface as typed errors. Single-importer only; store I/O and `prev_graph` diff still to come (4d, then 5). ## Behavior Mirrors upstream's `if (!opts.force && packageIsInstallable(...) === false)` gate at [lockfileToHoistedDepGraph.ts:200-211](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts#L200-L211): | Upstream return | `package_is_installable` (Rust) | Walker action | |---|---|---| | `null` (no constraint) | `Ok(Installable)` | emit node | | `true` (warn but proceed) | `Ok(ProceedWithWarning)` | emit node (warning emit deferred) | | `false` (optional + incompatible) | `Ok(SkipOptional)` | add to `result.skipped`, skip node | | throws `UnsupportedEngineError` (strict) | `Err(InstallabilityError::Engine)` | `HoistedDepGraphError::Installability` | | throws `InvalidNodeVersionError` | `Err(InstallabilityError::InvalidNodeVersion)` | `HoistedDepGraphError::Installability` | ## New options on `LockfileToHoistedDepGraphOptions` - `force: bool` — bypass the check entirely. Used by Slice 4d's `prev_graph` walk, where the previous lockfile is replayed wholesale so the orphan diff catches packages that would now filter out. Mirrors upstream's `force` at [lockfileToHoistedDepGraph.ts:73](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts#L73-L76). - `engine_strict`, `current_node_version`, `current_os`, `current_cpu`, `current_libc`, `supported_architectures` — host-derived axes the check consumes. ## New output on `LockfileToDepGraphResult` - `skipped: BTreeSet<String>` — the input `opts.skipped` cloned and extended with depPaths added during the walk. Upstream mutates the input set in place; pacquet returns the augmented set so the caller can persist it into `.modules.yaml.skipped` without sharing mutable state. ## Tests 15 walker tests total — the 10 from 4b survive (one extended to assert the input depPath survives into output `skipped`), plus five new ones: - `walker_skips_optional_dep_on_unsupported_platform` — Linux host, package targets darwin only, `optional: true` → added to `result.skipped`, no graph entry, no `hoisted_locations`. - `walker_emits_required_dep_with_unsupported_platform_as_warning` — same shape but `optional: false` → walker proceeds (matches upstream `packageIsInstallable === true`); warning log emit is out of scope for 4c. - `walker_errors_on_engine_strict_mismatch` — `engine_strict: true` + `engines.node = ">=99.0.0"` → `HoistedDepGraphError::Installability`. - `walker_force_bypasses_installability_check` — `force: true` emits an incompatible required dep without erroring. - `walker_emits_compatible_dep` — sanity: compatible host + no constraint mismatch → graph entry, no skip. |
||
|
Zoltan Kochan
|
20455ebe9f |
feat: --no-optional filter for frozen lockfile install (#434 slice 5) (#485)
Slice 5 of the [#434 umbrella](https://github.com/pnpm/pacquet/issues/434) (`Proper optionalDependencies support`). Slices 1–4 (#439, #456, #467, #474) handle installability + the fetch-failure swallow. This slice closes the user-facing `--no-optional` flag: the CLI flag already exists and threads through `Install::dependency_groups`, but only `SymlinkDirectDependencies` and the on-disk `included` field actually honored it. The rest of the install pipeline walked `optional_dependencies` unconditionally, so the optional subtree was still extracted, linked, and built. Mirrors upstream's depNode filter at [`installing/deps-installer/src/install/link.ts:109-111`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/link.ts#L109-L111): ```ts if (!opts.include.optionalDependencies) { depNodes = depNodes.filter(({ optional }) => !optional) } ``` ## Changes - **`SkippedSnapshots`**: gains a third disjoint subset `optional_excluded` alongside `installability` (slice 1) and `fetch_failed` (slice 4). The existing `contains` / `iter` already return the union, so every downstream consumer that filters by skip-set (`CreateVirtualStore`, `SymlinkDirectDependencies`, `BuildModules`, hoist, `link_bins`) now respects `--no-optional` through the same gate. `iter_installability` still returns only the persistent subset for `.modules.yaml.skipped` writes — `--no-optional` exclusions are transient (matching upstream's behavior of never updating `opts.skipped` at the filter site). - **`InstallFrozenLockfile::run`**: iterates the lockfile snapshots once and inserts every `snap.optional == true` entry into the new subset when the dispatch list doesn't contain `DependencyGroup::Optional`. The `SnapshotEntry::optional` flag is set by the resolver only when the snapshot is reachable **exclusively** through optional edges, so a snapshot reachable through any non-optional edge carries `optional: false` and survives the filter — covers [`optionalDependencies.ts:712`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/test/install/optionalDependencies.ts#L712) (`dependency that is both optional and non-optional is installed`). |
||
|
Zoltan Kochan
|
dc834bdb8e |
feat(tarball): binary fetcher for zip archives (#437 slice C) (#472)
Slice C of [#437](https://github.com/pnpm/pacquet/issues/437) — the zip half of pnpm's binary fetcher. Slices A (lockfile types) and B (variant picker) have landed; this slice adds the per-archive download and CAS-extraction pipeline. The install-dispatch site (slice D) and the `--no-runtime` flag (slice E) will follow. Mirrors upstream's [`downloadAndUnpackZip` / `extractZipToTarget`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/binary-fetcher/src/index.ts): - `extract_zip_entries` walks every entry. Path validation runs *before* the `is_dir()` early-skip so directory entries like `../evil/` still surface `PATH_TRAVERSAL`. The canonical `cas_paths` / `pkg_files_idx` key is built from `enclosed_name()` rebuilt as `Normal` components joined with `/` — `.` segments collapse, separators are cross-platform consistent, and the ignore filter sees the same string the map is keyed by. - `DownloadZipArchiveToStore` is the public sibling of `DownloadTarballToStore` — same store-index lookup, prefetch cache reuse, store-index writer queue. The retry policy, network-permit shape, and post-download semaphore gating exactly match the tarball path. - `ignore_file_pattern` is `Option<Arc<IgnoreEntryFilter>>` on both `DownloadTarballToStore` and `DownloadZipArchiveToStore`, so the Slice D dispatcher can construct a filter per fetch from runtime config without pinning to `'static`. Cloned per retry attempt; the inner trait object is shared. - `run_with_mem_cache` doc-blocks the "stable filter per URL" invariant: the cache keys solely on `package_url` (matching pnpm's `tarballCache`), and callers keep the (URL, filter) relation functional. Upstream's `archiveFilters` is keyed by `pkg.name`, and tarball URLs encode `(name, version, integrity)`, so this naturally holds. - New `TarballError::ReadZipEntries(std::io::Error)` for zip per-entry I/O failures (`try_reserve` / `read_to_end`); maps to `ERR_PACQUET_ZIP` in the retry classifier, distinct from the tar-specific `ERR_PACQUET_TARBALL_TAR`. Zip `unix_mode()` is masked to `0o777` before write so `CafsFileInfo.mode` stays permission-only (matches `store_dir::add_files_from_dir::file_mode_from`). `TarballError::TarballTooLarge` display generalized from "Tarball at …" to "Archive at …" so the zip pipeline's OOM path doesn't surface a misleading message. - Adds `zip = "5", default-features = false, features = ["deflate"]` to the workspace; reused in `pacquet-tarball`. |
||
|
Zoltan Kochan
|
4cf66757d6 |
feat(package-manager): hoisted dep-graph walker (#438 slice 4b) (#486)
Sub-slice 4b of #438. Implements `lockfile_to_hoisted_dep_graph(lockfile, opts) -> LockfileToDepGraphResult` — the walker that consumes the Slice 3 hoister output and produces the directory-keyed graph + hierarchy + hoisted_locations + direct-deps-by-importer-id + injection-targets-by-dep-path that 4a (#478) pinned the type shape of. Single-importer only (multi-importer lockfiles surface as `HoistError::UnsupportedWorkspace` via the underlying hoister). ## What's deferred Three things are intentionally left to follow-ups so 4b stays focused: - **Store fetch wiring** — `fetching` / `files_index_file` on the graph node are still defaulted. 4c will wire `StoreController::fetchPackage` (and the skip-fetch optimization that consults `currentHoistedLocations`). - **Installability check** — the walker honors `opts.skipped` as input but doesn't run `packageIsInstallable` to add new entries. 4c will plumb `package_is_installable` from `pacquet-package-is-installable` (already used by the isolated linker path). - **`prev_graph` diff** — `prev_graph` is `None`. 4d will walk the *current* lockfile alongside the wanted one (using `force: true` + empty `skipped` per upstream) and produce the orphan-removal input Slice 5's linker consumes. ## Two-pass design Upstream's `fetchDeps` walks asynchronously via `await Promise.all(deps.map(async (dep) => { ... }))`. The key invariant: each sibling's async function runs its sync prologue (insert + push to `pkgLocationsByDepPath`) before any continuation fires, because `Promise.all` sync-invokes each function and the inner `await fetchDeps(...)` only yields after the prologue completes. So by the time any node's post-recursion `graph[dir].children = getChildren(...)` line runs, every node in the entire tree is already in `pkgLocationsByDepPath`. Pacquet runs synchronously. To preserve that invariant cleanly: 1. **Pass 1 (recursive walk).** Insert each node into the graph with `children: BTreeMap::new()` and push its dir to `pkg_locations_by_dep_path`. Recursion order matches upstream's outer body (insert → push → recurse → push to `hoistedLocations`). 2. **Pass 2 (`fill_children`).** Iterate every graph node and resolve `children: alias → dir` from the snapshot's `dependencies` + `optionalDependencies` via `SnapshotDepRef::resolve`, consulting the now-complete `pkg_locations_by_dep_path`. A single-pass walker that computed children inline produced incorrect `children` maps for hoisted transitive deps — the canonical case `root → a → b` with `b` flattened to root left `a.children["b"]` empty because `b`'s pkg_locations entry hadn't been recorded yet. Caught by `walker_transitive_dep_flattens_under_root` before the refactor. ## Tests - `walker_empty_lockfile_produces_empty_result` — empty importer → empty graph, with the `"."` root importer key still present. - `walker_single_root_dep_emits_one_node` — `root → a`, node at `<lockfile_dir>/node_modules/a`. - `walker_transitive_dep_flattens_under_root` — `root → a → b`: `b` hoists to root, `a.children["b"]` points at the root-level dir. - `walker_version_conflict_keeps_loser_nested` — `root → {a@1, c}` with `c → a@2`: `a@1` at root, `a@2` nested under `c`; `hoisted_locations` records both directories; `c.children["a"]` points at the nested copy. - `walker_honors_pre_skipped_dep_path` — depPaths in `opts.skipped` are dropped from the walk entirely (and not recorded in `hoisted_locations`). - `walker_records_directory_resolution_as_injection_target` — `directory:` resolutions populate `injection_targets_by_dep_path` for the post-install re-mirror pass. |
||
|
Zoltan Kochan
|
2c513ad89f |
feat(store-dir): pacquet store prune for the global virtual store (#458) (#475)
Closes #458. Ports upstream's [`pruneGlobalVirtualStore`](https://github.com/pnpm/pnpm/blob/94240bc046/store/controller/src/storeController/pruneGlobalVirtualStore.ts) and the read half of [`projectRegistry`](https://github.com/pnpm/pnpm/blob/94240bc046/store/controller/src/storeController/projectRegistry.ts#L37-L100) (pacquet shipped only the write half in #432 / #444). The CLI wiring already existed — `StoreCommand::Prune` → `StoreDir::prune()` — but `StoreDir::prune` was a `todo!()`. With this PR `pacquet store prune` actually runs. ## What it does **Mark phase**: walk every registered project (`<store>/projects/<short-hash>` → project root). For each project, find every `node_modules/`, follow symlinks that land under `<store>/links/...`, record reachable `<scope>/<name>/<version>/<hash>` slots, and recurse into the slot's own `node_modules/` for transitive deps. **Sweep phase**: walk `<store>/links/<scope>/<name>/<version>/<hash>` and remove every `<hash>` not in the reachable set. Empty `<version>/`, `<name>/`, and `<scope>/` parents get cleaned up. **Self-healing registry**: `get_registered_projects` unlinks entries whose project directory has been deleted (`ENOENT` on stat). Permission / unrelated errors surface as `PROJECT_INACCESSIBLE` / `PROJECT_REGISTRY_ENTRY_INACCESSIBLE` — matches upstream's strict stance, since silently dropping an inaccessible registry entry could remove slots the project still references. |
||
|
Zoltan Kochan
|
006f9fde41 |
perf(git-fetcher): skip CAS re-import when packlist matches input (#436 follow-up) (#479)
* perf(git-fetcher): skip CAS re-import when packlist matches input (#436 follow-up) Port upstream's [`gitHostedTarballFetcher.ts:88-100`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts#L88-L100) fast path: when `resolution.path` is `None` and `packlist.len() == cas_paths.len()`, the materialized tree is byte-identical to the input CAS files and re-importing every entry through hash-dedup is wasted work. Two branches mirror upstream: - `!should_be_built`: queue a `PackageFilesIndex` row at the final key with `requires_build: Some(false)`, then hand the input `cas_paths` straight to the dispatcher. Upstream copies the `\traw` row to the prepared key; pacquet's tarball download doesn't write a raw row at this key today, so the row is synthesized from `cas_paths` instead — the CAS files themselves are already in place. - `should_be_built && ignore_scripts`: return input as-is *without* queueing a row, so subsequent installs re-check the build gate. The synthesized row's per-file digest comes from the CAS path itself (pnpm v11's `files/XX/<rest>[-exec]` layout makes this exact); size comes from `fs::metadata`, mode from the `-exec` suffix. No per-file `fs::read + sha512` — the whole point of the optimization. * fix(git-fetcher): tighten cas_path_digest to full sha512 shape; pin sub-path slow-path CodeRabbit and Copilot review on #479: - `cas_path_digest` now requires the stem to be exactly 126 hex chars (the remainder of a sha512 hex digest after the 2-char shard). Previously accepted any non-empty hex stem, so paths like `.../files/ab/cd` would synthesize a 4-char "digest" that could poison `index.db`. Fail closed on anything that isn't a real v11 CAS path. - Replace the bogus "empty stem" malformed-path case (which never triggered — `Path::file_name()` of `/tmp/ab/` is `Some("ab")`) with cases that actually exercise the new length check: too short, too long, and right-length-with-a-non-hex-byte. - Rebuild `sub_path_never_takes_fast_path` so `packlist.len() == cas_paths.len()` (single sub-package tarball). Only the `path.is_none()` guard now keeps the fast path out — if a future refactor drops it, the test catches the misshaped `packages/sub/...` keys leaking into the dispatcher's output. |
||
|
Zoltan Kochan
|
9b21a726ab |
feat(package-manager): hoisted dep-graph type skeleton (#438 slice 4a) (#478)
* feat(package-manager): hoisted dep-graph type skeleton (#438 slice 4a) Defines the directory-keyed dependency graph types used by the hoisted-linker install path. Types only — the walker that fills them from a lockfile lands in a follow-up. Ported from upstream: - `DependenciesGraphNode` mirrors deps/graph-builder `DependenciesGraphNode` minus the `fetching` / `files_index_file` fields that only the store-controller-bound walker can populate. - `DependenciesGraph` = `BTreeMap<PathBuf, DependenciesGraphNode>`, keyed by absolute directory path (unlike pacquet's existing depPath-keyed `deps_graph` module — hoisted nodes can occupy several directories when a name conflict forces nesting). - `DepHierarchy` is the recursive directory tree the linker walks to decide population order and which `<dir>/node_modules/.bin` to wire up. Newtype-wrapped because Rust doesn't allow recursive type aliases. - `DirectDependenciesByImporterId` is the per-importer alias-to-directory table the linker hands to the bin pass. - `LockfileToDepGraphResult` bundles everything the walker returns to the install pipeline, including `prevGraph` for the orphan-diff pass and `injectionTargetsByDepPath` for the injected-workspace re-mirror step. - `LockfileToHoistedDepGraphOptions` carries the subset of upstream's options that the to-be-ported walker actually reads today; fields tied to the store controller, fetch concurrency, and workspace project list will be added when their consumers land. Smoke tests cover empty-result default construction, node insertion by `dir`, recursive hierarchy nesting, and default-options shape. Upstream: - <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts> - <https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts> * docs(package-manager): v9 depPath in 4a tests + map-key typing notes (#478 review) - Replace v5-era `/accepts/1.3.7` depPath strings in the smoke test with v9-shape `accepts@1.3.7` (the format pacquet's `PkgNameVerPeer` produces and the lockfile snapshots use). The legacy `/name/version` shape is only kept for read-side `hoistedAliases` compatibility. Caught by Copilot. - Tighten doc-comments on `hoisted_locations`, `injection_targets_by_dep_path`, and `skipped` to spell out *why* the keys are plain `String` (not `DepPath`): upstream types those fields with raw `string` in the hoisted-specific interface, even though the values are populated from depPaths. Mirrors upstream's literal typing. |
||
|
Zoltan Kochan
|
b33a481b17 |
feat: proxy support (.npmrc + env-var cascade, HTTP/HTTPS/SOCKS) (#476)
* feat(network): add ThrottledClient::for_installs with proxy support Ports pnpm v11's `getDispatcher` (`network/fetch/src/dispatcher.ts` at SHA |