mirror of
https://github.com/pnpm/pnpm.git
synced 2026-07-19 20:22:21 -04:00
4cf66757d644662f02312153aea0b3bb7632e2dd
501 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Kochan
|
4cf66757d6 |
feat(package-manager): hoisted dep-graph walker (#438 slice 4b) (#486)
Sub-slice 4b of #438. Implements `lockfile_to_hoisted_dep_graph(lockfile, opts) -> LockfileToDepGraphResult` — the walker that consumes the Slice 3 hoister output and produces the directory-keyed graph + hierarchy + hoisted_locations + direct-deps-by-importer-id + injection-targets-by-dep-path that 4a (#478) pinned the type shape of. Single-importer only (multi-importer lockfiles surface as `HoistError::UnsupportedWorkspace` via the underlying hoister). ## What's deferred Three things are intentionally left to follow-ups so 4b stays focused: - **Store fetch wiring** — `fetching` / `files_index_file` on the graph node are still defaulted. 4c will wire `StoreController::fetchPackage` (and the skip-fetch optimization that consults `currentHoistedLocations`). - **Installability check** — the walker honors `opts.skipped` as input but doesn't run `packageIsInstallable` to add new entries. 4c will plumb `package_is_installable` from `pacquet-package-is-installable` (already used by the isolated linker path). - **`prev_graph` diff** — `prev_graph` is `None`. 4d will walk the *current* lockfile alongside the wanted one (using `force: true` + empty `skipped` per upstream) and produce the orphan-removal input Slice 5's linker consumes. ## Two-pass design Upstream's `fetchDeps` walks asynchronously via `await Promise.all(deps.map(async (dep) => { ... }))`. The key invariant: each sibling's async function runs its sync prologue (insert + push to `pkgLocationsByDepPath`) before any continuation fires, because `Promise.all` sync-invokes each function and the inner `await fetchDeps(...)` only yields after the prologue completes. So by the time any node's post-recursion `graph[dir].children = getChildren(...)` line runs, every node in the entire tree is already in `pkgLocationsByDepPath`. Pacquet runs synchronously. To preserve that invariant cleanly: 1. **Pass 1 (recursive walk).** Insert each node into the graph with `children: BTreeMap::new()` and push its dir to `pkg_locations_by_dep_path`. Recursion order matches upstream's outer body (insert → push → recurse → push to `hoistedLocations`). 2. **Pass 2 (`fill_children`).** Iterate every graph node and resolve `children: alias → dir` from the snapshot's `dependencies` + `optionalDependencies` via `SnapshotDepRef::resolve`, consulting the now-complete `pkg_locations_by_dep_path`. A single-pass walker that computed children inline produced incorrect `children` maps for hoisted transitive deps — the canonical case `root → a → b` with `b` flattened to root left `a.children["b"]` empty because `b`'s pkg_locations entry hadn't been recorded yet. Caught by `walker_transitive_dep_flattens_under_root` before the refactor. ## Tests - `walker_empty_lockfile_produces_empty_result` — empty importer → empty graph, with the `"."` root importer key still present. - `walker_single_root_dep_emits_one_node` — `root → a`, node at `<lockfile_dir>/node_modules/a`. - `walker_transitive_dep_flattens_under_root` — `root → a → b`: `b` hoists to root, `a.children["b"]` points at the root-level dir. - `walker_version_conflict_keeps_loser_nested` — `root → {a@1, c}` with `c → a@2`: `a@1` at root, `a@2` nested under `c`; `hoisted_locations` records both directories; `c.children["a"]` points at the nested copy. - `walker_honors_pre_skipped_dep_path` — depPaths in `opts.skipped` are dropped from the walk entirely (and not recorded in `hoisted_locations`). - `walker_records_directory_resolution_as_injection_target` — `directory:` resolutions populate `injection_targets_by_dep_path` for the post-install re-mirror pass. |
||
|
Zoltan Kochan
|
2c513ad89f |
feat(store-dir): pacquet store prune for the global virtual store (#458) (#475)
Closes #458. Ports upstream's [`pruneGlobalVirtualStore`](https://github.com/pnpm/pnpm/blob/94240bc046/store/controller/src/storeController/pruneGlobalVirtualStore.ts) and the read half of [`projectRegistry`](https://github.com/pnpm/pnpm/blob/94240bc046/store/controller/src/storeController/projectRegistry.ts#L37-L100) (pacquet shipped only the write half in #432 / #444). The CLI wiring already existed — `StoreCommand::Prune` → `StoreDir::prune()` — but `StoreDir::prune` was a `todo!()`. With this PR `pacquet store prune` actually runs. ## What it does **Mark phase**: walk every registered project (`<store>/projects/<short-hash>` → project root). For each project, find every `node_modules/`, follow symlinks that land under `<store>/links/...`, record reachable `<scope>/<name>/<version>/<hash>` slots, and recurse into the slot's own `node_modules/` for transitive deps. **Sweep phase**: walk `<store>/links/<scope>/<name>/<version>/<hash>` and remove every `<hash>` not in the reachable set. Empty `<version>/`, `<name>/`, and `<scope>/` parents get cleaned up. **Self-healing registry**: `get_registered_projects` unlinks entries whose project directory has been deleted (`ENOENT` on stat). Permission / unrelated errors surface as `PROJECT_INACCESSIBLE` / `PROJECT_REGISTRY_ENTRY_INACCESSIBLE` — matches upstream's strict stance, since silently dropping an inaccessible registry entry could remove slots the project still references. |
||
|
Zoltan Kochan
|
006f9fde41 |
perf(git-fetcher): skip CAS re-import when packlist matches input (#436 follow-up) (#479)
* perf(git-fetcher): skip CAS re-import when packlist matches input (#436 follow-up) Port upstream's [`gitHostedTarballFetcher.ts:88-100`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts#L88-L100) fast path: when `resolution.path` is `None` and `packlist.len() == cas_paths.len()`, the materialized tree is byte-identical to the input CAS files and re-importing every entry through hash-dedup is wasted work. Two branches mirror upstream: - `!should_be_built`: queue a `PackageFilesIndex` row at the final key with `requires_build: Some(false)`, then hand the input `cas_paths` straight to the dispatcher. Upstream copies the `\traw` row to the prepared key; pacquet's tarball download doesn't write a raw row at this key today, so the row is synthesized from `cas_paths` instead — the CAS files themselves are already in place. - `should_be_built && ignore_scripts`: return input as-is *without* queueing a row, so subsequent installs re-check the build gate. The synthesized row's per-file digest comes from the CAS path itself (pnpm v11's `files/XX/<rest>[-exec]` layout makes this exact); size comes from `fs::metadata`, mode from the `-exec` suffix. No per-file `fs::read + sha512` — the whole point of the optimization. * fix(git-fetcher): tighten cas_path_digest to full sha512 shape; pin sub-path slow-path CodeRabbit and Copilot review on #479: - `cas_path_digest` now requires the stem to be exactly 126 hex chars (the remainder of a sha512 hex digest after the 2-char shard). Previously accepted any non-empty hex stem, so paths like `.../files/ab/cd` would synthesize a 4-char "digest" that could poison `index.db`. Fail closed on anything that isn't a real v11 CAS path. - Replace the bogus "empty stem" malformed-path case (which never triggered — `Path::file_name()` of `/tmp/ab/` is `Some("ab")`) with cases that actually exercise the new length check: too short, too long, and right-length-with-a-non-hex-byte. - Rebuild `sub_path_never_takes_fast_path` so `packlist.len() == cas_paths.len()` (single sub-package tarball). Only the `path.is_none()` guard now keeps the fast path out — if a future refactor drops it, the test catches the misshaped `packages/sub/...` keys leaking into the dispatcher's output. |
||
|
Zoltan Kochan
|
9b21a726ab |
feat(package-manager): hoisted dep-graph type skeleton (#438 slice 4a) (#478)
* feat(package-manager): hoisted dep-graph type skeleton (#438 slice 4a) Defines the directory-keyed dependency graph types used by the hoisted-linker install path. Types only — the walker that fills them from a lockfile lands in a follow-up. Ported from upstream: - `DependenciesGraphNode` mirrors deps/graph-builder `DependenciesGraphNode` minus the `fetching` / `files_index_file` fields that only the store-controller-bound walker can populate. - `DependenciesGraph` = `BTreeMap<PathBuf, DependenciesGraphNode>`, keyed by absolute directory path (unlike pacquet's existing depPath-keyed `deps_graph` module — hoisted nodes can occupy several directories when a name conflict forces nesting). - `DepHierarchy` is the recursive directory tree the linker walks to decide population order and which `<dir>/node_modules/.bin` to wire up. Newtype-wrapped because Rust doesn't allow recursive type aliases. - `DirectDependenciesByImporterId` is the per-importer alias-to-directory table the linker hands to the bin pass. - `LockfileToDepGraphResult` bundles everything the walker returns to the install pipeline, including `prevGraph` for the orphan-diff pass and `injectionTargetsByDepPath` for the injected-workspace re-mirror step. - `LockfileToHoistedDepGraphOptions` carries the subset of upstream's options that the to-be-ported walker actually reads today; fields tied to the store controller, fetch concurrency, and workspace project list will be added when their consumers land. Smoke tests cover empty-result default construction, node insertion by `dir`, recursive hierarchy nesting, and default-options shape. Upstream: - <https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/lockfileToHoistedDepGraph.ts> - <https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts> * docs(package-manager): v9 depPath in 4a tests + map-key typing notes (#478 review) - Replace v5-era `/accepts/1.3.7` depPath strings in the smoke test with v9-shape `accepts@1.3.7` (the format pacquet's `PkgNameVerPeer` produces and the lockfile snapshots use). The legacy `/name/version` shape is only kept for read-side `hoistedAliases` compatibility. Caught by Copilot. - Tighten doc-comments on `hoisted_locations`, `injection_targets_by_dep_path`, and `skipped` to spell out *why* the keys are plain `String` (not `DepPath`): upstream types those fields with raw `string` in the hoisted-specific interface, even though the values are populated from depPaths. Mirrors upstream's literal typing. |
||
|
Zoltan Kochan
|
b33a481b17 |
feat: proxy support (.npmrc + env-var cascade, HTTP/HTTPS/SOCKS) (#476)
* feat(network): add ThrottledClient::for_installs with proxy support Ports pnpm v11's `getDispatcher` (`network/fetch/src/dispatcher.ts` at SHA |
||
|
Zoltan Kochan
|
279469632c |
feat: fetch-failure swallow for optional snapshots + ResolutionFailure payload (#434 slice 4) (#474)
Slice 4 of the [#434 umbrella](https://github.com/pnpm/pacquet/issues/434) (`Proper optionalDependencies support`). Slices 1–3 (#439, #456, #467) close the installability-driven skip path; this slice adds the second skip pathway upstream handles in the frozen install: an `optional: true` snapshot whose **fetch / extract** step fails at install time must not abort the install. Local-materialization errors (`CreateVirtualDir`) and config-shape errors still abort even for optional snapshots — matching upstream's `linkPkg` path which sits outside the catch. Mirrors the silent catch sites at [`deps/graph-builder/src/lockfileToDepGraph.ts:294-298`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L294-L298) and [`installing/deps-restorer/src/index.ts:912-921`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/src/index.ts#L912-L921): ```ts try { ... fetch ... } catch (err) { if (pkgSnapshot.optional) return; throw err } ``` ## Changes - **Reporter**: `SkippedOptionalPackage` becomes a `#[serde(untagged)]` enum with `Installed { id, name, version }` and `ResolutionFailure { name?, version?, bareSpecifier }` variants. Models upstream's discriminated union at [`core-loggers/src/skippedOptionalDependencyLogger.ts:10-31`](https://github.com/pnpm/pnpm/blob/94240bc046/core/core-loggers/src/skippedOptionalDependencyLogger.ts#L10-L31). The new variant is wire-shape-only in slice 4 — pacquet has no resolver yet, but a future resolver port at the upstream emit site ([`resolveDependencies.ts:1376-1383`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-resolver/src/resolveDependencies.ts#L1376-L1383)) lands without re-touching this type. - **`SkippedSnapshots`**: gains a `fetch_failed` subset disjoint from the existing `installability` subset. `contains` / `iter` return the union (downstream consumers treat both as absent); `iter_installability` returns only the persistent subset that `.modules.yaml.skipped` records, matching upstream's behavior of never updating `opts.skipped` at the fetch-failure catch sites. - **`CreateVirtualStore`**: cold-batch dispatch swallows per-snapshot errors when `snapshot.optional` is true **and** the error is fetch-side. A new `is_fetch_side_failure` helper restricts the swallow to `InstallPackageBySnapshotError::DownloadTarball` and `::GitFetch` — the same surface upstream wraps in `storeController.fetchPackage`. Local-materialization (`CreateVirtualDir`) and config-shape errors (`MissingTarballIntegrity` / `UnsupportedResolution`) propagate even for optional snapshots, matching upstream's `linkPkg` path which sits outside the catch. Side benefit: the warm-batch path (which only surfaces `CreateVirtualDir` errors) stays consistently abort-on-error without a parallel swallow site. The per-install `fetch_failed: HashSet<PackageKey>` rides out on `CreateVirtualStoreOutput`. - **`InstallFrozenLockfile::run`**: folds the returned `fetch_failed` into its mutable `SkippedSnapshots` so downstream phases (`SymlinkDirectDependencies`, `LinkVirtualStoreBins`, `BuildModules`, hoist) treat the dropped snapshots as absent through the same gate they already use for installability skips. ## Test plan - [x] `reporter::tests::skipped_optional_resolution_failure_event_matches_pnpm_wire_shape` — full payload (`bareSpecifier` camelCase, no `id`). - [x] `reporter::tests::skipped_optional_resolution_failure_omits_absent_name_and_version` — optional-field shape. - [x] `install::tests::frozen_install_silently_swallows_unreachable_optional_tarball` — ports [`installing/deps-restorer/test/index.ts:340-360`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-restorer/test/index.ts#L340-L360): unreachable optional tarball (`http://127.0.0.1:1/...`), install succeeds, slot not created, `.modules.yaml.skipped` left empty. **Test-the-test verified** by inverting the `optional` guard in the cold-batch match arm. Runs with `enable_global_virtual_store = false` so the slot-path assertion targets the legacy layout. - [x] `install::tests::frozen_install_propagates_non_optional_fetch_failure` — pins the polarity: same fixture, non-optional snapshot, install must error. - [x] `just ready` (typos + fmt + check + nextest + clippy). - [x] `taplo format --check`. - [x] `just dylint` (perfectionist). - [x] `RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --workspace`. ## Out of scope - Resolver-side `resolution_failure` emit site — needs a resolver, tracked separately. - `--no-optional` / `--omit=optional` plumbing — umbrella slice 5. - Surfacing fetch failures to the reporter on the frozen path — upstream itself is silent here; only the resolver-side emit fires `pnpm:skipped-optional-dependency`. Closes #471. |
||
|
Zoltan Kochan
|
5d023a0476 |
test(git-fetcher): port real-npm §E tests with skip_if_no_npm helper (#436 §E follow-up) (#477)
Closes the three §E deferrals that needed a real package manager on the test host. Adds a `skip_if_no_npm()` helper parallel to the existing `skip_if_no_git()` so hosts without node tooling silently skip rather than fail — the rest of the suite stays portable for Rust-only sandboxes. ### Three new tests - **`fetcher_runs_prepare_script_when_allowed`** — ports [`fetching/git-fetcher/test/index.ts:129`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/test/index.ts#L129). Manifest with `scripts.prepare = "node -e \"...writeFileSync('PREPARED.marker', 'ok')\""` and `allow_build` returning true. After the fetcher runs, asserts the marker file lands in `cas_paths` — proving the script actually executed, since the file didn't exist in the source tree. - **`fetcher_surfaces_prepare_failure`** — ports [`fetching/git-fetcher/test/index.ts:212`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/test/index.ts#L212). Prepare script exits non-zero (`node -e "process.exit(1)"`). Expects `GitFetcherError::Prepare(PreparePackageError::LifecycleFailed)` carrying the `ERR_PNPM_PREPARE_PACKAGE` diagnostic code. - **`fetcher_runs_prepare_when_allow_build_returns_true`** — ports [`fetching/git-fetcher/test/index.ts:280`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/test/index.ts#L280). Mirror of the existing block-test (`fetcher_blocks_build_when_not_allowed`) with a per-(name, version) `allow_build` closure: pins the gate's polarity so a regression that inverted "allow=true ⇒ run" would fail this test even though the block-test stays green. A shared `make_bare_repo_with_prepare_script` fixture builder takes a prepare-script string and returns the `(bare_repo, commit_sha)` pair. Manifest carries no dependencies so the synthesized `<pm>-install` step has nothing to fetch — keeps the test self-contained without verdaccio. ### `plans/TEST_PORTING.md` updates Three §E checkboxes (lines 583/587/591) flipped from `[ ]` to `[x]` with the covering pacquet test name. ## Out of scope (still on #436) - Two-slot CAS layout (`${filesIndexFile}\traw` + final). Pure perf optimization. - §E PATH-shim helper for the `still able to shallow fetch for allowed hosts` test (line 586). - §E Stage 2 install-level tests in `installing/deps-installer/test/install/fromRepo.ts` (resolver-dependent). |
||
|
Khải
|
9a0199c0a1 |
feat: auth (#337)
Resolves <https://github.com/pnpm/pacquet/issues/336>. ## Summary Ports pnpm v11's auth flow so `pacquet install` can talk to private registries on the same footing as `pnpm install`. Pacquet now parses credentials out of `.npmrc`, expands `${VAR}` references against the process environment, and attaches the right `Authorization` header on every metadata fetch and tarball download — including tarballs hosted on a CDN host that differs from the registry host. Upstream reference: [`pnpm/pnpm@601317e7a3`](https://github.com/pnpm/pnpm/tree/601317e7a3). ## What's wired up * `_auth`, `_authToken`, `username`, and `_password` keys, both default-registry (bare) and per-registry (`//host[:port]/path/:_authToken=…` family), parsed in `crates/config` (formerly `crates/npmrc`, renamed by #420). * `${VAR}` and `${VAR:-default}` substitution with backslash-escape semantics, ported from [`@pnpm/config.env-replace`](https://github.com/pnpm/components/blob/9c2bd17/config/env-replace/env-replace.ts). Unresolvable placeholders surface a `tracing::warn!` and leave the value verbatim — same best-effort behaviour as pnpm's [`substituteEnv`](https://github.com/pnpm/pnpm/blob/601317e7a3/config/reader/src/loadNpmrcFiles.ts#L156-L162). * URL-keyed lookup (`AuthHeaders`) in `crates/network`, mirroring [`createGetAuthHeaderByURI`](https://github.com/pnpm/pnpm/blob/601317e7a3/network/auth-header/src/index.ts): nerf-darts the request URL, walks parent path prefixes from longest to host-only, falls back to a port-stripped lookup matching upstream's [`removePort`](https://github.com/pnpm/pnpm/blob/601317e7a3/network/auth-header/src/helpers/removePort.ts) (any port, not just protocol defaults), and prefers inline `user:password@` basic auth when present. * Default-registry creds key against the *resolved* registry. The two-phase `apply_registry_and_warn` / `build_auth_headers` split inside `Config::current` ensures `pnpm-workspace.yaml`'s `registry` override propagates to the bearer-token nerf-dart key. * The `Authorization` header is attached on `Package::fetch_from_registry`, `PackageVersion::fetch_from_registry`, and inside `DownloadTarballToStore`'s retry loop. ## Test porting checklist Boxes ticked in `plans/TEST_PORTING.md` for the upstream auth tests this PR ports: * `network/auth-header/test/getAuthHeadersFromConfig.test.ts` — `should convert auth token to Bearer header`, `should convert basicAuth to Basic header`, `should handle default registry auth (empty key)`. * `network/auth-header/test/getAuthHeaderByURI.ts` — all 7 entries (`getAuthHeaderByURI()`, basic-auth-without-settings, basic-auth-with-settings, https-port-443, default-ports, registry-with-pathnames, default-registry-auth). * `config/reader/test/parseCreds.test.ts` — `authToken`, `authPairBase64`, `authUsername and authPassword`. The remaining auth checkboxes (`tokenHelper`, `pnpm auth file` precedence, redirect header-stripping, the `installing/deps-installer/test/install/auth.ts` integration tests) need features pacquet doesn't yet have — token helpers, the `auth.ini` layer, scoped registries, redirect handling — and stay open for follow-ups. ## Notes / scope * `always-auth` is intentionally not honoured. pnpm v11 doesn't either — the auth header is selected by URL match alone, and `always-auth` was deprecated upstream. * TLS / proxy / scoped `@scope:registry` keys remain unparsed; the parser silently ignores them, matching the pre-#336 behaviour. The `creds_by_uri` shape is ready to grow as those land. * The 401/403/404 fail-fast retry policy in `crates/tarball` was already in place from #259; this PR's auth header attaches before the retry loop, so the policy still applies unchanged. * Env-var injection follows the trait-per-capability DI pattern from [pnpm/pacquet#339](https://github.com/pnpm/pacquet/issues/339): `pub trait EnvVar` + `pub struct RealApi` in `crates/config/src/api.rs`, threaded through `env_replace`, `NpmrcAuth::from_ini`, and `Config::current` under one `Api: EnvVar` bound. Production callers turbofish `RealApi` explicitly: `Config::current::<RealApi, _, _, _, _>(...)`. |
||
|
Zoltan Kochan
|
3175a0f3b9 |
test(modules-yaml): pin hoistedLocations round-trip (#438 slice 2) (#473)
Slice 2 of #438 (the `nodeLinker: 'hoisted'` umbrella). Smaller than expected — when I looked, the schema field was already in place from #332's original `.modules.yaml` port, so the work reduced to pinning behavior rather than building plumbing. Two round-trip tests + a doc-comment on the field. ## What was actually missing The umbrella's §"Pacquet's current state" table claimed `hoistedLocations` was "Missing entirely" — that was wrong. The field is at `crates/modules-yaml/src/lib.rs:212` and serde-handles read/write correctly. What was missing: - **A test pinning the wire shape.** Nothing was stopping a future edit from breaking the camelCase mapping, the optional-with-skip-on-None behavior, or the `Record<string, string[]>` shape upstream relies on. - **A doc-comment** explaining what the field is for. Other Slice-related fields (`hoisted_dependencies`, `hoisted_aliases`) have full docstrings; this one was silent. ## Tests added Both live in `crates/modules-yaml/tests/real_fs.rs` (pacquet-side, no upstream counterpart — upstream's `installing/modules-yaml/test/index.ts` doesn't exercise `hoistedLocations` directly): - `hoisted_locations_round_trips` — a manifest with `hoistedLocations` populated survives write+read; the raw on-disk JSON keeps the per-depPath array shape (multi-entry arrays included, to pin the nested-conflict layout). - `absent_hoisted_locations_is_omitted_on_write` — `None` (the state every pacquet install writes today) serializes as the field *absent* from the file, matching upstream's `JSON.stringify(undefined)` behavior. Guards against accidental `Option::is_some` regressions on the `skip_serializing_if`. ## Regression catch verified Per `plans/TEST_PORTING.md`'s "break the subject to verify the test catches it" convention, I temporarily added `#[serde(rename = "wrongName")]` to the field. `hoisted_locations_round_trips` failed at the `expect("present")` on the deserialized value (the camelCase key no longer mapped). Reverted before commit. ## Type-shape decision Upstream's actual `ModulesRaw.hoistedLocations` is `Record<string, string[]> | undefined` — *not* `Record<DepPath, string[]>` despite the values being populated from depPaths internally. The umbrella's scope item ("`Option<BTreeMap<DepPath, Vec<String>>>`") was inconsistent with the upstream schema; I kept the existing `BTreeMap<String, Vec<String>>` because: 1. It already matches the upstream wire shape exactly. 2. Same pattern as the `hoisted_dependencies` field below, which deliberately keeps `String` keys (per its doc-comment at lines 148-153) because pnpm's `DepPath | ProjectId` union can't be statically disambiguated. |
||
|
Zoltan Kochan
|
b61c90d695 |
feat(git-fetcher): real npm-packlist semantics with .npmignore + bundleDependencies (#436 follow-up) (#468)
Replaces the MVP `files`-field-only packlist with a faithful port of [`npm-packlist`](https://github.com/npm/npm-packlist) / [`pnpm/fs/packlist`](https://github.com/pnpm/pnpm/blob/94240bc046/fs/packlist/src/index.ts). Four passes: 1. **`.npmignore` + `.gitignore` walk** via `ignore::WalkBuilder` with per-directory inheritance. A `.npmignore` in `lib/` applies to `lib/**` only; the root `.gitignore` applies to the whole tree. `parents(false)` keeps ignore-file lookup confined to `pkg_dir` so a `.gitignore` *above* the package can't leak into the published file set. 2. **`files`-field allowlist** compiled into a single `Gitignore` matcher. Uses `matched_path_or_any_parents` so a pattern like `cli` matches both `bin/cli` (file match) and `lib/cli/index.js` (directory match, where `lib/cli` is a dir). 3. **Always-included files**: `package.json`, `README*`, `LICEN[SC]E*`, `CHANGES*`, `CHANGELOG*`, `HISTORY*`, `NOTICE*` at the root, plus `main` / `bin` paths. Override `.npmignore` and the `files`-field filter, matching npm-packlist's `alwaysIncluded`. `main` and `bin` are vetted through the always-excluded check before insertion — a manifest with `"main": "package-lock.json"` or `"bin": ".git/hook"` does not get to re-add cruft. 4. **`bundleDependencies` / `bundledDependencies` recursion** with cycle detection: each bundled dep gets its own packlist pass under `node_modules/<name>/`. Both spellings accepted plus the `bundleDependencies: true` shorthand. Cycle protection uses a `HashSet<PathBuf>` of canonical visited paths + a `MAX_BUNDLE_DEPTH = 32` belt-and-braces cap so a symlink loop or self-referencing bundle can't stack-overflow the fetcher. `bundleDependencies` names are validated against the same path-traversal discipline `cas_io::join_checked` uses (`Component::Normal` / `CurDir` only; rejects `..`, root, drive prefix). ### Always-excluded set, split by type - `ALWAYS_EXCLUDED_DIR_SEGMENTS` — `.git`, `.svn`, `.hg`, `CVS`. Matched at any path segment, so VCS state is dropped at any depth. - `ALWAYS_EXCLUDED_BASENAMES` — `.npmrc`, `npm-debug.log`, `.DS_Store`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`. Basename-only match, so a regular file like `lib/cvs-tools.txt` ships even though it mentions `CVS`. - `ALWAYS_EXCLUDED_SUFFIXES` — `.orig` family. ### New runtime dep `ignore = "0.4.25"` (BurntSushi, used by ripgrep; `Unlicense OR MIT`, `cargo deny` clean). Replaces the hand-rolled `*` / `**` / `?` matcher in the old `packlist.rs` — the new matcher uses `ignore::gitignore::GitignoreBuilder::add_line`, which gives full gitignore semantics (negation, anchored vs unanchored, directory-suffix rules) for free. ### Intentional upstream divergences (documented in the module header) - When both `.npmignore` and `.gitignore` exist in the same directory, `ignore` combines their rules; npm-packlist would use only `.npmignore`. Same outcome for the common case (a `.npmignore` that's a strict superset); divergence only when `.npmignore` explicitly includes a path `.gitignore` excludes — rare in published packages. - `.git/info/exclude` and global `~/.gitignore` are NOT honored. The fetcher operates on a clean tarball / checkout, not the user's working tree. |
||
|
Zoltan Kochan
|
1c1f600816 |
feat(graph-hasher): engine-agnostic GVS hash gating (#459) (#469)
* feat(graph-hasher): engine-agnostic GVS hash gating (#459) Port pnpm's `transitivelyRequiresBuild` + `computeBuiltDepPaths` into `graph-hasher`, and extend `calc_graph_node_hash` with the `builtDepPaths`-driven engine-gating that mirrors upstream's [`calcGraphNodeHash`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-hasher/src/index.ts#L140-L142). When a snapshot — and its entire transitive subtree — has no entry in `allowBuilds` (and `dangerouslyAllowAllBuilds` is off), its GVS hash payload now sets `engine: null` instead of `engine: ENGINE_NAME`. Pure-JS packages share one slot directory across Node.js versions, OSes, and CPUs, matching upstream and unlocking the cross-host store sharing the GVS layout was designed for. `VirtualStoreLayout::new` grows an `Option<&AllowBuildPolicy>` parameter. `InstallFrozenLockfile::run` passes the install's `AllowBuildPolicy` through; callers that pass `None` retain the pre-#459 always-include-engine shape (matches upstream's `builtDepPaths === undefined` short-circuit). Tests: - six new `transitively_requires_build` unit tests covering self-hit, transitive ancestor, unrelated tree, missing node, cycle termination, and cycle-doesn't-mask-sibling-builder - four new `calc_graph_node_hash` tests covering the engine-agnostic vs engine-specific paths and the `built_dep_paths = None` shortcut - two end-to-end `VirtualStoreLayout::new` tests proving a pure-JS snapshot's slot directory is identical across two `engine` strings, and that a built snapshot's slot differs --- Written by an agent (Claude Code, claude-opus-4-7). * docs(graph-hasher): address Copilot feedback on calc_graph_node_hash docs - Drop the cross-module intra-doc link `[transitively_requires_build]: crate::dep_state::transitively_requires_build` that resolved to a pub(crate) item. The link rendered correctly under pacquet's CI (which runs `cargo doc --document-private-items`), but downstream doc builds without that flag would trip `private_intra_doc_links`. Switched to a plain-backticks reference + an inline "private to this crate" note so the doc remains informative either way. - Reword the `build_required_cache` hint to avoid the ambiguous `&mut HashMap::new()` phrasing — temporaries-in-call-args is a valid pattern but a confusing one to surface at the API doc. Pointed callers at the explicit `let mut cache = HashMap::new();` + `&mut cache` shape instead. |
||
|
Zoltan Kochan
|
91e537f656 |
feat: persist .modules.yaml.skipped + headless seed-and-recompute (#434 slice 3) (#467)
## Summary Slice 3 of the [#434 umbrella](https://github.com/pnpm/pacquet/issues/434) (`Proper optionalDependencies support`). Slice 1 (#439) computed a `SkippedSnapshots` set on every frozen-lockfile install but never serialized it; slice 2 (#456) closed the input side. This PR closes the loop by mirroring three upstream sites pinned at [pnpm/pnpm@94240bc046](https://github.com/pnpm/pnpm/tree/94240bc046): - **Write** — `Modules.skipped` was declared at [`crates/modules-yaml/src/lib.rs:197`](https://github.com/pnpm/pacquet/blob/97010013/crates/modules-yaml/src/lib.rs#L197) with sort-on-write but always serialized empty. `build_modules_manifest` now takes `&SkippedSnapshots` and produces the depPath string list pnpm writes at [`installing/deps-installer/src/install/index.ts:1625`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/index.ts#L1625). - **Seed** — `install_frozen_lockfile::run` reads `.modules.yaml.skipped` before computing the in-memory set and passes the parsed `PackageKey`s as the seed to `compute_skipped_snapshots`. Mirrors upstream's load at [`installing/read-projects-context/src/index.ts:79`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/read-projects-context/src/index.ts#L79) plus the early return at [`deps/graph-builder/src/lockfileToDepGraph.ts:194`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L194). - **Short-circuit** — `compute_skipped_snapshots` returns the seed verbatim on the fast path (no constraints in the lockfile) and, on the slow path, skips the per-snapshot re-check for keys already in the seed without emitting a duplicate `pnpm:skipped-optional-dependency` event. Non-seeded snapshots still re-run `package_is_installable`, covering the \"host arch changed since last install\" case upstream guards at [`:206-215`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-builder/src/lockfileToDepGraph.ts#L206-L215). `InstallFrozenLockfile::run` now returns a small `InstallFrozenLockfileOutput { hoisted_dependencies, skipped }` struct so `Install::run` can thread both into `.modules.yaml`. Read errors on `.modules.yaml` degrade to an empty seed — the file is a cache artifact, not authoritative. |
||
|
Zoltan Kochan
|
468a80d0a4 |
feat(real-hoist): multi-round convergence + hoistingLimits (#438 slice 3 close-out) (#465)
Closes out the Slice 3 hoist-algorithm port. Two additions on top of slice 3c's peer-aware hoist:
1. **Multi-round convergence.** The DFS runs in a fixed-point loop — a peer-shadow refusal in one round can be reconsidered in a later round once the blocking ident has moved out of the ancestor chain (or a previously-empty root slot has been filled with a compatible version). Bounded by O(N) rounds since every move is one-way.
2. **`hoistingLimits` enforcement.** The upfront `UnsupportedHoistingLimits` guard is gone; the algorithm now reads `opts.hoisting_limits[root_locator]` and refuses to hoist names listed there. New `AbsorbDecision::Border` variant alongside `Free` / `SameNode` / `Conflict` / `PeerShadow`.
Plus a backfill of behavior-pinning tests ported from `@yarnpkg/nm/tests/hoist.test.ts`.
## How multi-round works
- `hoist_subtree` returns `bool` for whether it moved any node in the current round.
- `hoist_into_root` wraps it in a `loop` that resets `visited` each iteration and exits when a round makes no moves.
- Mirrors upstream `hoistTo`'s `do { hoistGraph(); } while (anotherRoundNeeded)` without the per-candidate `dependsOn` bookkeeping; pacquet's coarse re-walk catches the same unlock cases at the cost of revisiting unchanged nodes.
Canonical case (`multi_round_unlocks_peer_friendly_hoist_after_blocker_moves`): `root → app → {widget(peer: x), x@1}` with root carrying no `x`.
- **Round 1.** Alphabetical iteration visits `widget` first. App supplies `x@1`, root has no `x` → mismatch → `PeerShadow`, widget stays at app. Then `x` is evaluated: `Free`, hoist to root.
- **Round 2.** Reconsider widget. App no longer has `x`; root has `x@1`. No ancestor disagreement → `Free`. Widget hoists.
- **Round 3.** No moves; loop exits.
## How hoistingLimits works
- Computed locator: `"{ident_name}@{reference}"` — for the wrapper's root that's `".@"`, matching pnpm's keying convention.
- Border-name set: `opts.hoisting_limits.get(root_locator)`, defaulting to empty when absent.
- New `AbsorbDecision::Border` is layered between the basic decision and the peer check. Names in the set never hoist; they stay where the lockfile placed them.
- Mirrors upstream's `isHoistBorder` flag set during `cloneTree`.
## Upstream test ports
From `yarnpkg/berry@4287909fa6` `packages/yarnpkg-nm/tests/hoist.test.ts`, expressed via lockfile fixtures so they exercise the wrapper too:
- **`hoisting_limits_keeps_blocked_name_at_parent`** — `should not hoist packages past hoist boundary`.
- **`hoisting_limits_blocks_multiple_names`** — `should not hoist multiple package past nohoist root`.
- **`hoisting_limits_keyed_on_unrelated_importer_is_inert`** — non-interference sanity check.
- **`self_dependency_does_not_loop`** — `should tolerate self-dependencies`.
- **`basic_cyclic_dependency_terminates`** — `should support basic cyclic dependencies`.
## Documented gaps
The `nm_hoist` doc-comment now lists what's deferred *intentionally* rather than "not done yet":
- **Popularity-based ident preference.** Upstream's `buildPreferenceMap` picks the ident with more incoming references when two distinct deps share an alias; pacquet picks the first-visited. Affects the handful of upstream test cases that exercise the tie-break (`should honor package popularity when hoisting`, etc.).
- **Multi-importer workspace hoist trees.** Still guarded upfront by `HoistError::UnsupportedWorkspace`. Workspace-aware hoisting needs per-importer roots and a different output shape.
- **`ExternalSoftLink` descendants.** The wrapper only creates soft-links as zero-children placeholders, so upstream's "only-hoist-when-all-descendants-hoist" rule has nothing to delay today. Tests for `should not hoist portal with unhoistable dependencies` and similar were skipped.
## Upstream reference
Aligned with `yarnpkg/berry@4287909fa6`:
- [`hoist.ts:109-119`](
|
||
|
Zoltan Kochan
|
cb6f9e58c3 |
feat(lockfile): select_platform_variant + PlatformSelector (#437 slice B) (#466)
* feat(lockfile): select_platform_variant + PlatformSelector (#437 slice B) Add the variant-picking logic for `VariationsResolution` ahead of Slice D's install-pipeline dispatch: - `PlatformSelector { os, cpu, libc: Option<String> }` mirrors upstream's [`PlatformSelector`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/resolver-base/src/index.ts#L78-L83). The `libc` tri-state encodes pnpm's `string | null | undefined` shape: `None` (host doesn't care about libc — macOS/Windows/BSD) and `Some("glibc")` collapse to the same matching arm (variant must have no `libc:` annotation); `Some("musl")` requires an exact `libc: "musl"` annotation so the glibc default doesn't silently win on a musl host. - `select_platform_variant(variants, selector)` ports [`selectPlatformVariant`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/resolver-base/src/index.ts#L92-L98). Iterates `variants` in declaration order and returns the first variant whose `targets[]` contains an `(os, cpu, libc)` triple matching the selector. Targets-array scan is linear because real archives ship 1–3 target entries per variant; quadratic cost is immaterial. - `libc_matches(variant_libc, requested_libc)` is the asymmetric helper from [`libcMatches`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/resolver-base/src/index.ts#L100-L107). Crate-private so the matching policy isn't part of the public API. Lives in `pacquet-lockfile` next to the variant types (mirrors upstream's `resolver-base` package boundary). The host-platform side (constructing the `PlatformSelector` from `pacquet-graph-hasher`'s existing `host_platform/arch/libc` helpers) lands at the install dispatcher in Slice D — keeps `pacquet-lockfile` free of the `graph-hasher` dep and lets tests drive the picker with synthetic hosts. Tests: first-match wins, multi-target variant matches any host triple, no-match returns `None`, musl-host rejects glibc-default variant, musl-host matches musl-annotated variant, and a six-row `libc_matches` truth table (None / "glibc" / "musl" / unknown-future-libc on both sides) pinning the asymmetric contract from upstream. Part of #437. --- Written by an agent (Claude Code, claude-opus-4-7). * docs: refresh stale references + add declaration-order tie-breaker test Two follow-ups from Copilot review on PR #466: - `resolution.rs:152-154` claimed "the variant picker checks at runtime that the resolved inner is atomic", but `select_platform_variant` does not. Reword to describe the actual contract: pacquet's `PlatformAssetResolution.resolution` is typed as the full `LockfileResolution` for serde uniformity, and the lockfile is trusted to honor upstream's atomic-inner invariant. No infinite-recursion risk because the install dispatcher doesn't call back into `select_platform_variant` for non-`Variations` inputs. - `resolution.rs:169` referenced `pick_variant` in `pacquet-package-manager`, but the picker actually lives in this module as `select_platform_variant`. Updated the pointer. - `pick_returns_first_when_multiple_variants_match` test: two variants both list the same `(darwin, arm64)` target; the test asserts the first one wins, pinning the `Array.prototype.find` semantics. Pnpm-written lockfiles can rely on declaration order (e.g., listing a preferred build before a fallback) — without this test, a future refactor that switched the iteration to a triple-keyed `BTreeMap` would silently break that. No behavior change; doc-comment text + test addition. --- Written by an agent (Claude Code, claude-opus-4-7). |
||
|
Zoltan Kochan
|
e2bebb1eb1 |
feat(lockfile): BinaryResolution + VariationsResolution (#437 slice A) (#457)
## Summary Slice A of #437. Adds the lockfile types pnpm v11 writes for runtime dependencies (`node@runtime:`, `deno@runtime:`, `bun@runtime:`); the install pipeline doesn't dispatch them yet (subsequent slices). No new workspace deps. - **`BinaryResolution { url, integrity, bin, archive, prefix? }`** — mirrors upstream's [`BinaryResolution`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/resolver-base/src/index.ts#L41-L49). `bin` is `BinarySpec::Single(String) | BinarySpec::Map(BTreeMap)` (untagged); `archive` is the lowercase `BinaryArchive::Tarball | BinaryArchive::Zip` discriminator; `prefix` (only set on the `.zip` branch upstream) skips serialization when `None`. - **`PlatformAssetTarget { os, cpu, libc? }`** + **`PlatformAssetResolution { resolution, targets }`** per [`resolver-base`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/resolver-base/src/index.ts#L60-L69). `libc` is `Option<String>` (not an enum) so an upstream addition beyond the current `musl` value lands without a serde migration. - **`VariationsResolution { variants }`** + the `LockfileResolution`/`TaggedResolution` arms routing both new shapes through the existing `from/into ResolutionSerde` round-trip. - **Lockfile major stays at 9** — confirmed against [`core/constants/src/index.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/core/constants/src/index.ts) (`LOCKFILE_MAJOR_VERSION = '9'`), so the existing `lockfile_version.major == 9` assertion holds for v11 lockfiles carrying runtime entries. **Install-dispatch stubs.** `install_package_by_snapshot.rs` and `create_virtual_store.rs` raise `InstallPackageBySnapshotError::UnsupportedResolution { kind: "binary" | "variations" }` until Slice D wires the fetcher. Adding these arms keeps the workspace compiling without forcing the full install path to land in this slice. The warm-prefetch path returns `Ok(None)` for `Variations`, routing through the cold dispatcher where the unsupported-kind error fires; upstream unwraps `Variations` before this layer ever sees one, so the branch is unreachable on well-formed input. |
||
|
Zoltan Kochan
|
0a89e132fc |
feat: add hoisting support (hoistPattern + publicHoistPattern) (#435) (#445)
Closes #435. ## Summary - Ports `@pnpm/config.matcher` (`*` glob, `!` negation, first-match-wins) into a new `pacquet_config::matcher` module — no `regex` dependency. - Ports `getHoistedDependencies` and `symlinkHoistedDependencies` from upstream `installing/linking/hoist/src/index.ts` into a new `pacquet_package_manager::hoist` module. BFS by depth + lex on nodeId, public wins ties, first-seen wins per alias, root importer's direct-dep names seed the "already hoisted" set. - Wires the hoist pass into `InstallFrozenLockfile::run` between the per-slot bin link and the `importing_done` emit. Result is threaded through `Install::run` into `.modules.yaml`'s `hoistedDependencies`. - Promotes `Config.hoist_pattern` and `Config.public_hoist_pattern` to `Option<Vec<String>>` so the install-time guard mirrors upstream's `!= null` check (issue §D). Upstream pinned at pnpm v11 [`94240bc046`](https://github.com/pnpm/pnpm/tree/94240bc046). ## Tests - **Matcher (`crates/config/src/matcher.rs`)** — 6 tests, including direct ports of upstream's `matcher()` and `createMatcherWithIndex()` test cases plus regex-special-char-literal coverage. - **Hoist algorithm (`crates/package-manager/src/hoist/tests.rs`)** — 12 unit tests covering empty graph, default `*` pattern, public hoist, public-wins-ties, negation, first-seen-wins, root direct-dep blocking, skipped-snapshot exclusion, bin-set collection, and graph-builder + direct-deps-builder helpers. - **End-to-end (`crates/cli/tests/hoist.rs`)** — 9 integration tests using pnpm to generate the lockfile and pacquet to install with `--frozen-lockfile`. Cover default private hoist, `publicHoistPattern: '*'`, both-empty, `shamefullyHoist` legacy, scoped-pattern filter, `!`-negation, `.modules.yaml` round-trip, and bin linking on both private and public sides. - **Known failures (17 stubs)** — Ports the remainder of `installing/deps-installer/test/install/hoist.ts` into `known_failures` modules via `pacquet_testing_utils::allow_known_failure!`. Each stub names the upstream URL line + the feature blocking it (workspace install #431, partial install #433, skipped optional deps, direct-dep bin precedence, `extendNodePath`). `plans/TEST_PORTING.md` updated to mark every ported `hoist.ts` entry with the corresponding pacquet test name. |
||
|
Zoltan Kochan
|
918e79176b |
test(git-fetcher): port §E git-fetcher and tarball-fetcher tests from upstream (#436 §E) (#462)
Ports five new tests against the local-bare-repo and write-to-CAS fixtures already established by §B+D and §C, plus the `plans/TEST_PORTING.md` checkbox updates for the §E subset that ships today (15 items checked off; the rest carry concrete deferral reasons — Stage 2 resolver work, real-npm requirements, perf benchmarks, or follow-up tests that need a PATH-shim helper). ### New tests in `crates/git-fetcher/src/fetcher/tests.rs` - `fetcher_packs_subfolder_when_path_set` — ports [`fetching/git-fetcher/test/index.ts:69`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/test/index.ts#L69). Builds a tiny monorepo bare repo, runs `GitFetcher` with `path: Some("packages/sub")`, asserts the returned `cas_paths` only contain files relative to the sub-dir (never sibling packages, never carrying the `packages/` prefix). - `fetcher_handles_repo_without_package_json` — ports [`fetching/git-fetcher/test/index.ts:150`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/test/index.ts#L150). A repo with no `package.json` at root still imports cleanly: `prepare_package` returns `should_be_built: false`, and the fetcher hands the install dispatcher the files the packlist found. - `fetcher_skips_build_when_ignore_scripts` — ports [`fetching/git-fetcher/test/index.ts:247`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/test/index.ts#L247). A repo whose `scripts.prepare` would fail if it ran (`exit 1`) still produces a successful fetcher run when `ignore_scripts: true`; observing success proves the lifecycle runner never spawned. Pins the `should_be_built: true` short-circuit so callers know the package wanted a build. ### New tests in `crates/git-fetcher/src/tarball_fetcher/tests.rs` - `tarball_path_traversal_attack_is_rejected` — ports [`fetching/tarball-fetcher/test/fetch.ts:610`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/tarball-fetcher/test/fetch.ts#L610). `path: Some("../escape")` must surface as `INVALID_PATH` from `prepare_package`'s `safe_join_path` before any extraction. - `tarball_path_to_missing_subdir_is_rejected` — ports [`fetching/tarball-fetcher/test/fetch.ts:637`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/tarball-fetcher/test/fetch.ts#L637). `path: Some("does/not/exist")` for a tarball that doesn't contain the sub-dir must surface as `INVALID_PATH` rather than silently packing the root. ### `plans/TEST_PORTING.md` updates 15 §E items moved from `[ ]` to `[x]` with the covering pacquet test name. Deferrals carry concrete reasons: - **Real-npm dependency** (3 items): `fetch a package from Git that has a prepare script`, `fail when preparing a git-hosted package`, `allow git package with prepare script`. Defer until a `skip-if-no-npm` test helper exists. - **PATH-shim helper required** (1 item): `still able to shallow fetch for allowed hosts`. The predicate is unit-tested; the end-to-end `fetch --depth 1` assertion needs a `git`-binary spy. - **Stage 2 resolver work** (most install-level fromRepo tests + the 4 git-resolver tests). - **Perf benchmarks** (2 items): `fetch a big repository`. ## Out of scope (still on #436) - Two-slot CAS layout (`${filesIndexFile}\traw` + final). - Real `npm-packlist` semantics (`.npmignore`, `.gitignore`, `bundleDependencies` walking). - The Stage 2 / real-npm / PATH-shim deferrals noted above. |
||
|
Zoltan Kochan
|
b53a50f037 |
feat(real-hoist): ancestor-path-aware peer-shadow refusal (#438) (#461)
## Summary Lifts the previous `UnsupportedPeerDependency` upfront guard and replaces it with the real peer check from upstream's `getNodeHoistInfo`. The hoister now accepts lockfiles whose packages declare peer dependencies and produces the layout pnpm would: a peer-constrained dep only floats up to the root when doing so doesn't change which package its peers resolve to. ## How it works - `HoisterResult` carries the input `peer_names` set forward. Upstream's `HoisterResult` doesn't (peer info lives on its intermediate `HoisterWorkTree`); pacquet runs the algorithm on `HoisterResult` directly, so the peer set rides along. - The hoist driver is a recursive `hoist_subtree` that walks the result graph depth-first. Each recursion receives the candidate parent's *current* ancestor path (`Vec<Rc<HoisterResult>>` exclusive of the parent). After a child's hoist decision is applied, the path passed into the child's recursion reflects its *new* position — a freshly-hoisted child recurses with `[root]`, a child that stayed nested recurses with `parent_path + [parent]`. (An earlier version of this PR used BFS with paths captured at queue time; that path went stale whenever a node was hoisted between being queued and dequeued, leading to over-refused hoists. The recursive form was Copilot's recommendation in code review; the bug regression is pinned by `peer_check_uses_post_hoist_ancestor_path_not_queue_time_path`.) - `would_shadow_peer` walks that path to decide whether a candidate hoist would change its own peer resolution. - New `AbsorbDecision::PeerShadow` variant alongside `Free` / `SameNode` / `Conflict`. Fires when: 1. Root declares the candidate's own name as a peer (root-shadow guard — vacuous today since the wrapper's `.` root has empty `peer_names`, kept for parity). 2. For each peer name `P` the candidate declares, the closest ancestor providing `P` does so with a different ident than the root's `P`. Promoting the candidate would silently re-resolve the peer. - The previous `UnsupportedPeerDependency` error variant and `find_first_peer_constrained` upfront scan are gone. The `#[non_exhaustive]` tag stays so future variants can be added without breaking callers. ## DAG-vs-tree caveat Upstream's `cloneTree` duplicates the work tree into a strict tree per parent path, so each candidate visit has a unique ancestor chain. Pacquet preserves the DAG (its identity-dedup is a feature) and records only the path DFS actually used to reach the candidate. In the rare case where the same `Rc` is reachable through both a peer-compatible and a peer-shadowing path, pacquet refuses to hoist — the layout ends up at most over-nested, never under-nested. The trade-off is documented on `would_shadow_peer`. ## Upstream reference - [`hoist.ts:414`]( |
||
|
Zoltan Kochan
|
970100131e |
feat: supportedArchitectures config + --cpu/--os/--libc CLI + real libc detection (#434 slice 2) (#456)
Slice 2 of the [#434 umbrella](https://github.com/pnpm/pacquet/issues/434) (`Proper optionalDependencies support`). Slice 1 (#439) wired `pacquet-package-is-installable` into the install pipeline but the `SupportedArchitectures` input was always `None`, so every install behaved as if the user had opted into \"host triple only\". This PR closes that gap and replaces the slice 1 libc stub with a real Linux probe. Mirrors the three upstream input sources pinned at [pnpm/pnpm@94240bc046](https://github.com/pnpm/pnpm/tree/94240bc046): - **`Config.supported_architectures` from `pnpm-workspace.yaml`** — same shape as upstream's `getOptionsFromRootManifest.ts` ([`config/reader/src/getOptionsFromRootManifest.ts#L23`](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/getOptionsFromRootManifest.ts#L23)). - **`--cpu` / `--os` / `--libc` flags on `install` / `add`** — multi-valued, comma-separated. Per-axis CLI override replaces the yaml axis wholesale, matching upstream's [`overrideSupportedArchitecturesWithCLI.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/overrideSupportedArchitecturesWithCLI.ts). - **Real Linux libc detection** — replaces the slice 1 `\"unknown\"` stub with a `/lib*/ld-musl-*` probe cached via `OnceLock`. Mirrors `detect-libc.familySync()` semantics: `\"musl\"` / `\"glibc\"` on Linux, `\"unknown\"` everywhere else. No new dep — open-coded `read_dir` is cheaper than spawning `ldd --version` and works in slim containers without `ldd` on PATH. Threaded through `Install` / `InstallFrozenLockfile` / `Add` as an explicit field instead of mutating `State.config`, because `State.config` is `&'static Config` — the CLI override merge happens at the CLI layer and the fully-resolved value lands on the install pipeline. ## Test plan Unit tests added; full suite ran via `just ready` + `cargo doc --no-deps` + `taplo format --check` + `just dylint`. - [x] `cargo test -p pacquet-config workspace_yaml::tests::*supported_architectures*` — yaml round-trip across `os` / `cpu` / `libc`, omission, partial-axis (3 tests). - [x] `cargo test -p pacquet-package-manager --lib installability::tests::supported_architectures*` — `supportedArchitectures` widens the accept set so a darwin-only package stays on a linux host when the user lists `darwin`; conversely, listing `linux` keeps the darwin-only package skipped (no implicit host include). - [x] `just ready` — typos + fmt + check + nextest (792 tests pass) + clippy. - [x] `taplo format --check`. - [x] `just dylint` (perfectionist). - [x] `RUSTDOCFLAGS=\"-D warnings\" cargo doc --no-deps --workspace`. ## Out of scope - `.modules.yaml.skipped` persistence (umbrella slice 3). - Current-lockfile diffing for `removeOptionalDependenciesThatAreNotUsed` (umbrella slice 6). - `--no-optional` plumbing (umbrella slice 5). Closes #453. |
||
|
Zoltan Kochan
|
cad3782c97 |
feat(git-fetcher): warm prefetch for git-hosted snapshots (#436 follow-up) (#454)
## Summary
Wires git-hosted installs into pacquet's existing `index.db` warm-cache path. Before this PR every git install cold-pathed, even when the snapshot was already in the store from a previous install. Now the second install of an unchanged lockfile reuses the previous install's clone + checkout + prepare + packlist work and skips the fetcher entirely — same as registry-tarball installs do today.
### Three pieces
1. **Both fetchers now write `PackageFilesIndex` rows.** `cas_io::import_into_cas` returns the full `HashMap<String, CafsFileInfo>` payload alongside the existing `cas_paths` map; the fetchers wrap that into a `PackageFilesIndex` and queue it on the shared `StoreIndexWriter` at the cache key the dispatcher passed in.
- `algo: "sha512"`, `manifest: None`, `requires_build` reflects `should_be_built`.
- `add_files_from_dir`'s `file_mode_from` is duplicated locally so the cache row encodes the same POSIX-mode shape pnpm's tarball entries use.
2. **`create_virtual_store::snapshot_cache_key` routes git arms through the warm batch.** For `Tarball { gitHosted: true }` and `Git` resolutions it now returns `Some(gitHostedStoreIndexKey(pkg_id, built=true))` (matches upstream's `pickStoreIndexKey` shape) instead of the previous `None` cold-batch shortcut. `built = true` matches the dispatcher's `ignore_scripts: false` default; both sites will flip together when an `ignore-scripts` config knob lands.
3. **Dispatcher threads writer + key into both fetchers.** `install_package_by_snapshot.rs` builds the same `gitHostedStoreIndexKey(package_id, built=true)` the warm path reads and hands it to `GitFetcher.files_index_file` / `GitHostedTarballFetcher.files_index_file`, plus the shared `store_index_writer` clone.
The warm prefetch's existing file-verification (`check_pkg_files_integrity`) runs unchanged — if any CAS file referenced by the row has been pruned, the snapshot falls through to cold and the fetcher re-runs.
## Out of scope
- Two-slot CAS layout (`${filesIndexFile}\traw` + final) for skipping the re-import when packlist == raw. Pure perf optimization on top of this PR. Tracked on #436.
- §E full integration-test matrix.
- Real `npm-packlist` semantics.
|
||
|
Zoltan Kochan
|
28046aab81 |
feat(real-hoist): single-pass parent-wins hoist (#438) (#452)
## Summary Replaces the stub `nm_hoist` (landed in #448) with a working hoister and the surrounding guardrails: BFS over the result graph pulling every eligible descendant up to the root, plus upfront refusal of inputs the algorithm doesn't yet model. ### What the algorithm models - **Free hoist.** A transitive dep with no name collision at the root surfaces at the root. - **Identity dedup.** A dep reachable through multiple parents shares one `Rc` thanks to the wrapper's cache; the hoist preserves that identity and strips the duplicate reference at the other parent path. - **Parent-wins on version conflict.** When two distinct deps share an alias but resolve to different snapshot keys, the first BFS visitor takes the root slot and the other stays under its declaring parent. Visit order matches the wrapper's alias-sorted insertion order, so the outcome is deterministic. - **Deep chains flatten in one pass.** `root → a → b → c → d` becomes `root → {a, b, c, d}` — each node, once hoisted, is queued for further descent; its own children evaluate against root's slots rather than against the (now-empty) intermediate parent. - **O(1) root-slot lookup.** A side `HashMap<String, RcByPtr<HoisterResult>>` mirrors root's direct deps, so the per-edge "is this name taken?" check doesn't degrade to O(N²) `IndexSet` scans on flat graphs. ### Fail-fast guards The algorithm doesn't yet model peers, hoisting limits, or multi-importer (workspace) hoist trees. Rather than emit a layout pnpm would reject, the wrapper refuses these inputs upfront with three new `HoistError` variants: - `UnsupportedPeerDependency { ident, peers }` — fires when scanning the constructed `HoisterTree` finds any node with non-empty `peer_names` (either `peerDependencies` from the `packages:` map or `transitive_peer_dependencies` on a `snapshots:` entry). - `UnsupportedHoistingLimits { len }` — non-empty `opts.hoisting_limits`. - `UnsupportedWorkspace { extra_importers }` — any importer beyond the root `.`. Each carries enough context for an operator to identify what triggered it. The `UnsupportedWorkspace` help points at `SymlinkDirectDependencies` — workspaces *do* work in pacquet's wider install path (workspace support landed in #443 for the isolated linker); only the hoister is restricted. ### Rebase pickup `collect_importer_deps` carries the `Link`-variant skip introduced by #443 (workspace support) — `spec.version.as_regular()` extracts the snapshot key for `Regular` deps and `continue`s past `Link` deps, since workspace siblings are direct symlinks materialised by `SymlinkDirectDependencies` and have no snapshot to hoist. ### Performance Nothing in this PR is reachable from `pacquet install` today (`crates/package-manager/src/install*.rs` doesn't import `pacquet-real-hoist` — the crate is dead code from the install pipeline's perspective until the hoisted-linker wiring lands in later slices). The benchmark results bear that out: cold Frozen Lockfile is within CI variance (+3.2% mean, +3.5% median, driven by one outlier), Hot Cache is 6% *faster* (same `stat → lstat` improvement that shipped in slice 1's `import_indexed_dir`), micro is identical. No code path explains a real regression. ## What's deferred - Peer-dependency-aware hoisting (`peer_names` constraints, peer-promise satisfaction). - Multi-round convergence — the BFS handles deep chains in one pass, so the cases requiring true multi-round are limited to peer interactions. - `hoistingLimits` runtime enforcement. - `dependencyKind` distinctions for workspaces and external soft links (today only `ExternalSoftLink` placeholders are added by the wrapper and stripped post-hoist; `Workspace`-kind nodes are blocked by the `UnsupportedWorkspace` guard upfront). ## Upstream reference - [`hoist.ts` algorithm overview]( |
||
|
Zoltan Kochan
|
03499dd884 |
feat: activate global-virtual-store install path (#432) (#449)
## Summary Threads `VirtualStoreLayout` (from #444) through the frozen-lockfile install pipeline so packages installed with `enableGlobalVirtualStore: true` (pacquet's default since #444) land at the upstream-shaped `<store_dir>/links/<scope>/<name>/<version>/<hash>/node_modules/<pkg>` path. Closes the rest of #432 (Section B + C activation + D entry tests). ## Pipeline changes (frozen-lockfile path) - `InstallFrozenLockfile::run` constructs the install-scoped `VirtualStoreLayout` from the lockfile + engine name + config, then threads `&VirtualStoreLayout` through `CreateVirtualStore`, `CreateVirtualDirBySnapshot`, `create_symlink_layout`, `SymlinkDirectDependencies`, `InstallPackageBySnapshot`, `BuildModules`, and `LinkVirtualStoreBins`. Every site that used to compute `virtual_store_dir.join(key.to_virtual_store_name())` now goes through one `layout.slot_dir(key)` lookup. - `Install::run` calls `pacquet_store_dir::register_project` **once per importer** when `enable_global_virtual_store` is on, writing `<store_dir>/projects/<short-hash>` per workspace package so a future `pacquet store prune` can find every reachable consumer of `<store_dir>/links/...`. Best-effort: registry write failures degrade to `tracing::warn!` and the install continues. The walk runs *after* `InstallFrozenLockfile::run` succeeds so importer keys have already been validated. - `Config::current` now applies `apply_global_virtual_store_derivation` after yaml — pacquet's variant of upstream's [`extendInstallOptions.ts:343-355`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/extendInstallOptions.ts#L343-L355). An explicit `globalVirtualStoreDir:` in yaml wins over the derivation; otherwise the field falls back to the user's pinned `virtualStoreDir` (under GVS-on) or to `<store_dir>/links`. ## Field-split deviation from upstream Upstream pnpm *mutates* `virtualStoreDir` in place under GVS, so every consumer that reads `virtualStoreDir` sees `<storeDir>/links`. Pacquet keeps `virtual_store_dir` at its project-local value and writes the GVS path into the separate `global_virtual_store_dir` field. The frozen-lockfile path consumes `global_virtual_store_dir` through `VirtualStoreLayout`; the legacy non-frozen `InstallWithoutLockfile` keeps reading `virtual_store_dir` (now via `VirtualStoreLayout::legacy`). The split is a pacquet-only concern: upstream has no without-lockfile install path and so doesn't need the second field. See the doc on `Config::apply_global_virtual_store_derivation` for the reasoning. ## Benchmarks The integrated-benchmark fixture pins `enableGlobalVirtualStore: false` so existing scenarios continue to compare apples-to-apples against the project-local `<modules>/.pnpm/<flat-name>` shape. Without the pin, every revision newer than this PR would silently switch to the GVS layout, while `pacquet@main` and the pnpm side of the comparison would still use the isolated layout — different shape, unfair comparison. GVS-specific benchmark runs are available by passing `--fixture-dir <dir>` at a copy of the fixture with the flag flipped. ## Tests - `install::tests::frozen_lockfile_under_gvs_registers_project_and_runs_clean` pins the registry write under default GVS-on (single-project case). - `install::tests::frozen_lockfile_under_gvs_registers_each_workspace_importer` pins per-importer registration: workspace root + `packages/web` each end up with their own symlink in `<store_dir>/projects/`. - `install::tests::frozen_lockfile_with_gvs_off_skips_project_registry` pins that GVS-off opts out of the registry write. - `config::tests::yaml_global_virtual_store_dir_wins_over_derivation` pins the yaml-override precedence. - All **829 workspace tests** pass under the new layout-threaded path. - Existing per-snapshot legacy tests construct `VirtualStoreLayout::legacy(...)` explicitly so they keep asserting the flat-name layout regardless of any global default. Partial-install tests from #442 pin `enable_global_virtual_store = false` so their seeded legacy-shape slots match the probe path. End-to-end port of upstream's [`installing/deps-installer/test/install/globalVirtualStore.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/test/install/globalVirtualStore.ts) is tracked as a follow-up — those tests need a small frozen-lockfile fixture against the registry-mock that doesn't exist yet. |
||
|
Zoltan Kochan
|
f33838cdd1 |
feat: workspace support for pacquet install --frozen-lockfile (#431) (#443)
Stage 1 of pnpm/pacquet#299 / closes #431 (Stage 1 scope) / partially closes #357. - **New crate `pacquet-workspace`** — ports pnpm's `workspace/root-finder`, `workspace/workspace-manifest-reader`, `workspace/projects-reader`, and `workspace/project-manifest-reader` (pinned to [`94240bc046`](https://github.com/pnpm/pnpm/tree/94240bc046)). `find_workspace_dir` honours `NPM_CONFIG_WORKSPACE_DIR` (and its lowercase spelling, with empty-value fall-through), walks up to `pnpm-workspace.yaml`, and rejects misnamed variants (`pnpm-workspace.yml`, `.pnpm-workspace.yaml`, ...) with `BAD_WORKSPACE_MANIFEST_NAME`. Project enumeration glob-expands `packages:` via `wax`, always includes the workspace root (https://github.com/pnpm/pnpm/issues/1986), filters `**/node_modules/**` + `**/bower_components/**`, sorts lex by `rootDir`, and preserves the omitted-vs-explicit-empty distinction so `packages: []` enumerates only the root rather than falling back to the recursive `['.', '**']` default. - **Per-importer install path** — `SymlinkDirectDependencies` iterates every entry in `Lockfile.importers`, computes per-project `rootDir` and `node_modules/`, and uses `rootDir` as the `prefix` for `pnpm:root added` events (matching upstream's [`linkDirectDeps`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/linking/direct-dep-linker/src/linkDirectDeps.ts#L131)). Install-wide events (`pnpm:stage`, `pnpm:summary`) still use `lockfileDir`, now resolved through `find_workspace_dir`. `MissingRootImporter` is gone — an empty importers map is a silent no-op. A custom `modulesDir` from `pnpm-workspace.yaml` propagates to every importer (the symlink stage reads `config.modules_dir.file_name()` rather than hard-coding `node_modules`). - **Path-anchoring consistency** — `Config::current` re-anchors `modules_dir` / `virtual_store_dir` to the workspace root when `pnpm-workspace.yaml` is found in an ancestor, mirroring pnpm v11's `pnpmConfig.dir = lockfileDir`. Running `pacquet install` from a workspace subdirectory no longer produces two inconsistent `node_modules` layouts (subdir virtual store vs workspace-root per-importer symlinks). `Config::current` also honours `NPM_CONFIG_WORKSPACE_DIR` so the env-var override drives both `find_workspace_dir` and the path anchors. `workspace_root` is threaded through `InstallFrozenLockfile` as a real `&Path` (no lossy-UTF-8 round-trip), so non-UTF-8 filenames survive the install layer intact. - **Lockfile `link:` support** — `ResolvedDependencySpec.version` is now an `ImporterDepVersion` enum (`Regular(PkgVerPeer) | Link(String)`), so a v9 workspace lockfile with `version: link:../shared` parses. The symlink stage resolves the link target against the importer's `rootDir` and points `node_modules/<name>` at the sibling project rather than the virtual store; `BuildModules`, `pacquet-real-hoist`, and the side-effects-cache write path all skip `Link` variants when collecting snapshot-key roots. - **Importer-key validation** — rejects absolute POSIX paths, Windows drive prefixes, backslash separators, `..` segments, and the empty string with a typed `UnsafeImporterPath` error, so a malformed (or hostile) lockfile cannot make `Path::join` create `node_modules` outside the workspace root. - **Symlink error propagation** — the prior `expect("symlink pkg")` is replaced with `try_for_each` + a `SymlinkDirectDependenciesError::SymlinkPackage` variant carrying the `importer_id`, `name`, and underlying `SymlinkPackageError`. A filesystem failure no longer crashes the rayon pool. - **Adds `wax = 0.7.0`** to `[workspace.dependencies]` for project globbing. |
||
|
Zoltan Kochan
|
5b4c090db7 |
fix(package-manager): add missing path field in installability test fixture (#455)
Landrace between #439 (installability tests fixture) and #451 (added `path: Option<String>` to `TarballResolution`). Both merged green against their own bases but main's tip won't compile: error[E0063]: missing field `path` in initializer of `TarballResolution` --> crates/package-manager/src/installability/tests.rs:56:49 Add the missing field with `path: None` so the struct literal matches the current `TarballResolution` shape. |
||
|
Zoltan Kochan
|
0f51f820df |
feat: frozen-lockfile freshness check (#447) (#450)
## Summary Closes #447. Adds a `satisfies_package_manifest` check to pacquet's frozen-lockfile dispatcher so a stale `pnpm-lock.yaml` no longer silently installs the wrong shape of `node_modules`. Before this, `pacquet install --frozen-lockfile` would happily materialize whatever the lockfile said, even when a dev had edited `package.json` (added, removed, or bumped a dep) without re-running the resolver. Upstream pnpm catches this at the dispatch site with `ERR_PNPM_OUTDATED_LOCKFILE` ([`pkg-manager/core/src/install/index.ts:808-832`](https://github.com/pnpm/pnpm/blob/94240bc046/pkg-manager/core/src/install/index.ts#L808-L832)); this PR ports the same gate. ### What's checked Ported from upstream's [`satisfiesPackageManifest`](https://github.com/pnpm/pnpm/blob/94240bc046/lockfile/verification/src/satisfiesPackageManifest.ts). Four phases, short-circuiting on the first failure: 1. **Flat-record specifier diff** over `dependencies ∪ devDependencies ∪ optionalDependencies`. Catches added/removed/modified deps in one bucket; rendered as a `SpecDiff` with per-bucket lists. 2. **`publishDirectory`** parity between the importer entry and the manifest's `publishConfig.directory`. 3. **`dependenciesMeta`** JSON equality between the importer and the manifest (with absent ≡ empty-object equivalence to match upstream's `?? {}` coercion). 4. **Per-field name-set + specifier match.** Catches same-name-same-specifier moves between fields the flat-record diff doesn't see (e.g. `react` moved from `dependencies` to `devDependencies`). Applies upstream's **precedence rule** (`optional` > `prod` > `dev`): a dep that appears in a higher-precedence manifest field is filtered out of the lower-precedence field's check, so a manifest listing the same dep in both prod and dev still satisfies a lockfile that records it under prod only. Reasons are surfaced as typed `StalenessReason` variants (`SpecifiersDiffer`, `DepSpecifierMismatch`, `PublishDirectoryMismatch`, `DependenciesMetaMismatch`, `NoImporter`) so callers match on the discriminant rather than parsing format strings, and tests assert on shape rather than wording. The `SpecDiff::Display` impl handles singular/plural ("1 dependency was added" vs "2 dependencies were added") so user-facing CI output reads cleanly. ### New errors - `InstallError::OutdatedLockfile { reason: StalenessReason }` — surfaced as `ERR_PNPM_OUTDATED_LOCKFILE` (miette code `pacquet_package_manager::outdated_lockfile`). Hint points at `pnpm install --lockfile-only` to regenerate. - `InstallError::NoImporter { importer_id }` — distinguishes "lockfile file is missing" (`NoLockfile`) from "lockfile is present but has no importer entry for this project." Renders as `importers["{id}"]` for readability. ### Performance Confirmed by hyperfine on a 1352-package fixture with 110-dep manifest (warm reinstall, `--warmup 2 --runs 10`): | | mean | range | |---|---:|---:| | main (no check) | 573.5 ms | 554-603 | | **PR (with check)** | **574.7 ms** | 549-602 | Ratio 1.00 ± 0.05 — within noise. The check is pure-CPU map/set operations on string keys with no syscalls or async; ~50-200 μs for the alot7 manifest. ### Out of scope (matching the issue) - Catalogs (pacquet has no catalog support yet). - `auto-install-peers` pre-pass (pacquet has no separate auto-install-peers mode). - `excludeLinksFromLockfile` and `link:` resolutions (#431 territory). - Semver-range-satisfies check (lives in pnpm's `localTarballDepsAreUpToDate`, outside this gate). - Multi-importer support (#431 workspace install — single-importer-only here). |
||
|
Zoltan Kochan
|
48a134f770 |
feat(git-fetcher): install git-hosted tarballs via preparePackage + packlist (#436 §C) (#451)
## Summary Builds on §B+D (#446). Adds the second half of the git-hosted install path: `TarballResolution { gitHosted: true }` lockfile entries — the ones whose tarballs land at `codeload.github.com` / `gitlab.com` / `bitbucket.org` archive endpoints — now run through a new `GitHostedTarballFetcher` after the existing `DownloadTarballToStore` settles their bytes into CAS. Mirrors pnpm's [`gitHostedTarballFetcher.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts): 1. Materialize the CAS-resident files into a writable temp dir (`fs::copy` per file — hardlinking here would let prepare scripts mutate shared CAS entries). 2. Run the existing `prepare_package` port that §B+D shipped — same `<pm>-install` + `prepublish` / `prepack` / `publish` dispatch, same `GIT_DEP_PREPARE_NOT_ALLOWED` gate. 3. Run the existing minimal `packlist` over the prepared tree to drop source maps, test fixtures, etc. that ship in the raw archive but not in the published file set. 4. Re-import the filtered file set back into CAS and hand the `HashMap<String, PathBuf>` to `CreateVirtualDirBySnapshot`. ### Shared `cas_io` module The CAS-import helpers (`import_into_cas`, `is_file_executable`, `map_write_cas`) plus a new `materialize_into` lift out of `fetcher.rs` into a shared `cas_io` module so both fetchers consume the same code path. No public-API change — helpers stay `pub(crate)`. `cas_io` also gains a `join_checked(root, rel)` helper that rejects absolute paths, `..` components, root, and drive-prefix components before joining, so a malicious tarball entry can't escape `target_dir` / `pkg_dir`. Errors surface as `GitFetcherError::Io(InvalidInput)`. Both `materialize_into` and `import_into_cas` route through it, taking `rel` directly (forward slashes are valid on every platform — `Path::components()` recognises both `/` and `\` on Windows — so no per-file `String` allocation is needed). ### Supporting changes - `TarballResolution.path: Option<String>` mirrors pnpm's `TarballResolution.path` so a lockfile pinning a git-hosted tarball from a monorepo (`path: packages/sub`) deserializes without tripping `deny_unknown_fields`. The fetcher threads this through to `prepare_package`'s `safe_join_path` so prepare and packlist run inside the right sub-directory. - The install dispatcher hoists the `allow_build` closure and the `ScriptsPrependNodePath` conversion above the resolution match so both the `Git` arm and the new `gitHosted: true` post-pass borrow from the same named locals — no temp closures across `.await`. - `InstallPackageBySnapshotError::GitFetch`'s doc now spells out that the variant covers failures from both fetchers (the git-CLI path and the git-hosted-tarball post-pass), since both share the same `GitFetcherError` taxonomy. ## Out of scope (follow-ups for #436) - Warm prefetch for git-hosted slots — neither fetcher writes `gitHostedStoreIndexKey` rows to `index.db` yet, so a re-install re-runs clone + prepare + packlist. Correct, but slow on the second run. - Two-slot CAS layout (`${filesIndexFile}\traw` + final). Upstream optimizes "raw == prepared" by promoting the raw entry in place. Pacquet always re-imports today; hash-dedup means duplicate work but not duplicate storage. - §E — full integration-test matrix in `plans/TEST_PORTING.md` 563-572. - Real `npm-packlist` semantics (`.npmignore`, `.gitignore` layering, `bundleDependencies` walking). |
||
|
Zoltan Kochan
|
ca88affd39 |
feat(package-is-installable): platform + engine check, skip optional incompatibles (#434 slice 1) (#439)
Slice 1 of #434 — foundational installability gate. Ports [`@pnpm/config.package-is-installable`](https://github.com/pnpm/pnpm/tree/94240bc046/config/package-is-installable) and threads the resulting skip-set through every phase of the frozen-lockfile install pipeline. Closes #266 once shipped (covers the "install respects every snapshot — no os/cpu/libc filter" gap). Does **not** close #434 — that umbrella has six more slices to follow. Upstream reference: pnpm/pnpm@94240bc046. ## What landed ### New crate: `pacquet-package-is-installable` Ports the upstream `config/package-is-installable` package's three helpers: - `check_platform` (`Option<&[String]>` for each `os`/`cpu`/`libc` axis, plus a `SupportedArchitectures` override) — returns `Option<UnsupportedPlatformError>` matching upstream's `ERR_PNPM_UNSUPPORTED_PLATFORM` code, message shape, and JSON payload. Handles negation entries (`!foo`), the `any` sentinel, the `current` placeholder, and the `currentLibc !== 'unknown'` skip from `checkPlatform.ts:38`. - `check_engine` — evaluates `engines.node` / `engines.pnpm` via `node-semver`. Approximates npm-semver's `includePrerelease: true` via a strip-prerelease fallback; one over-acceptance edge case (`>=X.Y.Z` against `X.Y.Z-rc1`) is pinned in the known-failures integration test for follow-up. - `package_is_installable` — composes the two, returns the tri-state verdict matching upstream's `boolean | null` (Installable / SkipOptional / ProceedWithWarning), plus an `Err` arm for `engine_strict` aborts and `ERR_PNPM_INVALID_NODE_VERSION`. `InstallabilityOptions<'a>` borrows its host strings so a caller running through many snapshots in a row can build the host part once and only toggle `optional` per snapshot. `WantedPlatformRef<'a>` plays the same role for the manifest axes so `check_platform` runs the happy path without any allocation. ### New module: `pacquet_package_manager::installability` `compute_skipped_snapshots` is the per-install entry point. For each snapshot: 1. Look up the matching `PackageMetadata`. 2. Run `check_package` (cached per peer-stripped `metadata_key` so peer-resolved variants of the same package share one verdict). 3. Dispatch on `(verdict, snapshot.optional, host.engine_strict)`: - `Installable`: nothing to do. - `SkipOptional` + `optional`: add to `SkippedSnapshots`, emit `pnpm:skipped-optional-dependency` (deduped per metadata key, matching upstream's emit-per-pkgId). - `Incompatible` + non-optional + `engine_strict`: abort. - `Incompatible` + non-optional + non-strict: `tracing::warn!` and proceed. (Upstream's `pnpm:install-check` channel isn't wired into pacquet's reporter yet — slice 1 follow-up.) `any_installability_constraint(packages)` is the caller-side fast path: if no metadata row declares an `engines.{node,pnpm}` or a non-empty / non-`["any"]` `cpu`/`os`/`libc`, the entire installability pass is skipped. The probe runs synchronously in `install_frozen_lockfile` *before* the `tokio::task::spawn_blocking` that would invoke `node --version` — so the common no-constraints lockfile pays nothing for the new pipeline, restoring main's overlap of node-binary startup with extraction. ### Install-pipeline plumbing The `SkippedSnapshots` set is threaded into every downstream phase of `InstallFrozenLockfile::run`: - `CreateVirtualStore`: installability-skipped snapshots are dropped from both `survivors` (no virtual-store slot extracted) and `skipped_entries` (no warm-cache row). Layered ahead of main's #442 already-installed-and-on-disk skip filter. - `SymlinkDirectDependencies`: a direct dep whose resolved snapshot is in the skip set is omitted from `node_modules/<name>` (no symlink, no `pnpm:root added` event, no bin link). - `LinkVirtualStoreBins`: per-slot bin link skips slots whose snapshot is installability-skipped (their virtual-store directories don't exist). - `BuildModules` via `build_sequence`: `get_subgraph_to_build` consults `skipped` *before* recursion, so a skipped snapshot's subtree doesn't contribute to the build graph via that edge. Descendants reachable from a non-skipped root still build normally. ### Performance CI integrated-benchmark on the 1352-package fixture, latest run: | Scenario | `pacquet@HEAD` | `pacquet@main` | Relative | |---|---|---|---| | Frozen Lockfile (cold) | 2.476 ± 0.083 s | 2.442 ± 0.071 s | 1.01 ± 0.05 | | Frozen Lockfile (Hot Cache) | 685.8 ± 59.3 ms | 700.2 ± 47.4 ms | 1.00 | Earlier iterations of this PR showed a ~5% cold-install regression from the `node --version` spawn landing on the extraction critical path. Closed by hoisting the no-constraints fast-path probe to the caller (commit `cf47ce51`) so the spawn is gated on actual constraint presence. Other perf passes folded in: - `compute_skipped_snapshots` caches the per-metadata-row check verdict so peer-resolved variants share one `check_package` call. - `check_platform` borrows its three wanted axes through `WantedPlatformRef<'a>`; the owned `WantedPlatform` only materialises in the error path. ## Tests | Suite | Count | What it covers | |---|---|---| | `pacquet-package-is-installable::tests::check_platform` | 16 | Port of upstream `checkPlatform.ts` — `any`/`current` sentinels, negation, `supportedArchitectures` override, libc unknown-skip | | `pacquet-package-is-installable::tests::check_engine` | 7 | Port of upstream `checkEngine.ts` — node/pnpm range checks, prerelease cases, `ERR_PNPM_INVALID_NODE_VERSION` | | `pacquet-package-is-installable::tests::package_is_installable` | 6 | Tri-state verdict + optional/engine-strict dispatch | | `pacquet-package-is-installable::tests::known_failures` | 1 | The `>=X.Y.Z` vs `X.Y.Z-rc1` over-acceptance, picked up by `just known-failures` | | `pacquet_package_manager::installability::tests` | 11 | Per-install skip-set computation: skip on bad OS, skip on bad node engine, dedup events across peer variants, fast-path triggers, constraint predicate's edge cases (`engines.npm` only, `["any"]` sentinel, empty lists) | | `pacquet_package_manager::build_sequence::tests` | 3 (new) | Skipped+patched doesn't enter build queue; skipped parent doesn't drag descendants in; descendant with non-skipped parent still builds | All ported tests verified to catch regressions by temporarily breaking the subject under test, observing the failure, then reverting. The "test the tests" workflow from CLAUDE.md. ## Deferred to follow-up slices - `.modules.yaml.skipped` write/read + headless re-check (slice 3). - `supportedArchitectures` config + `--cpu` / `--os` / `--libc` CLI flags (slice 2). - `pnpm:install-check` warn channel on the reporter side (currently `tracing::warn!`). - Real libc detection — `host_libc()` returns `"unknown"` today; matches non-Linux host behavior, but on Alpine/musl this over-installs glibc-only optional packages. Slice 2. - `engine_strict` config wiring — defaults to false today, so the error path is unreachable from production. Wired through end-to-end so the slice that flips the config doesn't churn the error enum. |
||
|
Zoltan Kochan
|
429a9179c8 |
feat(real-hoist): crate skeleton + lockfile-to-HoisterTree wrapper (#438 slice 3a) (#448)
* feat(real-hoist): crate skeleton + lockfile-to-HoisterTree wrapper (#438 slice 3a) First sub-slice of the Slice 3 hoister port from the umbrella. Lands the IO surface and the pnpm-side wrapper; the inner `@yarnpkg/nm` algorithm is stubbed (returns the input tree unchanged) and replaced in 3b. New crate `pacquet-real-hoist`: - `HoisterTree` / `HoisterResult` / `HoisterDependencyKind` / `HoistingLimits` / `HoistOpts` / `HoistError`, mirroring upstream's `@yarnpkg/nm` types and the pnpm wrapper's option object. - `RcByPtr<T>` wrapper providing identity-hashed `Rc<T>` so children stored in `IndexSet` keep JS `Set<HoisterTree>` semantics (a node added via two parent paths stays shared) without paying the cost of structural hashing. - `hoist(&Lockfile, &HoistOpts)` — ports the wrapper at `installing/linking/real-hoist/src/index.ts`. Builds the HoisterTree from the lockfile (root importer's dependencies/devDependencies/optionalDependencies merged, then recursive descent through `snapshots`), adds workspace importer children, plus `link:` placeholders for `externalDependencies`, runs the inner stub, and post-filters `externalDependencies` out of the top-level result. - `LockfileMissingDependency` error surfaced as a `miette` diagnostic with code `ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY`, matching the upstream error code at https://pnpm.io/errors. Five unit tests pin the wrapper's observable behaviour: - The "broken lockfile" error case is a direct port of pnpm's `installing/linking/real-hoist/test/index.ts` test. - An empty lockfile yields an empty root. - A `root → a → b` lockfile, under the stub, gives `root → a → b` (nothing hoisted). Pinning this means 3b's algorithm replacement is observably correct — the tree shape will change to `root → {a, b}` once real hoisting lands. - A diamond dep (`root → {a, c} → b`) keeps `b` as one identity through both parents (proves the wrapper's dedup cache). - `external_dependencies` are stripped from the result. Upstream references pin `pnpm/pnpm@94240bc0` and `yarnpkg/berry@4287909fa6`. * docs(real-hoist): drop slice/umbrella scaffolding from code comments Slice and sub-slice numbering is PR-organisation scaffolding — it's useful in commit messages and PR descriptions but it rots inside committed code the moment the referenced slices land, get renumbered, or are abandoned. Rewrite the relevant doc-comments to describe what the code *is* today rather than where it sits in a multi-PR sequence: - Top-level module doc drops the "This is Sub-slice 3a of #438" intro. - `hoist` doc drops the "Sub-slice 3a pins / later sub-slices replace" framing. - `nm_hoist` stub doc drops "Sub-slice 3b replaces this". - `external_dependencies` / non-root importer comments drop their "umbrella's Slice 10 / Slice 9" pointers. - One test doc rewords "Sub-slice 3b's algorithm replacement is observably correct" into a description of what the test pins without naming the future slice. * fix(real-hoist): aliased snapshot lookup, cycle-safe rc identity, non_exhaustive HoistError (#448 review) Address four Copilot review findings on the new crate: 1. `collect_snapshot_deps` looked up `SnapshotDepRef::Alias` deps under the alias name instead of the resolved target name. The snapshot key for an npm-alias is `<target>@<ver>`, not `<alias>@<ver>`, so any real aliased transitive would have surfaced as `LockfileMissingDependency`. `build_dep_node` now takes the resolved `&PkgNameVerPeer` directly and the caller builds it via `dep_ref.resolve(alias)` for snapshot deps or `PkgNameVerPeer::new(alias, spec.version)` for importer deps. 2. `build_dep_node`'s cycle handling re-allocated the node on the way out (placeholder Rc, recurse, *new* finished Rc) which left any cycle-visiting Rc pointing at the empty placeholder. `HoisterTree::dependencies` is now a `RefCell<IndexSet<...>>` and the construction populates the cell in place — the placeholder and the populated node are the same allocation, so a back-edge reads the eventually-populated set. Same JS `Set<HoisterTree>` mutation semantics the real hoist algorithm needs. 3. `convert` had the same bug and gets the same fix on `HoisterResult::dependencies` and `HoisterResult::references`. The stub's traversal now collects the input children into a Vec before recursing so the borrow on the input cell drops, and populates the output cell in place. 4. `HoistError` is now `#[non_exhaustive]`, matching the rest of pacquet's public error enums. A new regression test (`transitive_npm_alias_resolves_target_snapshot`) pins the fix for (1): the snapshot key the wrapper looks up matches the target package, the node's exposed `name` stays the alias, and `ident_name` / `reference` carry the resolved target's identity. Existing tests updated to take `Ref<'_, …>` via `.borrow()`. All six pass; `just ready`, `cargo doc --document-private-items`, `taplo`, and `just dylint` clean. |
||
|
Zoltan Kochan
|
540fa7664a |
feat(git-fetcher): install git-hosted packages via git CLI + preparePackage (#436 §B+D) (#446)
Builds on §A (#440). Adds a new `pacquet-git-fetcher` crate that handles `LockfileResolution::Git` snapshots during frozen-lockfile installs, plus the supporting `preparePackage` port. The install dispatcher now routes git resolutions through it instead of returning `UnsupportedResolution { resolution_kind: "git" }`. ### New crate `pacquet-git-fetcher` - **`GitFetcher`** shells out to the system `git` (clone, or `init` + `fetch --depth 1` on hosts in `git_shallow_hosts`), checks out the pinned commit, verifies via `rev-parse HEAD`, runs `preparePackage`, removes `.git`, computes a packlist, and imports the resulting file set into the CAS. Surfaces a friendly `GitNotFound` when `git` isn't on `PATH` — pacquet does not bundle it. Mirrors [`fetching/git-fetcher/src/index.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/fetching/git-fetcher/src/index.ts). - **`prepare_package`** ports [`exec/prepare-package/src/index.ts`](https://github.com/pnpm/pnpm/blob/94240bc046/exec/prepare-package/src/index.ts). Reads the manifest, decides via `package_should_be_built`, honors an `AllowBuildFn` (the dispatcher adapts pacquet's `AllowBuildPolicy` into this shape), runs `<pm>-install` plus any `prepublish` / `prepack` / `publish` scripts via the existing `pacquet_executor::run_lifecycle_hook`, then deletes `node_modules`. Emits the upstream error codes `GIT_DEP_PREPARE_NOT_ALLOWED`, `ERR_PNPM_PREPARE_PACKAGE`, `INVALID_PATH`. - **`detect_preferred_pm`** sniffs the cloned tree for a lockfile to pick the install pm. Workspace-root walking deferred. - **Minimal `packlist`** honors the manifest's `files` field with a tiny `*` / `**` / `?` glob matcher (`?` and single `*` reject `/`; `**` matches across directories), always-includes the README / LICENSE / package.json set, and always-excludes `.git`, `node_modules`, lockfiles for sibling pms, and common cruft. Full `.npmignore` / `.gitignore` semantics and `bundleDependencies` walking are deferred and tracked in module docs. ### Wiring in `pacquet-package-manager` - `InstallPackageBySnapshot` now matches `LockfileResolution::Git` and runs the fetcher in place of `DownloadTarballToStore`. - `&AllowBuildPolicy` is computed once per install in `InstallFrozenLockfile` and threaded through `CreateVirtualStore` → `InstallPackageBySnapshot`. - `snapshot_cache_key` returns `Ok(None)` for `Git` resolutions so they cold-batch. Warm prefetch for git-hosted slots is a follow-up alongside §C. ### Supporting changes - `GitResolution.path: Option<String>` so a lockfile pinning a sub-directory of a git-hosted monorepo deserializes without tripping `deny_unknown_fields`. - `Config::git_shallow_hosts` with pnpm's default list + `pnpm-workspace.yaml` override. - `pacquet_executor::run_lifecycle_hook` is now `pub` so the preparePackage port can dispatch by arbitrary stage name. ## Out of scope (follow-ups for #436) - §C — git-hosted *tarball* fetcher (the `gitHosted: true` shape). - §E — full integration-test matrix (`plans/TEST_PORTING.md` 563-572). This PR ships crate-level tests against a local bare repo that prove the path works end-to-end. - Warm prefetch for git resolutions (re-clones on every install today — slow but correct). - Full `npm-packlist` semantics (`.npmignore`, gitignore layering, `bundleDependencies` walking). |
||
|
Zoltan Kochan
|
725c51e57e |
feat: partial install with --frozen-lockfile (#433) (#442)
## Summary Closes #433. Implements partial install with `--frozen-lockfile`: - **Read** `<virtual_store_dir>/lock.yaml` at install start (`Lockfile::load_current_from_virtual_store_dir`), mirroring upstream pnpm v11's `readCurrentLockfile`. - **Skip** any snapshot whose `dependencies`, `optionalDependencies`, and `resolution.integrity` match the current lockfile **and** whose virtual-store slot exists on disk. No fetch, no extract, no relink for skipped snapshots. - **Write** `<virtual_store_dir>/lock.yaml` at end-of-install (atomically; deletes the file when the lockfile is empty), so the next install sees a fresh diff target. - **Emit** `pnpm:_broken_node_modules` (`BrokenModulesLog`) when the slot is gone despite the cache key matching, then fall through to a full install for that snapshot. - `pnpm:context.currentLockfileExists` now reflects reality (was hard-coded `false`); `pnpm:stats.added` reports the post-skip delta instead of the total snapshot count, matching upstream. - **Side-effects-cache prefetch still covers skipped snapshots** so `BuildModules`'s `is_built` gate fires on warm reinstalls — without this, packages with `allowBuilds: true` re-execute their lifecycle scripts every install (added in 1603f94f after benchmarking caught the regression — see [benchmark comment](https://github.com/pnpm/pacquet/pull/442#issuecomment-4440371477)). Tracks pnpm v11 `94240bc046` for `readCurrentLockfile` / `writeCurrentLockfile` / `lockfileToDepGraph`'s per-`depPath` skip and `isIntegrityEqual` helper. ## Performance Two scenarios benchmarked against a 1352-package fixture, hyperfine `--warmup 2 --runs 10`. **Pure warm reinstall** (every snapshot intact, full `lock.yaml` on disk): | | mean | sys time | |---|---:|---:| | baseline (`main` HEAD~2) | 969 ms | 985 ms | | **PR** | **896 ms** | **349 ms** | PR is 1.08× faster, with ~65% less kernel work — the skip filter bypasses the warm-batch's per-snapshot syscalls (1352 → 0). [Full breakdown + the regression we caught + fixed.](https://github.com/pnpm/pacquet/pull/442#issuecomment-4440371477) **Non-up-to-date `node_modules`** (N virtual-store slots deleted per iteration): | Damaged slots | PR | Baseline | PR/Baseline | |---:|---:|---:|---:| | 0 | 581 ms | 661 ms | **1.14×** | | 1 | 615 ms | 707 ms | **1.15×** | | 10 | 660 ms | 707 ms | 1.07× | | 100 | 2.25 s | 2.25 s | 1.00× | | 500 | 6.13 s | 7.93 s | **1.29×** | PR is consistently a small win and never a loss across the damage spectrum; `_broken_node_modules` debug events fire correctly for each missing slot. [Full sweep + caveats.](https://github.com/pnpm/pacquet/pull/442#issuecomment-4440646688) |
||
|
Zoltan Kochan
|
2f146d838c |
feat(package-manager): force + keep_modules_dir on package import (#438 slice 1) (#441)
Extends pacquet's package-import primitive to cover pnpm's
`importIndexedDir(..., { force, keepModulesDir })` call shape so the
hoisted node-linker (the umbrella of #438) has a primitive to build on.
- Renames `create_cas_files` → `import_indexed_dir` to match
pnpm's name (its docstring already said it mirrored
`tryImportIndexedDir`), and adds an `ImportIndexedDirOpts` struct
with `force` and `keep_modules_dir`. Both linkers now go through
the same function — pnpm uses one function for both, so does
pacquet.
- **Default opts** (`force: false, keep_modules_dir: false`): the
isolated linker's behavior. Existing target short-circuits; fresh
target gets `mkdir` + parallel `link_file`. Unchanged hot path
for the two existing call sites
(`create_virtual_dir_by_snapshot`, `install_package_from_registry`).
- **`force: true`**: stage the new contents in a sibling directory,
remove the old tree, rename stage into place. A regular file or
symlink occupying the target is unlinked first.
- **`force: true` + `keep_modules_dir: true`**: before the swap,
`dir_path/node_modules/` is moved into the staging directory so
nested deps survive the rebuild. On any failure after the move,
the staged copy is restored to `dir_path/node_modules/` before
staging is cleaned up — staging never holds the user's only copy
of nested deps. This is the call shape the hoisted linker will
use.
- No install-pipeline wiring for hoisted yet. Subsequent slices
(`linkHoistedModules` analog, hoist algorithm, `Install::run`
branch on `node_linker`) will pass `force + keep_modules_dir`
from the new hoisted path. The default-opts behavior is exercised
in production today via the existing isolated callers.
## Upstream reference
Ported from `pnpm/pnpm@94240bc0`:
- [`fs/indexed-pkg-importer/src/importIndexedDir.ts`](
|
||
|
Zoltan Kochan
|
00d13515f2 |
feat: global-virtual-store foundation (#432 partial) (#444)
Pacquet port of upstream pnpm's global-virtual-store (GVS) machinery — lands the config fields, the store-dir helpers, the graph-hasher path math, and a `VirtualStoreLayout` helper, but **not** the install-pipeline wiring. Carves the work in #432 along its Section A/B/C/D boundary so the foundation can land cleanly before the larger install refactor follows. ### What's in this PR **Section A — Config** - `Config::enable_global_virtual_store` (default `true`, matching upstream's [`config/reader/src/index.ts:392-394`](https://github.com/pnpm/pnpm/blob/94240bc046/config/reader/src/index.ts#L392-L394)) - `Config::global_virtual_store_dir` - `pnpm-workspace.yaml` deserialization for `enableGlobalVirtualStore` / `globalVirtualStoreDir` - `Config::apply_global_virtual_store_derivation` mirroring upstream's [`extendInstallOptions.ts:343-355`](https://github.com/pnpm/pnpm/blob/94240bc046/installing/deps-installer/src/install/extendInstallOptions.ts#L343-L355). **Not** called from `Config::current` yet — the install layer in the follow-up will invoke it explicitly so this PR doesn't silently redirect installs to `<store_dir>/links` before the rest of the layout math is wired. **Section C (foundation only) — Project registry helpers** - `StoreDir::links()`, `StoreDir::projects()`, `StoreDir::root()` - `pacquet_store_dir::register_project` (port of upstream's [`registerProject`](https://github.com/pnpm/pnpm/blob/94240bc046/store/controller/src/storeController/projectRegistry.ts)) - `pacquet_store_dir::create_short_hash` (port of [`createShortHash`](https://github.com/pnpm/pnpm/blob/94240bc046/crypto/hash/src/index.ts)) — pinned test vector against `printf pacquet | shasum -a 256 | head -c 32` **Section B (foundation only) — Graph-hasher + layout helpers** - `pacquet_graph_hasher::calc_graph_node_hash` (port of [`calcGraphNodeHash`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-hasher/src/index.ts#L122-L146)) - `pacquet_graph_hasher::format_global_virtual_store_path` (port of [`formatGlobalVirtualStorePath`](https://github.com/pnpm/pnpm/blob/94240bc046/deps/graph-hasher/src/index.ts#L155-L160)) - `pacquet_package_manager::VirtualStoreLayout` precomputes each snapshot's slot directory on disk and hides the GVS-vs-legacy path divergence behind one `slot_dir(&PackageKey) -> PathBuf` lookup ### What's deferred to the follow-up PR - Wiring `VirtualStoreLayout` through `CreateVirtualStore`, `CreateVirtualDirBySnapshot`, `SymlinkDirectDependencies`, `InstallPackageBySnapshot`, `BuildModules`, `link_bins`, and `create_symlink_layout` - Calling `register_project` from `Install::run` - Calling `apply_global_virtual_store_derivation` from `Config::current` - Updating ~20 existing test setups to opt out of the (now-default) GVS mode and porting `installing/deps-installer/test/install/globalVirtualStore.ts` ### No behavior change at install time Every new field, helper, and module is unreachable from the existing install path. `just ready` is green; new unit tests added for each helper: - `pacquet-config`: GVS derivation truth table (default re-points / disabled / user-pinned) - `pacquet-store-dir`: short-hash pinned vector + register idempotency + isSubdir guard - `pacquet-graph-hasher`: GVS hash determinism + engine sensitivity + child sensitivity + unscoped `@/` prefix - `pacquet-package-manager`: `VirtualStoreLayout` slot_dir under GVS-on and GVS-off Closes part of #432; #432 stays open until the install-pipeline wiring + test ports land. |
||
|
Zoltan Kochan
|
fe8cf4d0ee |
feat(lockfile,store-dir): gitHosted on TarballResolution + pickStoreIndexKey (#436 §A) (#440)
Foundation for git-hosted package install (#436 Section A). Three small, self-contained changes: - **`TarballResolution.gitHosted: Option<bool>`** in `crates/lockfile`. Recent pnpm lockfiles ship this field on tarballs sourced from a git host. Without it, pacquet's `deny_unknown_fields` strictness rejected real lockfiles before the install dispatcher ran — a silent correctness gap independent of the rest of #436. Mirrors upstream's [`TarballResolution.gitHosted`](https://github.com/pnpm/pnpm/blob/94240bc046/lockfile/types/src/index.ts#L88-L107). - **Back-fill on read.** When `gitHosted` is absent but the tarball URL prefix-matches a known git host (`codeload.github.com`, `gitlab.com`, `bitbucket.org`) with a `tar.gz` substring, set it to `true` during deserialize. Mirrors upstream's [`enrichGitHostedFlag`](https://github.com/pnpm/pnpm/blob/94240bc046/lockfile/fs/src/lockfileFormatConverters.ts#L158-L168) so every downstream reader can rely on the typed field instead of matching URL prefixes at each site. - **`git_hosted_store_index_key` + `pick_store_index_key`** helpers in `pacquet-store-dir`. Git-hosted cache content depends on whether the build ran during fetch, so the cache key carries a `built` / `not-built` dimension that the integrity-only key would collapse. Mirrors upstream's [`gitHostedStoreIndexKey` / `pickStoreIndexKey`](https://github.com/pnpm/pnpm/blob/94240bc046/store/index/src/index.ts#L60-L94). No call sites switch yet — the dispatcher continues to error with `UnsupportedResolution { resolution_kind: "git" }` until the git fetcher lands. |
||
|
Zoltan Kochan
|
c4fce797b6 |
feat(config): POSIX root auto-detect for unsafePerm (#397 §14 follow-up) (#430)
* feat(config): POSIX root auto-detect for unsafePerm (#397 §14) Completes the deferred half of #397 item 14. Adds `libc` to workspace deps (Unix-only at the crate level via `[target.'cfg(unix)'.dependencies]`) and implements `default_unsafe_perm()` mirroring upstream's [`extendBuildOptions.ts:83-86`](https://github.com/pnpm/pnpm/blob/94240bc046/building/after-install/src/extendBuildOptions.ts#L83-L86): ```ts unsafePerm: process.platform === 'win32' || process.platform === 'cygwin' || !process.setgid || process.getuid?.() !== 0 ``` Pacquet equivalent: - Windows → `true` (cfg!(windows) short-circuits; covers Cygwin too because Cygwin builds target windows-gnu, so the `'cygwin'` branch from upstream is implicitly handled). - POSIX, running as root → `false` (drop privileges). - POSIX, non-root → `true`. The `!process.setgid` branch upstream is a Node-version compatibility check for older Node where `setgid` doesn't exist; it doesn't translate to Rust (libc's `setgid` is always available on POSIX hosts where libc compiles). `Config.unsafe_perm`'s default switches from a hard-coded `true` to `default_unsafe_perm()`. Yaml override and the Windows force-override in `WorkspaceSettings::apply_to` stay as they were. `is_unsafe_perm_posix(uid)` is exposed for tests so the per-uid logic can be exercised without root privileges. Tests: new `is_unsafe_perm_posix_truth_table` (root vs non-root uid table); `default_unsafe_perm_on_posix_matches_runtime_uid` (POSIX-gated); `default_unsafe_perm_on_windows_is_always_true` (Windows-gated). The existing `parses_unsafe_perm_from_yaml_and_applies` drops the explicit "default is true" assertion since the default now depends on the runtime uid — the test's purpose (yaml-`false` wins on POSIX) is preserved. * fix(config): address PR #430 review (Cygwin + non-POSIX + unsafe fn) Four review-flagged issues from PR #430: 1. **Cygwin handling.** Rust's `x86_64-pc-cygwin` target emits `target_os = "cygwin"` with `cfg!(unix)` set and `cfg!(windows)` *unset*. The previous `cfg!(windows)` check fell through to the uid logic on Cygwin, which would flip to `false` when running as root — diverging from upstream's unconditional-true Cygwin branch. Added explicit `target_os = "cygwin"` matching alongside `windows`. 2. **Non-Unix, non-Windows targets.** The previous `posix_getuid` stub returned `0` on non-Unix targets, which made `default_unsafe_perm` return `false` (root-treating). Replaced with three cfg-gated `platform_unsafe_perm_default` overloads: Windows/Cygwin → `true`; POSIX-excluding-Cygwin → uid-based; anything else (`wasm32-*`, `redox`, etc.) → `true`. 3. **Over-marked `unsafe fn posix_getuid`.** Copilot was right that the safety comment described no per-call invariants, only the FFI boundary. Switched to a safe `fn` with the `unsafe` block contained inside, eliminating the unnecessary `unsafe` propagation to callers. 4. **Stale doc claim about Cygwin.** The doc said Cygwin builds target `windows-gnu` and that `cfg!(windows)` covers it. Neither is correct — `x86_64-pc-cygwin` is a distinct target. Updated the doc to describe the explicit `target_os = "cygwin"` branch with a link to rustc's platform-support page. Cargo dep gate also narrowed: `[target.'cfg(all(unix, not(target_os = "cygwin")))'.dependencies]` so libc isn't built on Cygwin (where the POSIX branch isn't taken). The `default_unsafe_perm_on_posix_matches_runtime_uid` test was correspondingly narrowed to the same cfg, and a new `default_unsafe_perm_on_cygwin_is_always_true` test pins the Cygwin short-circuit. |
||
|
Zoltan Kochan
|
2291bc6c2e |
feat: build concurrency, unsafe-perm, scriptsPrependNodePath (#397 items 12, 14, 15) (#429)
Bundles the three remaining moderate/minor parity items from #397 into one PR. Each adds a `pnpm-workspace.yaml` setting, threads it through `Config`, and applies it to `BuildModules` so a pnpm yaml that exercises these knobs produces the same install behavior in pacquet. - **Item 12 — `childConcurrency`** ([upstream `getWorkspaceConcurrency`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/config/reader/src/concurrency.ts#L25-L34); [`runGroups(...)`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L124)). Adds `Config.child_concurrency: u32` with the upstream-matching default `min(4, availableParallelism())` and the negative-offset semantics (`n < 0` → `max(1, parallelism - |n|)`). `BuildModules::run` now dispatches each chunk's members across a bounded rayon thread pool via `par_iter().try_for_each` — chunks themselves remain sequential to preserve topological order. `ignored_builds` and `deps_state_cache` are wrapped in `Mutex` so the recursive memo and the dedup set survive concurrent chunk-member access. - **Item 14 — `unsafePerm`**. Adds `Config.unsafe_perm: bool` (default `true`) and threads it through to `RunPostinstallHooks`. When `false`, the executor sets `TMPDIR=node_modules/.tmp` for the spawn. The uid/gid drop side is a no-op upstream too because pnpm's [`@pnpm/npm-lifecycle`](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/index.js#L204-L220) never populates `opts.user` / `opts.group`. The POSIX auto-detect (`getuid() === 0 && setgid → unsafePerm = false`) needs `libc`, which isn't in `[workspace.dependencies]` yet — for now, root-run CI must set `unsafePerm: false` in yaml explicitly. Windows is force-overridden to `true` in `WorkspaceSettings::apply_to`, matching upstream's `process.platform === 'win32'` gate. - **Item 15 — `scriptsPrependNodePath`** ([`Config.scriptsPrependNodePath`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/config/reader/src/Config.ts#L108)). Adds a tri-state `ScriptsPrependNodePath` enum in `pacquet-config` (custom serde `Deserialize` for the `boolean | "warn-only"` yaml shape) and converts to `pacquet_executor::ScriptsPrependNodePath` at the `BuildModules` call site so the executor crate stays free of serde wiring. Default is `Never` to match upstream's [`StrictBuildOptions.scriptsPrependNodePath: false`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/after-install/src/extendBuildOptions.ts#L78). - **Item 16 — `getSubgraphToBuild` filter trimming**. Already implemented in pacquet's `get_subgraph_to_build` via the `child_should_be_built || needs_build || has_patch` gate. Confirmed during this slice; no code change needed. Why one PR: items 12 / 14 / 15 are three independent yaml-driven config knobs, all small. Bundling keeps the cross-crate signature churn from happening three times (`BuildModules`'s field list, `install_frozen_lockfile`'s call site, and the test fixtures all touch each one), and the upstream sources for the three settings live next to each other so the porting context is shared. |
||
|
Zoltan Kochan
|
f9b14b15ec |
feat(package-manager): version unions in allowBuilds keys (#397 item 5 completion) (#428)
Completes pacquet#397 item 5. Slice A+B's PR #425 moved `AllowBuildPolicy` off `package.json` onto `pnpm-workspace.yaml`; this commit ports the second half of upstream's matcher — the `expandPackageVersionSpecs` step that lets users write keys like `foo@1.0.0 || 2.0.0` in their `allowBuilds` map. ## What this adds New `pacquet_package_manager::version_policy` module porting [`config/version-policy/src/index.ts`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/config/version-policy/src/index.ts) at commit `b4f8f47ac2`. `expand_package_version_specs` parses each `allowBuilds` key into one or more `name` / `name@version` literal strings: - `foo` → `{"foo"}` - `foo@1.0.0` → `{"foo@1.0.0"}` - `foo@1.0.0 || 2.0.0` → `{"foo@1.0.0", "foo@2.0.0"}` - `@scope/foo@1.0.0` → `{"@scope/foo@1.0.0"}` Two error codes mirror upstream: - `ERR_PNPM_INVALID_VERSION_UNION` when a `||` member isn't valid semver - `ERR_PNPM_NAME_PATTERN_IN_VERSION_UNION` when a `*` wildcard in the name is combined with a version part Whitespace around `||` and within each version is trimmed before parsing, matching Node-semver's `valid()`. ## What this does NOT add Wildcards in the name (`is-*`, `@scope/*`) are accepted by the parser and land in the expanded set as literal strings, but `HashSet::contains` lookups mean they never match real package names. Mirrors upstream's `'should not allow patterns in allowBuilds'` test at [`building/policy/test/index.ts:28-34`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/policy/test/index.ts#L28-L34) — the original #397 audit incorrectly claimed `@scope/*` should work in `allowBuilds`; it doesn't, neither upstream nor here. `createPackageVersionPolicy` (which DOES support wildcards via `Matcher`) is a separate upstream function used by `minimumReleaseAgeExclude` / `dlx` — pacquet doesn't have those features yet. ## AllowBuildPolicy refactor `AllowBuildPolicy`'s internal storage changes from `HashMap<String, bool>` to two `HashSet<String>` (`expanded_allowed` and `expanded_disallowed`), populated through `expand_package_version_specs`. The `check` function checks `disallowed` before `allowed`, both against bare `name` and `name@version`, mirroring upstream's order at [`building/policy/src/index.ts:35-44`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/policy/src/index.ts#L35-L44). This fixes a pre-existing pacquet divergence: the old matcher checked exact-version first, then bare name. With upstream's order, a bare-name disallow now correctly wins over an exact-version allow. New `disallow_bare_name_wins_over_allow_exact_version` test pins the behavior; the old `exact_version_takes_precedence` test was removed (it asserted the divergent behavior). `AllowBuildPolicy::new` now takes the expanded sets directly (pure constructor — no IO). `AllowBuildPolicy::from_config` returns `Result<Self, VersionPolicyError>` so spec-parse failures surface at install time instead of being silently dropped. New `InstallFrozenLockfileError::VersionPolicy` variant propagates the error. ## Tests - 12 new tests in `version_policy::tests` covering: bare name, exact version, version union, scoped names, whitespace trimming in unions, name-with-wildcard-alone (literal), invalid version union (error), mixed valid/invalid (error), wildcard-with-version (error), empty input, duplicate collapse. - Ports of upstream's `building/policy/test/index.ts` cases: `allow_via_version_union` (version-union allows), `wildcard_name_in_allow_builds_does_not_match_real_package` (wildcards inert), `disallow_bare_name_wins_over_allow_exact_version` (matcher order), `disallow_exact_version_with_allow_bare_name` (converse). - `from_config` error-propagation tests for both `VersionPolicyError` variants. 598 tests pass; `just ready`, `just dylint`, `cargo doc -D warnings --document-private-items`, `taplo format --check` all clean. |
||
|
Zoltan Kochan
|
13082fc1bf |
feat: apply patchedDependencies before postinstall (#397 item 9) (#427)
Completes pnpm/pacquet#397 item 9 (`patchedDependencies` support). Patches now actually land on disk: pacquet applies configured patches to extracted package directories before postinstall hooks run, includes patched-only packages in the build trigger, and surfaces the four upstream diagnostic codes. Slices A + B (PR #425, merged) landed the foundation, Config plumbing, and the side-effects-cache key wiring. This PR wires the rest. ## Pure-Rust patch applier `pacquet_patching::apply_patch_to_dir` ports upstream's [`applyPatchToDir`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/patching/apply-patch/src/index.ts) using the [`diffy`](https://crates.io/crates/diffy) crate (MIT OR Apache-2.0, pure Rust, no subprocess). Upstream's own comment notes that `patch` is unavailable on Windows and `git apply` is awkward inside an existing git repo — pnpm vendored `@pnpm/patch-package` for that reason; pacquet sidesteps it the same way with an in-process applier. Supported `FileOperation`s: `Modify`, `Create`, `Delete`. `Rename` and `Copy` fall through to `ERR_PNPM_PATCH_FAILED` with a descriptive message — they don't appear in `patch-package`-style patches in practice. ## Build pipeline - `build_sequence::get_subgraph_to_build` now considers `patch.is_some()` alongside `requires_build`. A patched-only package becomes a build candidate the same way upstream's filter at [`buildSequence.ts:47,60`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/buildSequence.ts#L40-L67) treats them. - `BuildModules::run` per-snapshot loop refactored to mirror upstream's flow: - Inner build trigger becomes `requires_build || patch.is_some()`. - `allowBuilds` check only gates scripts (upstream's `if (node.requiresBuild)` at [`during-install/src/index.ts:88-101`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L82-L106)). Disallow / ignored sets `should_run_scripts = false` rather than `continue` — patches still apply. - `cache_key`'s `include_dep_graph_hash` is now `should_run_scripts` (was unconditionally `true`), so a patched-only snapshot's cache key omits the deps-hash, matching upstream's `includeDepGraphHash: hasSideEffects` at [line 202](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L199-L204). - Patch application happens before `run_postinstall_hooks`, mirroring [`during-install/src/index.ts:171-178`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L168-L191). - Cache-write gate becomes `(is_patched || has_side_effects) && side_effects_cache_write` (was just `side_effects_cache_write`), matching upstream's `(isPatched || hasSideEffects)` at line 199. ## Diagnostics Four upstream codes surface via `BuildModulesError`: - `ERR_PNPM_PATCH_NOT_FOUND` — patch file missing on disk - `ERR_PNPM_INVALID_PATCH` — unified-diff parse error - `ERR_PNPM_PATCH_FAILED` — hunk wouldn't apply, target missing, IO error on a target path - `ERR_PNPM_PATCH_FILE_PATH_MISSING` — resolved patch entry has a hash but no path (lockfile-only shape with no live config); mirrors [`during-install/src/index.ts:172-176`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L172-L176) ## Dependency New workspace dep: `diffy = "0.5.0"`. MIT OR Apache-2.0, both in the deny.toml allow-list. `cargo deny check licenses` ok. |
||
|
Zoltan Kochan
|
f2d41415af |
feat: patchedDependencies foundation + cache key + AllowBuildPolicy from Config (#397 items 9 A+B, 5) (#425)
Combined slices A and B of pnpm/pacquet#397 item 9 (`patchedDependencies` support), plus the related #397 item 5 fix (`AllowBuildPolicy` moves off `package.json`). This PR adds the foundation (`pacquet-patching` crate) and threads patches as far as the side-effects-cache key. The build-trigger update and the actual patch application are deferred to slice C — they only make sense paired together. ## Foundation (`pacquet-patching` crate) Ports `@pnpm/patching.types` + `@pnpm/patching.config` + `calcPatchHashes` at pnpm SHA [`b4f8f47ac2`](https://github.com/pnpm/pnpm/tree/b4f8f47ac2): - **Types** (`types.rs`) — `PatchInfo`, `ExtendedPatchInfo`, `PatchGroupRangeItem`, `PatchGroup`, `PatchGroupRecord` mirroring [`patching/types/src/index.ts`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/patching/types/src/index.ts). - **Key parser** (`key.rs`) — minimal port of [`dp.parse`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/deps/path/src/index.ts#L120-L168). - **Grouping** (`group.rs`) — `group_patched_dependencies` mirroring [`groupPatchedDependencies.ts`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/patching/config/src/groupPatchedDependencies.ts), with `ERR_PNPM_PATCH_NON_SEMVER_RANGE`. - **Matcher** (`get_patch_info.rs`) — `get_patch_info` (exact → unique-range → wildcard) mirroring [`getPatchInfo.ts`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/patching/config/src/getPatchInfo.ts), with `ERR_PNPM_PATCH_KEY_CONFLICT`. - **Verify** (`verify.rs`) — `all_patch_keys` + `verify_patches` with `ERR_PNPM_UNUSED_PATCH`. - **Hashing** (`hash.rs`) — SHA-256 hex with CRLF→LF normalization (matters for cross-platform stability of the cache key). - **Resolve helper** (`resolve.rs`) — `resolve_and_group(workspace_dir, raw_map)` resolves relative patch paths against the workspace dir, hashes, and groups. ## Config plumbing (workspace yaml is the source of truth) pnpm v11 reads install settings from `pnpm-workspace.yaml`, not `package.json`. This PR matches that for `patchedDependencies` and (as the #397 item 5 fix) `allowBuilds` / `dangerouslyAllowAllBuilds` too. The misleadingly-named upstream [`getOptionsFromRootManifest.ts`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/config/reader/src/getOptionsFromRootManifest.ts) is called at [`config/reader/src/index.ts:814`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/config/reader/src/index.ts#L814) with the *workspace* manifest. - `Config.workspace_dir: Option<PathBuf>` — recorded by `WorkspaceSettings::apply_to` so install-time code can resolve patch file paths against the same dir upstream uses. - `Config.patched_dependencies: Option<BTreeMap<String, String>>` — raw map. Hashing + resolution deferred to `Config::resolved_patched_dependencies` so `apply_to` stays infallible. - `Config.allow_builds: HashMap<String, bool>` and `Config.dangerously_allow_all_builds: bool` — replace `AllowBuildPolicy::from_manifest`'s `package.json` reads. - `WorkspaceSettings` deserializes the matching three yaml keys (`patchedDependencies`, `allowBuilds`, `dangerouslyAllowAllBuilds`); `apply_to` pushes them. ## Install-pipeline threading - `InstallFrozenLockfile::run` calls `config.resolved_patched_dependencies()` once per install, then for each snapshot calls `pacquet_patching::get_patch_info(&groups, name, version)` and collects matches into `HashMap<PackageKey, ExtendedPatchInfo>` keyed by the peer-stripped key. Mirrors upstream's per-resolution lookup at [`resolveDependencies.ts:1482`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/pkg-manager/resolve-dependencies/src/resolveDependencies.ts#L1482), adapted for pacquet's lockfile-driven flow. - `BuildModules` gains a `patches: Option<&HashMap<PackageKey, ExtendedPatchInfo>>` field. The `calc_dep_state` call at the existing `build_modules.rs:326` placeholder now passes `patch_file_hash: patch.map(|p| p.hash.as_str())`, matching upstream's [`patchFileHash: depNode.patch?.hash`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L201). ## `AllowBuildPolicy` (#397 item 5) - Drop `AllowBuildPolicy::from_manifest` (the `package.json` reader was a workaround until Config plumbing landed). - Add `AllowBuildPolicy::from_config(&Config)` consuming the new yaml-sourced fields. - `install_frozen_lockfile.rs:156` switches over. ## What's NOT in this PR Tracked for slice C: - **Build trigger update** (`requires_build || patch.is_some()`) and **patch application** to extracted package directories before postinstall hooks. These belong together — wiring the trigger in isolation produces an incoherent state (the cache key advertises a patch that isn't on disk). - Decision on the patch applier (Rust crate vs `git apply` subprocess) will be raised when slice C opens. - Diagnostics: `ERR_PNPM_PATCH_FILE_PATH_MISSING` / `PATCH_NOT_FOUND` / `INVALID_PATCH` / `PATCH_FAILED`. - End-to-end test parity with upstream's [`simple-with-patch`](https://github.com/pnpm/pnpm/tree/b4f8f47ac2/pkg-manager/headless/test/fixtures/simple-with-patch) fixture. |
||
|
Zoltan Kochan
|
f80275abbd |
docs(agents): require confirming pnpm/pnpm main is up to date before reading (#426)
Local clones drift, so porting from a stale checkout silently bakes in old upstream behavior. Tell agents to fetch (or use GitHub's `/blob/main` view) before consulting upstream, and clarify this doesn't conflict with the permalink-when-citing rule. |
||
|
Zoltan Kochan
|
57174b869f |
feat: side-effects cache WRITE path (#421 slice B) (#424)
## Summary Implements slice (B) of #421 — the WRITE path for pnpm's side-effects cache. With this PR, a successful postinstall now re-CAFSes the built package directory, diffs against the pristine `PackageFilesIndex.files` row, and mutates the row's `side_effects` map so a future install (pacquet *or* pnpm) can skip the rebuild via the READ path landed in #423. User-visible behavior: lifecycle scripts that produce build artifacts (`node-gyp` outputs, generated source files, etc.) get cached the first time the install runs; subsequent installs against the same lockfile + Node major skip the script and reuse the cached overlay. ## What lands - **`pacquet_store_dir::add_files_from_dir`** — directory walker that re-CAFSes a post-build package directory. Mirrors pnpm/pnpm@`7e3145f9fc`:`store/cafs/src/addFilesFromDir.ts:16-194` + `worker/src/start.ts:312-383`. Containment: `dunce::canonicalize` each symlink target, drop if out of root; read/recurse via the resolved canonical path so retargeting a symlink between containment check and read can't smuggle out-of-root data into CAFS. Skips top-level `node_modules` (matches upstream `includeNodeModules: false`). Cycle-safe via a recursion-stack `visited: HashSet<PathBuf>` (entries inserted on descend, removed on return). - **`pacquet_store_dir::upload` + `calculate_diff`** — `upload(store_dir, built_pkg_location, files_index_file, cache_key, writer)` walks the package via `add_files_from_dir` and queues a `WriteMsg::SideEffectsUpload { key, cache_key, current_files }` to the writer task. The actual R/M/W of the existing `PackageFilesIndex` row happens inside the writer task (see below), so concurrent uploads to the same row stay commutative. - **`StoreIndexWriter` now drives a `WriteMsg` enum** (`Replace` vs `SideEffectsUpload`). The batch loop coalesces per-key updates into one `PackageFilesIndex` value before flushing — multiple side-effects uploads for the same row apply in arrival order against a single in-memory state instead of racing against a stale read on a separate readonly handle. The R/M/W (load existing row → check `algo == HASH_ALGORITHM = "sha512"` → `calculate_diff` → insert into `side_effects` map) happens inside the writer's transaction. Loading short-circuits with a `debug!` / `warn!` when the base row is missing or unreadable, matching upstream's `if (!existingFilesIndex) return` / `ALGO_MISMATCH` bail-outs at `pnpm/pnpm@7e3145f9fc:worker/src/start.ts:342-371`. - **`Config.side_effects_cache_readonly: bool`** (default `false`) — mirrors pnpm's `sideEffectsCacheReadonly`. Two derived helpers consolidate the gate semantics so precedence stays single-sourced: - `Config::side_effects_cache_read()` = `cache || readonly` - `Config::side_effects_cache_write()` = `cache && !readonly` - **`BuildModules` WRITE-path call site** — after `run_postinstall_hooks` returns `Ok`, if `side_effects_cache_write` is on and `StoreIndexWriter` + `StoreDir` are both threaded in, calls `pacquet_store_dir::upload(...)` with the snapshot's `store_index_key(integrity, pkg_id)` and the `calc_dep_state` cache key. All upload errors are swallowed with `tracing::warn!`, mirroring upstream's `try { upload } catch { logger.warn }` at `pnpm/pnpm@7e3145f9fc:building/during-install/src/index.ts:208-215`. - **`build_deps_subgraph`** — the cache-hashing dep graph is now bounded to the forward closure of `requires_build` snapshots. Pure-JS installs feed in an empty root iterator and the function returns immediately; installs with buildable deps walk only what `calc_dep_state` will actually traverse. Upstream's `lockfileToDepGraph` builds the full graph because its consumers extend beyond cache hashing; pacquet's graph is consumed only by `calc_dep_state` today, so the closure-bounded walk produces byte-identical cache keys with strictly less work. Restored the cold-install benchmark to parity with `main`. - **`StoreIndexWriter` hoisted up to `InstallFrozenLockfile::run`** — previously spawned + drained inside `CreateVirtualStore::run`; now it spans both the prefetch/download phase and the build phase so the WRITE-path upload can queue through it. The final batch flushes after `BuildModules::run` returns. - **Byte-stable msgpack encoding** — `encode_package_files_index` now iterates `files`, `side_effects`, and `SideEffectsDiff.added` in sorted-key order so two pacquet writes of the same logical row produce identical bytes. Was prompted by a review on PR #424 pointing out that `HashMap` iteration randomization would otherwise make every row look "changed" on byte-diff even when content was identical. - **`cache_key` refactor in `BuildModules`** — `calc_dep_state` now fires once per snapshot (before the gate check) so both the READ gate and the new WRITE site read the same value. ## Tests - **`add_files_from_dir::tests`** (9 cases): top-level files, nested forward-slash paths, exec-bit propagation, top-level-vs-nested `node_modules`, symlink-out-of-root drop, symlink-in-root follow, directory-cycle termination, missing-root error. - **`upload::tests`** (6 cases): identical / added-only / deleted-only / digest change / mode change / mixed — covers every branch of upstream's `calculateDiff`. - **`deps_graph::tests`** (new for slice B): empty-roots → empty subgraph; forward-closure inclusion + exclusion; cycle termination. - **`workspace_yaml::tests`**: `sideEffectsCacheReadonly` parses + applies; the 4-state truth table for `side_effects_cache_read()` / `side_effects_cache_write()`. - **`build_modules::tests`** (new): - `write_path_populates_side_effects_row` — full WRITE-path: pre-seed a base row, run a postinstall that creates `generated.txt`, drain the writer task, assert `side_effects[cache_key].added` contains `generated.txt` and NOT pristine `index.js`. Mirrors the spirit of upstream's `'a postinstall script does not modify the original sources added to the store'` at `pnpm/pnpm@7e3145f9fc:sideEffects.ts:189-223`. - `write_path_disabled_skips_upload` — same scenario with `side_effects_cache_write: false` → row's `side_effects` stays `None`. Counterpart of upstream's gate on `sideEffectsCacheWrite`. - `upload_error_does_not_interrupt_install` — postinstall produces a 0-permission file in the package dir; the walker fails on `fs::read` with `EACCES`, surfacing as `UploadError::AddFilesFromDir(_)` which `BuildModules` swallows with `tracing::warn!`. Asserts (a) the postinstall-created `generated.txt` is on disk and (b) the pre-seeded base row's `side_effects` stays `None`. Mirrors `'uploading errors do not interrupt installation'` at `pnpm/pnpm@7e3145f9fc:sideEffects.ts:166-187`. ## Out of scope - **Patch-file hash in cache key** — depends on `patchedDependencies` (#397 item 9). Today's WRITE side stamps `patch_file_hash: None`; same as the existing READ side. - **`requires_build` recompute** in the R/M/W path. Upstream recomputes from `(manifest, filesMap)` when the existing row's field is `None`; pacquet leaves the field as-is (cold-batch downloads already populate it with a real value). - **GVS / hoisted-node-linker** variants of the upstream tests. |
||
|
Zoltan Kochan
|
03bb90a3fc |
feat(package-manager): is_built gate for the side-effects cache READ path (#421 slice A) (#423)
## Summary Implements slice (A) of #421 — the READ-path gate that consumes the foundation merged in #422. With a matching side-effects-cache entry on disk (typically seeded by a prior `pnpm install`), `BuildModules` now skips lifecycle scripts for that snapshot, mirroring upstream's [`!node.isBuilt` filter](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L73-L77). User-visible behavior: a warm `pacquet install` after `pnpm install` no longer rebuilds packages whose post-build state is already in the store. ## What lands - **`pacquet_graph_hasher::detect_node_major()`** — spawns `node --version` once at install start, parses the major version. The parse helper is factored out so the rule is unit-tested without a child process. Missing-node returns `None` and the gate falls through to "rebuild" (safe). - **`DepsGraphNode` flipped to own its strings** — was borrowed `&'a str`, now `String`. Lets callers building the graph from a lockfile skip a separate-arena lifetime story. No external consumers besides in-crate tests yet, so this is internal-only. - **`PrefetchResult.side_effects_maps`** surfaces the per-row `cache_key → FilesMap` overlay that the prefetch loop previously dropped (since #422 added it to `VerifyResult`). `Arc`-wrapped for the same cold-batch cheap-clone reason `cas_paths` is. - **`CreateVirtualStore` returns a new `CreateVirtualStoreOutput`** struct: `package_manifests` (existing) plus `side_effects_maps_by_snapshot: HashMap<PackageKey, Arc<…>>` (new). Peer-variants of the same package share one `Arc<…>` because the store-index row is keyed peer-stripped. - **New `pacquet_package_manager::deps_graph::build_deps_graph` adapter** — walks `snapshots` + `packages` and builds a `DepsGraph<PackageKey>` for the hasher. `full_pkg_id` derives from `<pkg_id>:<integrity>` for registry / tarball resolutions, or `<pkg_id>:<hashObject(resolution)>` for git / directory / integrity-less tarball, matching upstream's [`createFullPkgId`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/deps/graph-hasher/src/index.ts#L263-L292). - **`BuildModules` is_built gate** — after the policy gate, before extraction, the build loop computes `calc_dep_state` per snapshot (with `include_dep_graph_hash: true` mirroring upstream's `hasSideEffects`), looks the cache key up in the threaded-in `side_effects_maps`, and `continue`s on hit. The `Config.side_effects_cache: false` flag short-circuits the lookup entirely. - **`InstallFrozenLockfile`** detects the engine name once and threads it (plus the prefetch's side-effects map and the config flag) into `BuildModules`. ## Tests - `using_side_effects_cache_skips_rebuild` ports the upstream `'using side effects cache'` case from [`installing/deps-installer/test/install/sideEffects.ts:79`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/installing/deps-installer/test/install/sideEffects.ts#L79-L131). Hand-builds the `side_effects_maps_by_snapshot` entry with the exact cache key `BuildModules` will compute, asserts no `pnpm:lifecycle` event fires for `@pnpm.e2e/failing-postinstall`'s postinstall (i.e. the build was skipped — if the gate were broken the script would have run and the install would have failed with `LifecycleScript(_)`). - `side_effects_cache_disabled_bypasses_the_gate` (negative pair): same overlay shape with `side_effects_cache: false`, expects the build to run and the install to fail with the underlying `exit 1` from the postinstall. - `deps_graph::tests`: registry-resolution uses integrity verbatim, dependencies + optionalDependencies fold into children, snapshot without metadata is skipped. The upstream test runs the install twice — first to populate the cache via the WRITE path, then to consume it. Pacquet doesn't have a WRITE path yet (#421 slice B), so the port hand-crafts the same in-memory state directly. The behavior under test is identical. ## Out of scope - **WRITE path** — pacquet seeding the cache itself (re-CAFS post-build, calc diff, mutate index row). Tracked as slice (B) in #421. - **Patch-file hash in the cache key** — depends on `patchedDependencies` (#397 item 9). |
||
|
Zoltan Kochan
|
e164bdbf36 |
feat: foundation for the side-effects cache (#397 item #10, part 1) (#422)
Foundation for porting pnpm's side-effects cache (item #10 of #397). This PR lands the three pieces that the actual cache read/write paths depend on: - **New `pacquet-graph-hasher` crate** porting pnpm's `@pnpm/crypto.object-hasher` (`hashObject` / `hashObjectWithoutSorting`) plus `@pnpm/deps.graph-hasher` (`calcDepState` / `calcDepGraphHash`) and `ENGINE_NAME`. Byte-for-byte parity with the JS [`object-hash@3.0.0`](https://github.com/puleos/object-hash/blob/v3.0.0/index.js) bytestream format is load-bearing — the cache key is persisted on disk and shared with pnpm. The headline parity test pins `hashObject({b:1,a:2}) == "48AVoXIXcTKcnHt8qVKp5vNw4gyOB5VfztHwtYBRcAQ="` against the upstream fixture in [`crypto/object-hasher/test/index.ts:6`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/crypto/object-hasher/test/index.ts#L6). - **`VerifyResult.side_effects_maps`** in `pacquet-store-dir`. The verify path (`check_pkg_files_integrity` and `build_file_maps_from_index`) used to drop `PackageFilesIndex.side_effects` after extraction. It now applies the `added`/`deleted` overlay per cache key and surfaces a `HashMap<cache_key, FilesMap>` — the same shape pnpm uses for its `PackageFilesResponse.sideEffectsMaps`. Mirrors [`applySideEffectsDiffWithMaps`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/store/create-cafs-store/src/index.ts#L103-L121). - **`Config.side_effects_cache`** config knob (default `true`, matching pnpm). Surfaced on the wire **only** — no consumer yet. Wires up cleanly once the build-phase gate lands in #421, with no config migration needed for downstream callers. ## Scoped narrowly This is **only the foundation**. There is no `is_built` field, no `BuildModules` skip gate, and no end-to-end test of the rebuild-skip behavior. Those land in a separate follow-up tracked in #421, which also covers the WRITE path (pacquet seeding the cache itself). The natural slice for review: graph-hasher unblocks both READ and WRITE, the verify-path surfacing is the only intrusive cross-crate change, and the config knob is forward-looking. ### Hashing scope, explicitly The `pacquet-graph-hasher` port implements only the type arms pacquet actually feeds into `hashObject` for the cache-key path: strings, objects, numbers, booleans, null, and arrays in their ordered form. `Set` / `Map` / `Date` / `Buffer` / `Array` unordered-permutation arms from [upstream `object-hash`](https://github.com/puleos/object-hash/blob/v3.0.0/index.js#L257-L389) are unimplemented — no caller in pacquet's tree feeds them in today, and the test fixtures upstream uses (`hashObject({ b: new Set([…]), a: [...] })`) hash deterministically across orderings only because they're testing the library, not exercising a real call path. Adding them can land alongside a future caller that needs them. |
||
|
Zoltan Kochan
|
582ef43b6c |
refactor(config): rename pacquet-npmrc → pacquet-config and drop ini wiring (#420)
## Summary - pnpm 11 reads project-structural settings (storeDir, hoist, nodeLinker, fetchRetries, …) from `pnpm-workspace.yaml` and limits `.npmrc` to auth/registry/network keys. The crate that holds pacquet's resolved runtime config was still named `pacquet-npmrc` with a `Npmrc` type and carried `serde_ini` deserializer wiring on every field, which no longer reflected how the config is actually loaded on this branch. - Two-commit refactor: rename the crate / type / imports, then drop the dead serde_ini machinery and the seven duplicate ini-based tests. ## Commits 1. **`refactor(config): rename pacquet-npmrc crate to pacquet-config`** — mechanical rename. `crates/npmrc/` → `crates/config/`, `Npmrc` → `Config`, `pacquet-npmrc`/`pacquet_npmrc` references updated workspace-wide. `NpmrcAuth` (the `.npmrc` auth-subset parser) keeps its name. 2. **`refactor(config): drop ini deserializer wiring from Config`** — removes `Deserialize` + `#[serde(rename_all = "kebab-case")]` from `Config`, every per-field `deserialize_with`/`default = "..."` attribute, the helper fns that backed them (`bool_true`, `deserialize_bool`/`u32`/`u64`/`pathbuf`/`store_dir`/`registry`), and the `serde_ini` workspace dep. Drops seven ini-based `lib.rs` tests that already had equivalent yaml/npmrc-auth coverage. Renames `custom_deserializer` → `defaults` (only default factories left), and the `npmrc: &mut Config` local in `apply_to` / `Config::current` becomes `config`. After the cleanup `Config` is purely a merged runtime struct: yaml deserializes into `WorkspaceSettings` and applies onto a `Config`, `.npmrc` is hand-parsed by `NpmrcAuth::from_ini`, and `Config` itself is no longer deserialized from any file. |
||
|
Zoltan Kochan
|
2f64c727ea |
feat(executor): swallow optional-dep build failures and report via pnpm:skipped-optional-dependency (#397) (#419)
## Summary Closes item #6 of #397. Optional dependencies whose `postinstall` hooks fail no longer abort the install — pacquet now reports the failure through the `pnpm:skipped-optional-dependency` channel (reason `build_failure`) and continues, matching [pnpm v11's behavior at `building/during-install/src/index.ts:218-240`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L218-L240). Three commits: - **`feat(lockfile): surface snapshot-level optional flag`** — pacquet's lockfile reader previously dropped the `snapshots[<key>].optional` field. Adding it as `pub optional: bool` on `SnapshotEntry` with `#[serde(default, skip_serializing_if = "is_false")]` so absent ↔ false and `false` never serializes. The flag is pre-computed by pnpm's resolver at install time (ALL-paths- optional fold), so pacquet trusts the precomputed value rather than re-deriving it — matching [`lockfileToDepGraph` at deps/graph-builder/src/lockfileToDepGraph.ts:315](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/deps/graph-builder/src/lockfileToDepGraph.ts#L315). - **`feat(reporter): add pnpm:skipped-optional-dependency event`** — new `LogEvent::SkippedOptionalDependency(SkippedOptionalDependencyLog)` variant + `SkippedOptionalPackage` + `SkippedOptionalReason` enum. Wire shape mirrors [pnpm's `SkippedOptionalDependencyMessage`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/core/core-loggers/src/skippedOptionalDependencyLogger.ts). All four upstream reasons (`BuildFailure`, `UnsupportedEngine`, `UnsupportedPlatform`, `ResolutionFailure`) are declared even though only `BuildFailure` is emitted today — keeps the enum closed so the other emit sites can land without widening it. - **`feat(package-manager): swallow optional-dep build failures`** — `BuildModules.run` reads `snapshot.optional`. When `run_postinstall_hooks` returns an `Err`, the dep's optional flag decides: optional ⇒ emit a `pnpm:skipped-optional-dependency` event (`reason: build_failure`, `package.id` = the dep's pkg dir to match upstream's `depNode.dir`) and continue; non-optional ⇒ propagate as `BuildModulesError::LifecycleScript` so the install aborts. Two `cfg(unix)`-gated tests cover both branches. |
||
|
Zoltan Kochan
|
1452682669 |
feat(executor): port makeEnv, extendPath, and shell selection for lifecycle scripts (#397) (#418)
## Summary Closes three of the critical items in #397 by porting the corresponding behaviors from `@pnpm/npm-lifecycle@d2d8e790` and `pnpm/pnpm@b4f8f47ac2`: - **#1 — Lifecycle env vars.** New `make_env` module ports `makeEnv` and the surrounding env block in `lifecycle()` ([npm-lifecycle/index.js:73-104](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/index.js#L73-L104), [:354-414](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/index.js#L354-L414)) plus the pnpm wrapper's `extraEnv` additions ([runLifecycleHook.ts:119-124](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/exec/lifecycle/src/runLifecycleHook.ts#L119-L124)). Lifecycle scripts now see `npm_lifecycle_event`, `npm_lifecycle_script`, `npm_node_execpath`/`NODE`, `npm_package_json`, `npm_execpath`, `npm_package_*` (`name`, `version`, and recursive `config`/`engines`/`bin`), `npm_config_node_gyp`, `npm_config_user_agent`, `INIT_CWD`, `PNPM_SCRIPT_SRC_DIR`, and `TMPDIR` (when `unsafe_perm` is false). The spawn now strips inherited env (`env_clear()`) so leftover `npm_*` keys from a wrapping invocation cannot leak through. - **#2 — PATH ancestor walk.** New `extend_path` module ports `extendPath` ([npm-lifecycle/lib/extendPath.js:5-27](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/lib/extendPath.js#L5-L27)) plus the tri-state `scriptsPrependNodePath` gating ([:29-61](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/lib/extendPath.js#L29-L61)). For a dep at `<root>/node_modules/.pnpm/<slot>/node_modules/<pkg>`, `PATH` now contains the dep's own `.bin`, the `.pnpm` slot's `.bin`, the project root's `.bin`, the bundled `node-gyp-bin` (when supplied), `extra_bin_paths`, and finally the inherited PATH. - **#4 — Shell selection.** New `shell` module ports the [shell-selection block](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/index.js#L241-L252) and the [pnpm-side `.bat`/`.cmd` guard](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/exec/lifecycle/src/runLifecycleHook.ts#L63-L71). `cmd /d /s /c` on Windows, custom `scriptShell` on either platform, otherwise `sh -c`. The Windows-batch-file `scriptShell` case surfaces as `ERR_PNPM_INVALID_SCRIPT_SHELL_WINDOWS` (matching upstream's error code). `RunPostinstallHooks` grows seven new fields to surface these knobs; `BuildModules` passes safe defaults (`None` / `true` / `Never`) for all of them — full config plumbing for `user-agent`, `unsafe-perm`, `scripts-prepend-node-path`, `node-gyp` bundling, and `script-shell` are tracked as separate items in #397 (#14, #15) or follow-ups. ### Explicit non-goals in this PR Three caveats called out in #397 that are deliberately deferred: - `windowsVerbatimArguments` (Rust equivalent: `CommandExt::raw_arg`) is signalled by `SelectedShell.windows_verbatim_args` but not yet applied to the spawned `Command`. - `@yarnpkg/shell` / `shellEmulator: true` has no clean Rust port; pacquet ignores the flag for now. - `unsafe_perm` uid/gid drop (#14) — `BuildModules` passes `true`, which keeps current behavior (no TMPDIR creation, no privilege drop). ### Test parity Per the project guide ("port the relevant pnpm tests too whenever they translate"), this branch ports: - `test('makeEnv')` from [npm-lifecycle/test/index.js:97-124](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/test/index.js#L97-L124). - The `extendPath` ordering test from [npm-lifecycle/test/extendPath.test.js:5-8](https://github.com/pnpm/npm-lifecycle/blob/d2d8e790/test/extendPath.test.js#L5-L8). - `runLifecycleHook() does not set npm_config env vars` from [pnpm/exec/lifecycle/test/index.ts:65-77](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/exec/lifecycle/test/index.ts#L65-L77), adapted to a file-dump model so we don't need the IPC fixture. - `onlyOnWindows('pnpm shows error if script-shell is .cmd')` from [pnpm/exec/commands/test/index.ts:509-542](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/exec/commands/test/index.ts#L509-L542) and the custom-shell case from [:478-508](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/exec/commands/test/index.ts#L478-L508). Plus fills the gaps where upstream coverage is thin: tri-state `scriptsPrependNodePath`, two-level pnpm virtual-store PATH walks, `extra_bin_paths` ordering, TMPDIR gating on `unsafe_perm`, and `extra_env` precedence vs `npm_lifecycle_script`. |
||
|
Zoltan Kochan
|
b8223ec6fc |
fix(ci): pass tag_name to softprops/action-gh-release
v3 of the action requires an explicit tag_name when GITHUB_REF is not
a tag ref. The release workflow runs on pushes to main, so the API
rejected the create-release call with "Missing tag_name parameter"
once dependabot bumped the action from v1 to v3 in
|
||
|
Zoltan Kochan
|
fdbdc09121 | chore(release): 0.2.2-6 | ||
|
Khải
|
bbfa18f12b |
feat: bin (#333)
Resolves #330.
Mirrors pnpm v11's `@pnpm/bins.resolver`, `@pnpm/bins.linker`, and `@zkochan/cmd-shim` (resolved at pnpm `4750fd370c` and cmd-shim `0d79ca9534`):
- New `pacquet-cmd-shim` crate parses the `bin` field of a package manifest into commands. Both `bin: string | object` and the `directories.bin` glob fallback are supported, with the same path-traversal and URL-safe-name guards as upstream. Conflict resolution follows `pkgOwnsBin` + lexical compare.
- Shim generator writes all three flavors per bin: Unix `/bin/sh` (byte-compatible with pnpm's `generateShShim`), Windows `.cmd` (CRLF, `%~dp0` paths), and PowerShell `.ps1` (`$basedir`/`$exe` header with Windows/POSIX-pwsh detection). Mirrors `generateCmdShim` and `generatePwshShim` minus the unused `nodePath`/`prependToPath`/`progArgs` branches.
- All filesystem IO behind per-capability DI traits — `FsReadHead`, `FsReadFile`, `FsReadString`, `FsReadDir`, `FsWalkFiles`, `FsCreateDirAll`, `FsWrite`, `FsSetExecutable`, `FsEnsureExecutableBits` — following the pattern documented in [#332](https://github.com/pnpm/pacquet/pull/332#issuecomment-4345054524). `FsWrite` is a thin shim over `std::fs::write` and is **not atomic**; a future atomic-write capability can build on it without renaming. Production callers turbofish the unit-struct provider `::<RealApi>`; tests inject unit-struct fakes to cover IO error paths real fs can't trigger portably. Generic param is named `Api` (not `Fs`) so the same provider can grow non-fs capabilities without a rename.
- New `LinkVirtualStoreBins` step walks each virtual-store slot and links its child packages' bins into the slot's own `node_modules/.bin`, matching `linkBinsOfDependencies` in pnpm's `building/during-install`. Slot iteration, per-slot child reads, and shim writes parallelised on rayon.
- `SymlinkDirectDependencies` and `InstallWithoutLockfile` call the bin linker after the symlink layout is in place. Direct-dep linking and the symlink loop are exposed as free `link_direct_dep_bins` / `symlink_direct_deps_into_node_modules` functions so they're unit-testable without the mock-registry scaffold.
Tests port the relevant `bins/resolver` and `bins/linker` cases from pnpm (`directories.bin` discovery, scoped bin names, dangerous locations, idempotent shim skip, conflict resolution) and add fakes-injected coverage for the IO error variants.
**Not in this PR**: hoisted-bin precedence and lifecycle-script-created bins. Both depend on subsystems pacquet hasn't built yet — hoisting (`hoistPattern` / `publicHoistPattern` resolution + the lift step) and lifecycle scripts (`exec/lifecycle/` runner + `allowBuilds` plumbing). The bin-linking side already does the right thing for both (shims are paths, so they pick up files generated post-extract; the precedence rule is a small addition once hoisting tags candidates as hoisted). Tracked separately in with full porting coordinates.
## Performance
When the bin-linking step landed it added ~7% to the warm-cache Frozen Lockfile install ([benchmark](https://github.com/pnpm/pacquet/pull/333#issuecomment-4344568317)). Closing that gap took a sequence of changes that mirror how pnpm v11 avoids the same costs in its own `linkBinsOfDependencies` ([
|
||
|
Zoltan Kochan
|
eb2ae4d32b | docs(LICENSE): sync with pnpm/pnpm repo | ||
|
Khải
|
a8e8dd3091 |
ci: setup dylint with perfectionist (#416)
Co-authored-by: Claude <noreply@anthropic.com> |