`tempfile` creates every temp file 0600, and `pnpm_fs::write_atomic`
kept that for new files, so any writer that didn't think about modes
left project files owner-only: `pnpm-workspace.yaml` (an edit also
tightened an existing file), `pnpm deploy` output, patch files, the patch
state, the lockfile `dedupe --check` restores, `.cargo/config.toml`, and
every new file written through `write_atomic`. A Docker image built as
root and run as another user could not read them. pnpm 11 follows the
umask.
- `write_atomic` gives a new file 0666 less the umask; an existing file
keeps its mode, and its replacement is staged at that mode so the
contents never become readable by more users.
- `write_atomic_private_if_new` keeps 0600-when-new for files that may
hold a credential: `.npmrc`/`auth.ini` from `pnpm config set`, and
caches of URLs that can carry userinfo (tarball HTTP, Python index and
wheel, Cargo sparse registry, pnpr upstream indexes).
- `write_atomic_private` is unchanged.
- The hand-rolled private-temp-file writers in dedupe, patch-commit,
patch state and deploy call `write_atomic`; package-manifest and pack
stage at the existing mode through `named_tempfile_with_mode_in`.
- clippy `disallowed-methods` rejects `NamedTempFile::new_in` and
`Builder::tempfile_in` outside the `pnpm_fs` helpers, so new writers
must choose a mode.
- A CLI test runs install, patch and patch-commit under umask 022 and
fails on any owner-only file in the project.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
An install run from a project whose `node_modules` is a symlink (for
example to another project's `node_modules`) computed the relative
targets of direct-dependency and hoisted links from the symlink's
lexical location. The OS resolves them from the real directory, so with
the global virtual store every recreated link dangled while the install
still exited 0.
Direct-dependency links and both hoist directories are now written from
the real path whenever a symlink lies between the project (or lockfile)
directory and the modules dir, matching the `realpathMissing` of the
modules dir in pnpm v11, which does not have this bug. Ordinary layouts
keep their lexical link contents.
Closes https://github.com/pnpm/pnpm/issues/17448
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
pnpm/pnpm#17447 reports `pnpm install --frozen-lockfile --filter app`
rejecting a fresh lockfile when a pnpmfile `readPackage` hook adds a
peer to an injected workspace package. A filtered install passes only
the selected projects to the lockfile check, so the injected project is
read from its directory and must be hooked the way the resolver hooked
it. pnpm/pnpm#17431 made the check apply the resolver's hook chain to
those manifests, which fixes this case too; 12.16.1 still fails and
pnpm 11.28.9 passes.
The test fails with the issue's "local dependency is outdated" error
when the check skips the hooks for such a directory. The changeset of
pnpm/pnpm#17431 now names this case and links the issue.
Closespnpm/pnpm#17447
With package selectors, `update --latest` now re-resolves the locked
subtree of every matched direct dependency of the updated project, so an
unchanged parent no longer keeps an outdated compatible child pinned.
Unmatched direct dependencies, the packages excluded aliases resolve to,
their subtrees, and peers keep their pins, matching pnpm v11.
The lockfile reachability walk used for pruning takes a root and an
edge filter so both callers share it.
Closespnpm/pnpm#17444
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`npx get-pnpm --no-modify-path`, or PNPM_NO_MODIFY_PATH set to anything
but "", "0", or "false", runs `pnpm setup --no-modify-path`
(pnpm/pnpm#17317). pnpm is installed into PNPM_HOME, the shell config
and the Windows registry are left alone, and setup prints the lines to
add. The uninstall hint then names only PNPM_HOME.
The option first shipped in pnpm 12.16.0. For an older version, the
install stops before setup changes anything.
From pnpm 11, a project `setup` script replaces the built-in `pnpm
setup`, so get-pnpm now runs `pnpm pm setup` on the versions that have
the prefix (pnpm 11, and 12.1.0 on).
This is get-pnpm's half of pnpm/get.pnpm.io#80, moved here with the
package; install.sh and install.ps1 keep theirs in that PR. The
argument parsing moves from runCli into parseCliArgs, which the unit
tests cover.
deepmerge has no patched release upstream for GHSA-gcqx-w48m-w2h4. Patch
deepmerge@4.3.1 to reject __proto__ keys by name in mergeObject() across
index.js, dist/cjs.js, and dist/umd.js (TehShrike/deepmerge#274), and
ignore GHSA-gcqx-w48m-w2h4 in auditConfig.
Also use recentTimestamp() across checkDepsStatus.test.ts so mocked
sub-second mtimes never land on an exact second when Date.now() % 1000 === 0.
A hosted registry that declares `tarballs: tar` is only ever served
through tar registries, so the published `.tgz` has no reader there.
Publish now verifies the `.tgz`, records its tar hash as before, and
stores the tar compressed with zstd at level 19, within the window pnpm
decodes, as `<name>-<version>.tar.zst` in place of the `.tgz`. A `.tgz`
that does not decode in full fails the publish with a 400.
The version route serves the stored bytes to a zstd client and the tar
decoded to any other, which the compression layer gzips when asked. These
versions skip the per-replica level-19 variant cache.
Config loading rejects a router without `tarballs: tar` that lists such a
registry, since it would have no `.tgz` to serve. Unpublish and packument
rewrites remove the stored tar, `pnpr verify` counts it as the version's
blob instead of an orphan, and `pnpr export` writes it.
Decided in pnpm/tasks#226.
The workspace state file was written through `private_named_tempfile_in`,
so `node_modules/.pnpm-workspace-state-v1.json` ended up 0600 while pnpm 11
left it 0644 (`fs.writeFile`). Another user could not read it, so the
deps-status check before `pnpm run` treated `node_modules` as stale and
reinstalled. A Docker image that installs as root and runs as a non-root
user hits this on every start.
Add `pnpm_fs::umask_named_tempfile_in`, which creates the temp file with
0666 narrowed by the umask, and use it for the state file. The write
stays without an fsync, as before.
An npm alias added with `pnpm add` now saves the range of the version
it resolved to, matching v11's calcSpecifier: `v2@npm:foo` records
`npm:foo@^1.0.0` and `v2@npm:foo@^100.0.0` records `npm:foo@^100.1.0`.
A re-add of an alias without a version keeps the existing range.
A dist-tag other than `latest` that `pnpm add` writes into a catalog
entry records the exact version the tag points to, in every catalog mode,
as v11's getCatalogUpdatesOfProject does. The manifest specifier of an
uncataloged dependency keeps its range. A dependency added without a
version counts as naming the configured `tag` setting, as v11's
parseWantedDependencies substitutes it. `latest` and aliased tags are not
pinned, matching v11's isExplicitDistTagSpecifier.
Related to pnpm/tasks#191.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The resolver runs every resolved manifest through one chain: package
extensions and the compatibility database, the pnpmfile's `readPackage`
with the resolution's directory, then overrides and the
`ignoredOptionalDependencies` removal. The lockfile freshness check read
a directory package that no workspace project describes, such as a
`file:` dependency outside the workspace or anything nested below one,
with only the overrides applied. So a `readPackage` hook that changed
such a package's dependencies failed `pnpm install --frozen-lockfile`
right after the install that wrote the lockfile, and an
`ignoredOptionalDependencies` entry for such a package's optional
dependency made every repeat install resolve again. pnpm 11 handles
both.
`ManifestTransformHooks::apply` is now the chain, and the resolver's
walk calls it. The check builds the same hooks from the config and,
before its synchronous walk, follows the same directory edges level by
level, hooking each manifest it will compare under the declaration that
reaches it. This replaces
`hooked_injected_manifests`, which covered only the direct injected
dependencies of a project with its own lockfile, and only
`packageExtensions` and `readPackage`.
Part of pnpm/tasks#220 (item 1, the manifest a checker reads).
* chore(release): pacquet 12.16.1
* docs(release): curate the pacquet 12.16.1 release notes
Add a lead line, order the entries by impact (hang, packed files,
broken links, then behavior and output fixes), and restate the
peer-dependency and injected-symlink entries by what users see.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(release): mark the old pnpm unpublish suggestion as past behavior
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The oracle's renamed-bin scenario checked only that the dependent links
the new name. Both CLIs now also remove the old one, pnpm v11 since
pnpm/pnpm#17371, so the scenario asserts that too, for every layout and
driver.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The parity proptest from pnpm/pnpm#17415 found
`files: ["**", "!dist/index.js", "dist/**"]` packing dist/index.js, which
npm-packlist and `@pnpm/fs.packlist` exclude. npm-packlist keeps entries
that name an existing file in its strict rules, applied after the rules
directories and globs produce, so a later `dist/**` cannot re-include a
file an exclusion names, and a named file ships past a later glob
exclusion. The Rust port kept every entry in one ordered list. The pack
dotenv warning now counts a literal entry naming the path as listing it.
File entries now go into StrictRules, appended after the ordinary rules.
Strict exclusions come after strict inclusions, so an exclusion naming a
file always wins over an entry naming it. npm-packlist lets the earlier of
the two win, which packs `lib/secret.js` for
`["lib", "lib/secret.js", "!lib/secret.js"]`. v12 keeps excluding it,
recorded as DeliberateDivergence::FilesFieldExclusionNamingFile with a
witness.
The failing seed is saved in proptest-regressions so every run replays it.
Add `cache-dir` to `BARE_SETTING_FLAGS` as a text setting, so
`--cache-dir <dir>` and `--cache-dir=<dir>` are pulled out of argv before
clap sees them and recorded in the CLI layer's setting values, the same path
`--config.cache-dir=<dir>` takes. Config loading then sets `cacheDir` and
records it as an explicit setting.
No command declares its own `--cache-dir` option and no global option uses
the name, so claiming it on every command line is safe.
Fixespnpm/pnpm#17422
Closespnpm/tasks#222.
Unify the pinned-pnpm switch and pin-recording decision across all CLI entry
points (commands, `--version`, unknown flags, `self-update`, and spawned
installs) in both pnpm v12 (`pnpm/`) and pnpm v11 (`pnpm11/`):
- Replace `VersionArgv`, `UnparsedArgv`, `PinFlags`, and `SwitchInput` in v12
with a single `Invocation::from_argv(argv, entry_point)` model.
- Extract `apply_cli_layers(config, invocation, config_overrides, dir)` so
pre-command loads and the command's full config load apply `--config.<key>`
overrides and CLI flags at the same `lockfileDir`, forwarding CLI config
into the switched pnpm, `verifyDepsBeforeRun` installs, pin recording, and
bootstrap downloads.
- Introduce a pure `decide(pin, settings, invocation)` (`decidePinAction` in
v11) function returning `Switch | Record(at) | Nothing` used by all entry
points including `self-update`.
- Load the full config at most once and reuse the config loaded during pin
recording for the subsequent command execution.
- Add the shared `fixtures/pin-decision.json` test matrix and integration
tests (`pin_decision_matrix`, `cli_config_reaches_every_child`, and
`pin_recorded_only_from_full_config_at_lockfile_dir`) in both stacks.
overwriteDirSync moves an existing package directory aside before
renaming the staged one in. On overlayfs, renaming a directory from a
lower layer fails with EXDEV, which happens when node_modules is copied
in from an earlier Docker build stage and pnpm prune re-imports a
package. Treat EXDEV like EPERM/EACCES/EBUSY: copy a backup aside and
remove the target in place. If that removal fails partway, copy the
missing entries back from the backup.
Closespnpm/pnpm#17424
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`pnpm unpublish pkg@1.0.0` on a package whose only version is 1.0.0 is a
full unpublish, so it needs --force. The refusal reused the no-version
message, which ends with "If you want to unpublish a specific version, run
pnpm unpublish pkg@<version>": the command that had just failed. Both CLIs
now give the range case its own message, under the same
ERR_PNPM_UNPUBLISH_CONFIRM code, naming the spec and the --force command.
* fix(binary-fetcher): retry zip downloads that drop mid-body
The zip path of the binary fetcher (Node.js runtimes on Windows) read the
response body outside any retry. fetchFromRegistry retries only until the
response headers arrive, so a TLS connection that closed during the body
rejected the whole install with undici's bare "terminated" error, which
names no URL.
The download and checksum step now runs under the same retry policy as
tarball downloads: 401, 403, 404 and non-retryable network errors fail at
once, anything else is retried with the configured backoff and logged
through the request-retry logger. A download that still fails names its
URL. The response status is now checked, so an error page is no longer
hashed as the archive.
The tarball downloader already retries a mid-body drop, which was checked
against a server that destroys the socket after a partial body. pnpm v12
already retries zip downloads as a whole.
Closes https://github.com/pnpm/pnpm/issues/17411
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(binary-fetcher): keep retry defaults and hide URL secrets in retries
The install client passes every retry field, with unset ones as
undefined, and `@zkochan/retry` copies undefined over its defaults, so an
unset fetch-retries disabled zip retries. Undefined fields are now
dropped before merging with the defaults.
The request-retry event no longer carries the error message, which for a
checksum mismatch embedded the raw URL, signed-URL tokens included. The
checksum error itself now shows the redacted URL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(binary-fetcher): reword a comment for the spell checker
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A cold `pnpm install` in a workspace with Cargo crates or a Rust
toolchain hung forever on pnpm 12.16.0, and so did every CI job after
the repository moved to it.
`InstallPlan::run` polled the npm install and the other ecosystem
participants together with `join_all`, on one task. Since
pnpm/pnpm#17281, the cold download drain blocks its task with
`block_in_place` and drives only the npm downloads with `block_on`. The
sibling participants then went unpolled while holding permits of the
shared throttled HTTP client, and the npm downloads waited for those
permits forever.
On the multi-thread runtime, the coordinator now drives each participant
on a scoped thread of its own, so a participant that blocks its task
cannot starve the others.
The storage summary formatted `Option<String>` with `{:?}`, so it showed
`prefix: Some("pnpr")` or `prefix: None`. It now prints the normalized
prefix the store uses for its keys, and nothing when there is none. The
custom object store line gets the same treatment.
Packages such as esbuild publish their binary entry with a
`#!/usr/bin/env node` shebang and overwrite that file with a native
executable during their `postinstall` lifecycle script.
In isolated mode, privately hoisted bin shims under
`<install_state_dir>/node_modules/.bin` are linked before dependency
builds run so that dependency lifecycle scripts can resolve hoisted
commands on `PATH`. When a package's `postinstall` replaced a script
with a native binary, the privately hoisted shim was not refreshed
either between build levels or during post-build bin relinking,
leaving a `node` wrapper that failed when executed directly or from a
dependent's `postinstall`.
- In Rust (`pnpm/`), preserve `resolved_location` and `origin` when
refreshing a `DirectoryBinPlan` entry, seed `HoistedBinPlans` with
the privately hoisted bin sources and aliases from
`write_hoist_links`, and refresh
`<install_state_dir>/node_modules/.bin` both after each mutated
snapshot build and in `post_build_bins::relink_dependency_bins`.
- In TypeScript (`pnpm11/`), extract `relinkModulesDirPackageBins` into
`@pnpm/bins.linker`, refresh privately hoisted and hoisted-location
bin shims during `@pnpm/building.during-install` and
`@pnpm/building.after-install`, and pass `forceForPackages` through
post-build bin relinking in `@pnpm/installing.deps-restorer` and
`@pnpm/installing.deps-installer`.
Closespnpm/pnpm#17402
Found on the Cloudflare trial: with `tarballs: tar` on the main router,
pnpm 12.16.0 failed with "names no tarball URL" until the project declared
the base with `serverType: pnpr`, which the pnpm registries docs require for
any tar registry.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
With sharedWorkspaceLockfile: false each project installs from its own
lockfile and records its own workspace state, and the bins digest that
proves an importer installed covered only the projects of that lockfile.
A sibling's bin field was in none of them, so the repeat-install gate,
the frozen-tree check and the run gate all reported a dependent up to
date after the sibling renamed its command, and the dependent's .bin kept
the old name.
The digest now covers every workspace project in that case. The install
takes them from the workspace the dedicated loop already discovered, and
the run gate loads them, as it does for a shared lockfile. A state that
an older version recorded has no such digest, so the first install after
the upgrade is a full one in a workspace whose projects declare commands.
The repeat-install oracle's renamed-bin scenario now runs with dedicated
lockfiles too.
Related to pnpm/tasks#224.
Read manifests for packages whose locked metadata declares peers, and route
them through normal resolution. Snapshot dependencies conflate own children
and peer providers, so they cannot independently reconstruct that split.
Keep snapshot reuse for subtrees without declared peers.
Share manifest PeerSet construction with directory freshness checks and move
the generated reuse tests alongside the production lockfile writer. Compare
serialized writer output and cover arbitrary incremental addition order when
provider versions are fixed. Preserve valid optional-peer pins explicitly;
unrestricted incremental/fresh equality would change install semantics.
Pass trustLockfile into cached manifest reads for kept versions so this walk
does not require registry access merely to recover their manifests. Never
apply that exemption when the resolver reopens a version.
Retain hashed recovery for lockfile-only consumers that lack manifests and
owner scope. Related to pnpm/tasks#221.
Route every package file selector (pack/publish, git-fetcher,
directory-fetcher, patch-commit, deploy, and the injected-copy writers)
through `pnpm-fs-packlist` (`select_package_files`,
`select_dir_package_files`, `packlist_dir`) and the shared internal
symlink admission and placement helpers in `pnpm_fs::internal_symlink`
(`admit_internal_symlink`, `resolve_symlink_placement`,
`recreate_internal_symlink`, `SymlinkTargets`, `AdmittedSymlink`).
Test `pnpm-fs-packlist` against `npm-packlist` and `@pnpm/fs.packlist`
on a generated `proptest` corpus (`generated_trees_match_npm_packlist`),
against `npm pack --dry-run --json` (`pack_matches_npm_pack_dry_run`,
run nightly in `cargo-equivalence.yml`), and across all file-selection
consumers (`every_file_selector_agrees_with_pack`). Keep deliberate
divergences in `DELIBERATE_DIVERGENCES` with witness tests.
Fix two bugs surfaced by the new tests:
1. Normalize string `bin` fields into an object in `@pnpm/fs.packlist`
before passing the manifest to `npm-packlist`.
2. Canonicalize symlink sources in `import_indexed_dir` when falling
back to importing an internal symlink to an unselected file as a
regular file, so hardlink imports link the target file rather than
the symlink inode.
Closes https://github.com/pnpm/tasks/issues/223
Limit pruning to metadata cache directories and preserve configured stores and global data when cacheDir contains them. Fix shell assertions that can fail on an early grep match under pipefail. Related to pnpm/tasks#191.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every writer of an injected copy, and `pnpm deploy`, read its source
through a `DirectoryFetcher` it built itself, with
`include_only_package_files` computed at each call site as
`!config.deploy_all_files`. pnpm/pnpm#16684 and pnpm/pnpm#17033 were
writers that got that flag wrong.
`pnpm_directory_fetcher::CopyFiles` now owns the decision: the packlist
or every file, symlinks listed by their own path, and no path escape.
The install's directory fetch, the custom fetcher's directory callback,
the sync after scripts, the edit watcher, the dedicated-lockfile sync
and deploy all list their source through `CopyFiles::fetcher`, and take
a `CopyFiles` built from the `deployAllFiles` setting instead of a bool.
This is item 2 of pnpm/tasks#220 for pnpm v12. No behavior changes.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* chore(release): 11.28.9, pacquet 12.16.0, pnpr 0.1.0-alpha.17, get-pnpm 0.0.10
* docs(release): curate the 11.28.9, 12.16.0, and pnpr alpha.17 release notes
- Add a lead line to each page and group the patch entries by what the
reader was doing, with security fixes first.
- Merge the duplicate strictSsl/localAddress entries and the repeat-install
setting entries into one list per page.
- pnpr: split the removals into commands, flag aliases, and camelCase
config keys, and say that pnpr no longer serves PUT /-/pnpm/v1/publish,
so batch publishing needs pnpm 11.28.9 or 12.16.0.
- Drop the "Updated dependencies: get-pnpm" line from the pnpm v12 page
and list the get-pnpm credential fix first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`force_symlink_inner` retargets a stale link by unlinking it and
reissuing the create. Nothing bounded that step: when the create
reported `AlreadyExists` again, the link was read, found stale again,
unlinked again, and the function called itself again. On a filesystem
that answers the unlink with `NotFound` but keeps serving the entry (a
`node_modules` mapped from a lockfile by a FUSE daemon does this for an
entry it synthesizes), the recursion only ended when the rayon thread
ran out of its 2 MiB stack, and the host process died with SIGSEGV.
Seen as a crash of Bit's CI runner during a rebuild of install-script
packages against such a mount.
`TriedOnce` gains `retargeted`, set on the retry after the unlink. A
link that still reads as stale on that retry is reported through the
create's own error, with the link, its current target and the wanted
target in the message, so the caller sees which entry the filesystem
would not let go of.
The test's mock create puts the stale link back before reporting the
conflict, which is what the caller observes on such a filesystem.
Without the bound the test aborts with a stack overflow.
The repeat-install oracle now edits overrides, a pnpmfile hook,
patchedDependencies, dedupePeers, a workspace project's bin and a scope's
registry route, and checks every gate against a full install after each
edit. Its snapshot records each importer's .bin entries and the files of
every registry dependency, and a selection's view adds its .bin entries and
the lockfile snapshots it reaches.
Two gate bugs it found are fixed:
- With dedicated lockfiles, the install fast path and the run check read
patch files from the project's own directory, while the config hashes
them from the workspace directory, so an edited patch went unnoticed.
Both now take the paths from Config::patch_file_paths.
- pnpm v11 records each patch file as an absolute path and pnpm v12 as
configured, so each took the other's record for a changed setting. Both
now resolve the recorded paths against the workspace directory before
comparing them.
The run gate compares workspace-wide inputs, so after an edit outside the
manifests the oracle allows it to refuse a selection the edit leaves as it
is. The route scenario is checked against the install in place, because
the routes a lockfile was resolved through live in the workspace state.
The bin scenario skips dedicated lockfiles until pnpm/tasks#224 is fixed.
Related to pnpm/tasks#219.
With nodeLinker: hoisted and autoInstallPeers: false, a package that two
workspace projects use with different peer versions got only one copy,
so the second project resolved the first project's peer. pnpm v11 nests
the second variant; this was a v12-only regression.
The hoister now keeps a peer-bearing node's own snapshot key as its
reference. A node without peer names still collapses onto the
first-seen variant, as pnpm v11's depPathByPkgId does, which keeps
peer-variant-heavy lockfiles from exploding the per-path walk. When
autoInstallPeers is on, peer names are empty and every variant
collapses as before.
The hoisted dep-graph walk indexes package locations by exact snapshot
key as well as by package id. Importer direct dependencies and child
edges resolve to their own variant's copy first, then fall back to the
package id for variants the hoister collapsed.
Keeping the collapse and adding yarn's peer check to the hoister's
dedup path also places the copies correctly, but the walker then sees
two copies with one snapshot key. The hoisted linker treats those as
duplicates, builds one, and symlinks the other, which cross-links the
variants again.
On a large real lockfile with autoInstallPeers off, the hoisted tree
grows from 8358 to 17157 nodes. The yarn-parity alternative produces
17570, so the growth is the copies peer resolution requires.
Closespnpm/pnpm#17395
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The real-path guard from pnpm/pnpm#16619 accepted only files whose real path
lies under the project directory. When publishConfig.directory is reached
through a symlink, every file in it resolves outside the project, so pack
and publish shipped only package.json.
Bound the packed files and the bundled dependencies by the real paths of
both the resolution root and the packed directory. Links that resolve
outside both stay excluded. Resolve bundled dependencies from the real
packed directory upward, then from the resolution root when the packed
directory lies outside it, so dependencies in the publish directory's own
node_modules and in the project's node_modules are both bundled. The
fallback applies to the resolution root alone: a bundled package's own
dependencies resolve from its real directory the way Node resolves them,
never from the publish directory's node_modules.
pnpm v12 is not affected: it keeps regular files under the packed directory
and compares symlink targets against the canonical package directory.
Closespnpm/pnpm#17390
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
When the live frame outgrew the terminal, the overflow handover shrank
the frame to the committed lines and a fresh differ redrew every
visible line. A frame that grows by one line per event, like
`pnpm run --stream` output, rewrote the whole screen for every line.
The handover now diffs the whole uncommitted frame against the one on
screen, which still fits the terminal, so only changed and new lines
are written and the overflow scrolls away. The restarted differ takes
the visible lines as already drawn. Fixed in both the v12 reporter and
the v11 live output.
Closespnpm/pnpm#17387
The global `config.yaml` and `PNPM_CONFIG_*` may set the legacy `proxy`
and `noproxy` keys, and pnpm v12 overlays them onto its bootstrap. The
v11 bootstrap and get-pnpm read them from the `.npmrc` only, so a
`proxy: false` in `config.yaml` left the environment proxy in effect.
Both now take every proxy key from the first layer that names it.
get-pnpm also reads the global `config.yaml` the way pnpm does: a file
it cannot read or parse throws instead of being skipped, `%NAME%`
references in `LOCALAPPDATA` and `XDG_CONFIG_HOME` are expanded on
Windows (one that names no variable throws), and `${NAME-default}` /
`${NAME:-default}` placeholders fall back to their defaults.
Covered by a v11 bootstrap test and four get-pnpm tests; each fails
with its fix removed.
Related to https://github.com/pnpm/tasks/issues/215
When a workspace package renames or removes a bin entry in package.json,
its link: reference in pnpm-lock.yaml stays identical, so repeat
installs only linked the current bin commands and left stale shims in
dependent projects' node_modules/.bin and in the workspace root's
node_modules/.bin when hoist-workspace-packages is enabled.
Track the command names claimed during each project bin-linking pass in
both pnpm v12 (pnpm-cmd-shim + pnpm-deps-restorer) and pnpm v11
(@pnpm/bins.linker + @pnpm/installing.deps-restorer +
@pnpm/installing.deps-installer), and prune any unclaimed symlinks or
pnpm-generated shims (including .cmd, .ps1, and .exe siblings on
Windows) from real project .bin directories once the pass completes.
Claim interrupt relay slots before spawning child processes and
register child process groups with the group watchdog from pre_exec
before execve so signals arriving while spawn is in flight are queued
and delivered.
Fixes https://github.com/pnpm/tasks/issues/211
The Config classification test from pnpm/pnpm#17383 listed settings a
full install acts on and the fast path never compared. These now go into
the workspace state, and the fast path and the run check compare them:
- `virtualStoreDirMaxLength`: a full install renames the virtual store
slots.
- `dangerouslyAllowAllBuilds`: turned on, it allows the builds the last
install ignored.
- `trustLockfile`: turned off, the lockfile no install verified has to be
verified.
- `globalVirtualStoreDir` (v12 only): a full install links from the new
directory.
The first three exist in both CLIs and are fixed in both. An unrecorded
value reads as the default, so a state written before this change keeps
the fast path for anyone on the defaults, and each CLI reads the other's
state without seeing a change. v11 writes `virtualStoreDirMaxLength` and
`dangerouslyAllowAllBuilds` with every state, so v12 writes them too.
`trustLockfile` and `globalVirtualStoreDir` are written only while they
differ from the default.
The run check ignores `virtualStoreDirMaxLength` and
`globalVirtualStoreDir`, as it ignores `virtualStoreDir`: a script runs the
same whatever the virtual store's slots are named or where they live.
Part of pnpm/tasks#219.
The worker pool and the default workspaceConcurrency took the CPU count
from os.availableParallelism(), which counts the CPUs in the process's
affinity mask. In a container with a cgroup CPU quota smaller than that
mask, such as a Kubernetes pod with a 4-CPU limit on a 233-CPU node,
pnpm 11 created 232 workers per install, and four concurrent installs
exhausted a 16 GiB memory limit while the workers were still loading.
Node's libuv does try to apply the quota, but its cgroup v1 parser
looks for ":cpu," and keeps "cpuacct:" in the path it builds, so it
never finds the quota file there, and both it and Rust's standard
library give up when the cgroup path from /proc/self/cgroup is not
mounted under /sys/fs/cgroup, as happens with a host cgroup namespace
and a bind-mounted controller.
A new package, `@pnpm/os.available-parallelism`, reads the quota itself:
the smallest limit/period walking from the process's cgroup up to the
hierarchy root, for cgroup v2 (cpu.max) and v1 (cpu.cfs_quota_us and
cpu.cfs_period_us), rounded down and floored at 1 like Rust's
std::thread::available_parallelism, which pnpm 12 already uses. The
cgroup directory is located through /proc/self/mountinfo, which says
which cgroup each mount shows, so a container that sees only its own
cgroup bind-mounted at /sys/fs/cgroup is read correctly; the standard
mount points are the fallback. The worker pool and the config reader's
workspace-concurrency default both use it, so the two stay in step.
pnpm 12 is unaffected: every Rust pool is sized with
std::thread::available_parallelism, which honours the quota.
Closespnpm/pnpm#17382
A plain install that selected only some workspace projects purged the
shared root modules directory when a layout setting changed, but relinked
only the selected projects. The projects it skipped lost their
dependencies: under the hoisted linker they live in the root modules
directory, and under the isolated linker their links point into the
purged virtual store.
Both CLIs now recreate the modules directory only for an install of every
project. A filtered install fails with the drift's error instead. This
covers nodeLinker, hoistPattern, publicHoistPattern,
virtualStoreDirMaxLength, the store, the virtual store, and the layout
version. In v11 a project installed with other dependency groups still
purges as before.
In v12 the unsafe-target error for a filtered install is unreachable now
and is removed. The refusal accepts some drift that the layout check
flags, such as a tree linked from the sibling v3 store. For that case the
install clears is_inconsistent, so it merges into the recorded current
lockfile and .modules.yaml.
Closespnpm/tasks#217.
pnpm v11 built the proxy settings of the package-manager bootstrap
(`pnpm self-update` and the download of the pnpm version pinned in
`packageManager`) from the trusted `.npmrc` files and the `HTTPS_PROXY`
family of environment variables only. `httpProxy`, `httpsProxy`, and
`noProxy` set in the global `config.yaml`, through `PNPM_CONFIG_*`, or as
`--config.httpsProxy` applied to ordinary registry requests but not to
the bootstrap. pnpm v12 already overlays those layers onto the
bootstrap proxy.
`createPackageManagerNetworkConfig` now picks each proxy key from the
first layer that names it, in the project precedence: command line
(`configFromCliOpts`, camelCased so both spellings), then
`PNPM_CONFIG_*`, then the global `config.yaml`, then the trusted
`.npmrc`. The existing cascade then runs over the result: the legacy
`proxy` key, the https-to-http fallback, and the unprefixed environment
variables. A layer that names a key with a value that reads as unset
masks the lower layers and falls through to the environment, as in v12.
For `noProxy`, only a host list or `true` counts as configured: an
empty string, `false`, or an array (which `PNPM_CONFIG_NO_PROXY` parses
JSON into) falls through to the `noproxy` spelling and then `NO_PROXY`,
as ordinary requests and v12 do.
The environment's proxy settings are read up front by
`readEnvProxySettings` instead of being mirrored in `applyEnvSetting`
like `strictSsl`: the bootstrap's `httpProxy` derives from its
`httpsProxy`, so overwriting the resolved value after the cascade ran
would leave the two inconsistent. `applyGlobalYamlConfig` returns the
file's three proxy settings next to `strictSsl` and `localAddress`, and
`OptionsFromRootManifest` types `noProxy`. The project
`pnpm-workspace.yaml` stays out of the bootstrap.
Covered by seven tests next to the "package manager bootstrap" tests in
`pnpm11/config/reader/test/index.ts`. With the overlays removed, the
four precedence tests fail; with the `noProxy` check removed, the
three fallback tests fail.
get-pnpm: `pnpmProxyConfig` resolves the proxy for the Corepack
download the same way: `PNPM_CONFIG_*` over the global `config.yaml`
over the user `.npmrc`, each key from the first source that names it,
then the legacy `proxy` / `noproxy` keys and the environment variables.
Empty, `false`, and `null` read as unset; `proxy=false` turns the proxy
off, environment included; `noProxy: true` becomes Node's `*`. The
project `.npmrc` and `pnpm-workspace.yaml` stay out. There is no
command-line layer, since Corepack hands the launcher pnpm's own
arguments. `corepackDownloadOptions` returns the result as `proxy`,
also for `COREPACK_NPM_REGISTRY`, and `downloadPnpmExecutable` hands it
to `useProxyFromEnv`, which now takes the proxy variables as a
parameter and still defaults to `process.env`. Covered by unit tests of
`pnpmProxyConfig` and `corepackDownloadOptions`, and by two download
tests against a forwarding proxy; with the proxy dropped from either
function, three of them fail.
Closes https://github.com/pnpm/tasks/issues/215
A contract test reads the fields of `Config` from its source and requires
each to be classified: recorded in the workspace state, ruling the fast
path out, reaching it through another input, not an install input, or a
known gap. A recorded field is proven by changing it and asserting the
install gate reports its setting key, and a blocking field by asserting
the fast path is ruled out. A new `Config` field fails the test until it
is classified.
The gaps the classification found were checked with the CLI: changing
`virtualStoreDirMaxLength`, `globalVirtualStoreDir`, or turning on
`dangerouslyAllowAllBuilds`, or turning off `skipRuntimes` leaves the fast path reporting the tree up
to date while a full install changes it.
Also drop the `dedupe_peers` value every freshness caller passed through
from the config (`GatePolicy`, `CheckLockfileSettingsDriftOptions` and
the content checks). The checks now read it from the config, so no gate
can compare the lockfile against a hard-coded value again, the bug fixed
by pnpm/pnpm#17328.
Part of pnpm/tasks#219.
`pnpm store prune` keeps the packages listed in each registered project's
installed lockfile, which it finds under the project's modules directory.
A workspace that gives each project its own lockfile can move one
project's modules directory through `packageConfigs.<name>.modulesDir`.
The prune ignored that entry, missed the lockfile, and fell back to the
hard-link count, so copied, cloned, patched, and built packages of that
project could be removed.
The prune now reads `packageConfigs`, `sharedWorkspaceLockfile`, and
`lockfileDir` next to `modulesDir` and applies the entry naming the
project, as the install does: only when the workspace does not share one
lockfile. pnpm 12 already registers each project of such a workspace on
its own, so no registration change is needed.
Closespnpm/tasks#214
macOS has no `pipe2`, so Rust's std creates a pipe and sets `FD_CLOEXEC`
on it afterwards. When a parallel run spawns its scripts from several
threads, a script forked in between inherits the write end of the
watchdog's lifeline. After pnpm dies that script still holds the pipe
open, the watchdog never reads end of input, and no group is killed.
Every spawn of the executor crate now holds a shared lock and the
watchdog's spawn an exclusive one, so no fork happens while the watchdog's pipe is
inheritable, and the watchdog cannot inherit another child's pipes
either, such as the `ps` listing's output read on cancellation.
Closespnpm/tasks#216
`recorded_registry_routes_after_install` took a bare `resolved_all: bool`,
which its callers passed as `false` or as an unnamed expression. A
`ResolvedImporters` enum names the three cases the rule distinguishes:
every importer, the selected ones, or none. The name avoids the lockfile
crate's `LockfileResolution`, which is a resolution kind.
Addresses the CodeRabbit nitpick on https://github.com/pnpm/pnpm/pull/17373,
which merged before the change was pushed.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
checkLinkedPackagesAreUpToDate creates one local directory checker per
importer. Its checks share the `visited` map and the recorded peers, so a
snapshot reached through several direct dependencies is walked once per
peer scope instead of once per direct dependency.
The visited key (snapshot plus peer scope) and cycle handling are
unchanged. A revisit returns true while the first visitor may still be
checking the snapshot. That is safe because the importer's results are
combined with AND, so the first visitor's failure still marks the importer
outdated. This is the same contract siblings within one walk already had.
The recorded peers are collected on the first check, so an importer with
no local directory dependency does not scan the locked packages.
The one-shot localDirectoryDepIsUpToDate wrapper had no callers left and
is removed.
On the issue's 200-package file: DAG, a repeat install went from ~1.75s to
~0.37s and a frozen install from ~1.0s to ~0.35s.
Closespnpm/pnpm#17374
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
