mirror of
https://github.com/pnpm/pnpm.git
synced 2026-10-12 23:27:00 -04:00
pnpr@0.1.0-alpha.8
137
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zoltan Kochan
|
83350dea56 |
feat: integrate the remote side-effects cache with install (#14171)
Integrate the remote side-effects cache proof of concept with dependency installation in both TypeScript pnpm and pacquet. Plan eligible build candidates before contacting pnpr, batch lookup requests, verify the configured P-256 trust keys and artifact integrity, hydrate verified files into CAFS, and select the restored side-effects map before lifecycle scripts run. Fall back to the ordinary local build on all cache failures. Configure the feature through the remoteSideEffectsCache setting, assembled by the config reader from the workspace file, the global config file and the environment. A repository declares which organization and packages are eligible and nothing else: everything describing the act of signing stays with the machine holding the key, so a cloned repository cannot turn that key into a signing oracle. Capture and publish the actual post-build diff from trusted builders. Bound artifact hydration and blob downloads, retain CAFS paths instead of downloaded buffers, and withhold direct store writes from a store opened read-only. Add transactional owner and global storage accounting to pnpr alongside request, response, timeout, streaming-memory, manifest, variant, descriptor, lock-namespace, concurrency, and crash-residue bounds. Keep the proof of concept organization-scoped and Linux glibc-only. Defer lockfile pinning, persistent quarantine, publisher ownership, and key lifecycle policy to the RFC. Related to pnpm/rfcs#20. |
||
|
Zoltan Kochan
|
66398009c7 |
feat(config): honor the settings the Rust CLI only recognized (#14109)
Section 4 of the parity sweep in pnpm/pnpm#14101: settings present in pacquet's `types` table — so `pnpm config` handled them — that no code read. - `updateNotifier`: pacquet had no update notifier at all. `install` and `add` now ask the project's registry for pnpm's `latest` tag once a day, emit `pnpm:update-check`, and the default reporter prints the notice when that version is newer. The cadence lives in `<stateDir>/pnpm-state.json` under `lastUpdateCheck` — the same file, key, and `toUTCString` format pnpm writes, so the two CLIs share one throttle. The check is best-effort: an unreachable registry or an unwritable state dir cannot fail the install. Two deliberate gaps: an `install` finishing through the repeat-install fast path returns before the async runtime exists and is not covered, and the "to update, run" line never names `@pnpm/exe` — the native binary is published under both names with no marker telling them apart, and a global install of either replaces the other. - `legacyDirFiltering`: `use_glob_dir_filtering` was hardcoded on. - `initAuthorName` / `initAuthorEmail` / `initAuthorUrl` / `initLicense` / `initVersion`: `pnpm init` now writes them over the scaffold's placeholders, in place, so the key order is unchanged. The TypeScript reader dropped `init-version` from the `PNPM_CONFIG_*` env schema to keep the two `init` implementations aligned; that exclusion goes away with this commit. - `maxsockets` was inert in *both* stacks, not just in pacquet: the reader folded npm's lowercase spelling into `maxSockets` before any config file or env var had been applied, and `.npmrc` stopped carrying the key when the non-auth settings moved to yaml. The fold moves after the env loop, and pacquet gains the alias in both the settings file and the environment. The default stays `None` in pacquet (uncapped per origin, bounded by `networkConcurrency`) rather than npm's 50, which would cap it below the network concurrency it already resolves. `npmPath` is the fifth item on that list and needs no port: nothing reads it in the TypeScript CLI either, since publishing stopped shelling out to the npm CLI. Its one remaining mention — a `Pick` in `recursivePublish` that no code consumes — is dropped, so the deadness is visible. The pacquet command harness turns the notifier off for every suite: no test may reach the registry for pnpm's own `latest` tag or record the check in the developer's state directory, which the suites leave un-isolated. Related to pnpm/pnpm#14101. |
||
|
Zoltan Kochan
|
bf46078c19 |
fix(deploy): drop excluded dependency groups from the deployed project (#14069)
A shared-lockfile deploy prunes the package graph to the included dependency groups but kept every direct dependency in the deployed package.json and in the deployed lockfile's importer. The deployed lockfile therefore referenced packages it did not define, and the next install in the deploy directory tripped over that: pacquet linked the missing packages and left dangling symlinks, while the TypeScript CLI refused the install with ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY. A --prod deploy of a project whose dev workspace dependency no longer exists pointed the importer at a missing directory the same way. Fill only the included groups into the deploy importer, in pacquet's create_deploy_files and in the TypeScript CLI's createDeployFiles, so the manifest, the importer, and the package graph agree. The graph walk no longer needs its own include gating for the importer maps; the snapshot-level optional handling from pnpm/pnpm#13641 stays as it is. Also drop an emptied group from pacquet's deploy importer instead of writing it as `{}` — the TypeScript lockfile writer omits an empty dependency map, so the two stacks were producing different deploy lockfiles for the same input. Resolve pacquet's --prod/--dev/--no-optional flags the way the TypeScript CLI does: --prod wins over --dev, and a dev-only install drops optional dependencies along with the production ones. Without it `pnpm install --dev` installed an optional dependency pnpm skips, and the deploy change above would have carried that divergence into `pnpm deploy --dev`'s manifest and lockfile. Closes pnpm/pnpm#13623 |
||
|
Pratik DulalandZoltan Kochan
|
03fd4caa00 |
fix(pack): write POSIX ustar headers for packed tarballs (#13925)
Fixes https://github.com/pnpm/pnpm/issues/13924 ## Summary - Write tar entries in the full POSIX ustar header form npm and pnpm 11 emit: `Header::new_ustar` for the `ustar\0` magic and `00` version, plus an explicit `0` regular-file typeflag (the typeflag byte stays NUL otherwise). - Add a regression test that pins the raw typeflag, magic, and version header bytes, since tar readers accept every header form and decoded values would not catch a format regression. - Add the required pacquet patch changeset, and `typeflag` to the cspell dictionary. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
daecf8765f |
perf(link): prefer hardlink over clone in Auto on Linux (#14012)
Bun materializes the same warm-store `node_modules` in roughly a third of pacquet's wall time on btrfs. Profiling the gap showed it was never syscall mechanics — it was the tier `Auto` picks: pacquet reflinks there, and a reflink is a new inode plus extent bookkeeping inside the filesystem's metadata trees, where a hardlink is one directory entry and an nlink bump. ## Measurements alotta-files fixture (39k files), warm store + lockfile, cold `node_modules`, btrfs, interleaved A/B: | | clone-first (before) | hardlink-first (after) | |---|---|---| | 32 threads | 0.91s wall / 5.4s sys | **0.53s / 2.9s** | | 4 threads | 1.14s / 1.5s | **0.66s / 0.9s** | Hardlink-first is also the default Bun ships (`--backend=hardlink`). ## Scope: pnpm 12 only This changes what the default materializes on disk, so it ships behind the v12 major. **pnpm 11's TypeScript importer deliberately keeps clone-first** — the two `Auto` implementations intentionally diverge on this until pnpm 11 is retired. The ladder's rustdoc and the changeset record the decision, and the changeset bumps only `pacquet`. ## What doesn't change - **pnpm 11**, entirely. - **ext4** (GitHub CI, the published benchmark): `FICLONE` is unsupported there, so `Auto` always ended up hardlinking after one failed reflink. Nothing moves. - **macOS** keeps clone-first — APFS `clonefile` is the platform's cheap primitive. - Explicit `packageImportMethod: clone` / `clone-or-copy` / `hardlink` / `copy` are untouched. ## The trade Clone-first bought store isolation on Linux CoW filesystems: a clone can't be corrupted by a package that mutates its own files at runtime. But every ext4 and Windows install already runs without that isolation, and the store's real guard is `verify-store-integrity`. v12 makes Linux stop paying extra for a protection the other platforms never had; users who want the isolation keep it with `packageImportMethod: clone`. ## What was tried and rejected Bun's other structural difference — `linkat` from open directory fds (a 256-entry store-prefix fd table plus a per-package dirfd) instead of absolute-path resolution — was implemented and benchmarked too. Every link was confirmed on the fd path (counted: 35k+ hits, 0 fallbacks), kernel time fell ~15%, and wall **regressed** (link phase 335ms → 531ms at 4 cores). Linux resolves hot cached paths through the lock-free RCU dcache walk; the fd anchoring saves nothing that was expensive. Making the per-package file loop sequential also regressed (straggler tail on thousand-file packages). Both reverted; noting it here so nobody re-walks that path without new evidence. ## Implementation The downgrade cache moves from `fetch_max` (which encoded the ladder in the constants' numeric order) to a compare-exchange step along a per-platform ladder (`next_auto_tier`), so racing rayon workers still converge without a lock. `pnpm:progress imported` telemetry reports the platform's ladder head instead of unconditionally claiming clone (a review catch). Tests pin the ladder order, the fresh-state hardlink, the telemetry mapping, and that a stale compare-exchange can neither skip nor regress a tier. |
||
|
cakeniandZoltan Kochan
|
e5007150fa |
fix: refetch metadata when publish times are incomplete (#13750)
Detect partial per-version publish-time maps before applying minimumReleaseAge. Some registries provide timestamps for only a subset of releases in abbreviated packuments. Re-fetch full metadata unless every version has a publish timestamp, so mature versions remain resolution candidates. Keep the TypeScript and pacquet implementations and regression coverage in sync. Closes pnpm/pnpm#13741. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
fe8ec8fbec |
refactor(pacquet): rename the Rust crates from pacquet- to pnpm- (#13948)
The Rust CLI ships as `pnpm`; `pacquet` survives only as the in-repo package identifier that keeps the npm wrapper from colliding with the TypeScript CLI's name. Carrying that identifier into all 75 Cargo crate names made every `use` statement and every `cargo nextest run -p ...` spell a name users never see. This renames the crates to the product they belong to. Mechanical, name-only: package/lib/bin names, the root `[workspace.dependencies]` keys, every dependency entry, and every Rust path reference. All the crates are `publish = false`, so nothing about the released artifacts changes and no changeset is needed. Also updated the tooling and docs that spell crate names: the `codecov` cargo alias, the `registry-mock` just recipe, the release workflow's `-p` targets and `libpnpm_napi.*` artifact paths, the `libpacquet` -> `libpnpm` cspell dictionary entry, and the crate names quoted across `AGENTS.md`, `CONTRIBUTING.md`, `CODE_STYLE_GUIDE.md`, and the plans. Two spots needed more than a token swap. The five insta snapshot files carry the crate name in their filename, since insta derives it from the module path, so they are renamed too — otherwise the tests fail with "snapshot not found" rather than a diff. And the private module in `pnpm-lockfile-verification` that mirrors the npm verifier's violation codes is named after the crate it shadows, so it moved with it. Two knock-on fixes: the shorter names let rustfmt fit three macro invocations onto one line, where `perfectionist::macro_trailing_comma` rejects the trailing comma that was legal while they were wrapped; and the `libpacquet` cspell entry became `libpnpm` for the NAPI addon's shared library. Left alone deliberately, none of which are crate names: the product name in prose and in the changeset package identifier, the `--pacquet` benchmark flag and testbed IDs, test helpers and temp-dir prefixes (`pacquet-test-*`, `../pacquet-store`), the released changesets that quote error codes shipped versions actually emitted, the `.github/workflows/pacquet-*.yml` filenames, and the `@pnpm-private/pacquet-registry-mock-launcher` npm package. |
||
|
Zoltan Kochan
|
7bee7dace4 |
fix(injected-deps-syncer): remove what the source dropped before replacing its parent (#13840)
When a workspace package replaced a directory with a file of the same name, that path landed in `modified` while the directory's contents landed in `removed`. The two ran concurrently, so a removal could be issued against a path whose parent had already been relinked as a file and come back ENOTDIR, which removeRecursive rethrows — aborting syncInjectedDepsAfterScripts partway through. Await the removals before applying any change. Nothing a change needs can be removed: a path absent from the source has all its children absent too, so no removal is ever an ancestor of an added or modified path. The Rust CLI reached the same ordering in pnpm/pnpm#13834, where a sequential patcher made this a deterministic failure rather than a race. |
||
|
Zoltan Kochan
|
4a5a10d2ca |
fix(setup): declare the module type in the manifest written for a standalone executable (#13732)
`pnpm setup` writes a package.json next to a standalone executable whose directory has none — the tarball that https://get.pnpm.io/install.sh downloads — so the global install has a package to install. That manifest omitted `"type": "module"`, so Node.js reparsed the ESM files shipped beside the executable (`dist/worker.js`) as CommonJS first and printed a MODULE_TYPELESS_PACKAGE_JSON warning on every spawn. The published `@pnpm/exe` manifest already declares the module type, so this only affected installs made through the standalone script. Fixed in both stacks. The manifest construction moves into `standaloneManifest` / `standalone_manifest`, so it can be asserted without running the global install the surrounding function performs. |
||
|
Zoltan Kochan
|
dbe2dad610 |
feat(pacquet): ship node-gyp beside the native binary (#13586)
pnpm v12 shipped no node-gyp, so any package that shells out to it during an install script died with `spawn node-gyp ENOENT`: `node-gyp-build` with no matching prebuild, `node-pre-gyp`, a plain `"install": "node-gyp rebuild"`, and the `node-gyp rebuild` script pnpm synthesizes for a package that ships a `binding.gyp` without one. Embedders worked around it by putting their own node-gyp on the lifecycle PATH. pnpm 11 solves this without a JS bundle: node-gyp is declared `external`, and `pnpm deploy` resolves its whole tree from this repo's lockfile at release time into a `dist/node_modules/node-gyp` directory that sits beside the entry point, with `dist/node-gyp-bin` holding the wrappers that go on a lifecycle script's PATH. Those are ordinary files next to the executable, so the native binary can ship them the same way — nothing about the mechanism depended on there being a bundle to hide node-gyp inside. Resolving the tree at release time rather than at install time is the point, not an implementation detail: the dependency tree is frozen and reviewed per pnpm release instead of being fetched from the registry at the moment an install script is about to execute arbitrary code. scripts/bundle-node-gyp.mjs builds the payload, reusing the TypeScript CLI's wrapper scripts so the two stacks cannot drift on how a script's node-gyp is dispatched, and refusing to produce a partial dist/ — a PATH entry that resolves nothing is worse than shipping none at all. Both wrappers ship it, since each places the binary next to itself. node-gyp moves to a catalog entry so both stacks pin one version; the Node.js-floor hold-back note moves with it. At runtime bundled_node_gyp_bin finds the wrapper dir relative to the executable and returns None when nothing was shipped, which is what a checkout build sees — lifecycle scripts then fall back to whatever node-gyp the environment provides, exactly as before. The wrapper itself honors npm_config_node_gyp and only falls back to the shipped copy when it is unset, so a user-supplied node-gyp, a workspace one, and a package's own dependency all keep winning. Refs https://github.com/pnpm/pnpm/issues/13580 |
||
|
Zoltan Kochan
|
c2b7cfb930 |
feat(napi): add returnListOfDepsRequiringBuild install option (#13532)
The napi install result populated depsRequiringBuild from the ignored- builds accumulation only, so an embedder that allows dependency builds to run (Bit) always received an empty list and wiped its recorded bit.depsRequiringBuild lockfile block on every install. Mirror the TypeScript engine's returnListOfDepsRequiringBuild option: when set, the fresh-resolve path reports every non-skipped package whose files carry install scripts, regardless of the allow-build policy, through a sink threaded into the install pipeline. Installs served from the frozen-lockfile path leave the field undefined so the embedder keeps its previously recorded list, matching the TS engine, which only computes the list from a fresh resolve's fetch results. |
||
|
Neil de CarteretandZoltan Kochan
|
3729d83d4c |
fix: approve git-hosted tarball builds by repository url (#12985)
Normalizes git-hosted tarball dep paths back to the canonical `name@git+https://host/org/repo.git` key that a clone of the same repository produces, so one hashless entry approves the package whether pnpm clones it or downloads a tarball. GitHub (codeload), GitLab archive, and Bitbucket download URLs are covered. The host is part of each derived key, and the GitHub and Bitbucket download hosts are matched against a literal value (GitLab's is captured generically to allow self-hosted instances), so a look-alike host cannot be rewritten into an unrelated repository key. Approving or denying a specific resolved commit by its full tarball dep path continues to work. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Abdullah Alaqeel
|
d47ab916ad |
fix(self-update): stop the project config from steering the pnpm download (#12813)
Fixes pnpm/pnpm#12803. `pnpm self-update` resolved the pnpm download through the project's own registry/auth config, loaded the repo's default `.pnpmfile.(c|m)js`, and took its `minimumReleaseAge` policy from the active workspace — so the outcome of a global tooling operation depended on the directory it ran in, and on config a checkout controls. Route the fetch through the trusted package-manager bootstrap config — the channel `switchCliVersion` already uses, which excludes the project `.npmrc` and workspace manifest — and stop auto-loading the repo pnpmfile, whose `updateConfig` hook and custom resolvers/fetchers reach the same requests. `getPackageManagerBootstrapConfig` moves into `@pnpm/config.reader` so the command package can reuse it. Stop reading the project's `minimumReleaseAge` and `trustPolicy` settings, and its `ci` flag, for self-update. Each is dangerous in both directions for a command that replaces the global binary: a cooldown lowered waives the protection the user configured, raised it pins the machine to the installed pnpm, including past a release that fixes a vulnerability in it; a trust policy turned off accepts a release whose evidence the user meant to reject, turned on blocks the update the same way; and `ci` decides whether an immature pick may be confirmed at the keyboard at all. Unlike a blocked dependency upgrade, those decisions follow the user into every other project. The policies come from the built-in defaults, the global config, the environment, and CLI flags instead. Other commands keep reading them from `pnpm-workspace.yaml`, and there is no new default. When an immature version is refused, an interactive run offers to update anyway, matching how a strict install prompts; non-interactive runs still fail closed. Mirrored in pacquet: `Config::current_for_self_update` skips the same settings, and `self-update` gained the matching prompt and error. |
||
|
Zoltan Kochan
|
29efb4ae49 |
fix: honor same-second edits on whole-second-mtime filesystems (#12975)
The optimistic repeat-install fast path and verify-deps-before-run decide whether a package.json, .pnpmfile.cjs, or patch file changed since the last install by comparing its mtime against the last-validated timestamp with a strict `>`. On a filesystem that records mtimes at whole-second resolution (ext4 with 128-byte inodes, HFS+, some CI runner disks) a file edited in the same second as the install gets an mtime that rounds down below the timestamp, so the edit looks unchanged, the fast path short-circuits, and re-resolution is skipped — the second install is a no-op and keeps the stale result. Detect a whole-second mtime (no sub-second component) and treat its whole second as possibly-after the reference, falling through to the authoritative content check instead of trusting the rounded-down mtime. Erring toward "modified" only ever runs the content check; it never skips a needed install. On sub-second filesystems the mtime carries a fractional part, so the comparison stays exact and the common case is completely unaffected — the existing deps-status unit tests pass unchanged. Landed in both stacks: pacquet's optimistic_repeat_install and pnpm's checkDepsStatus, with a regression test on each side that forces a whole-second mtime so the coarse-filesystem path is exercised on any filesystem. |
||
|
Zoltan Kochan
|
0dfce95b76 |
feat(release): compose changelogs at publish time (registry storage) (#12971)
Implement RFC 0006 `changelog.storage: registry` and make it the default: no CHANGELOG.md is committed. A release's section is parked under .changeset/changelogs/ at `pnpm version -r` time and composed into the published tarball at publish, on top of the previous published version's changelog (highest published version semver-lower than the one being published — never a dist-tag lookup, so concurrent lines chain correctly). Consumed intents are garbage-collected by a later `pnpm version -r` only once the registry confirms the ledgered version is published and its tarball's CHANGELOG.md carries the composed section; a foreign-published version keeps the intent instead of losing it. The pure `versioning` crate/package stays network-free: the publish check is injected (a callback in TS, a confirmed-key set in Rust) from the command layer, which already has registry access. The parked-section store captures dependency-update lines and exact formatting that can't be recomposed from intents at publish time. `versioning.changelog.storage: repository` keeps the previous committed CHANGELOG.md behavior. Landed in both stacks. |
||
|
armanandZoltan Kochan
|
b46f96165f |
fix: kill spawned process trees with taskkill on Windows error exits (#12925)
Main thread panicked: begin > end (105 > 28) when slicing `@pnpm/npm-lifecycle@1100.0.0(patch_hash=e3541c…)(supports-color@10.2.2)` at pacquet/crates/resolving-deps-resolver/src/resolve_peers.rs:2490:51 --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
826366b7bc |
feat(releasing): native workspace release management in both stacks — change intents, release plans, per-package release lanes (#12953)
Implements phases 1-2 of the native monorepo versioning plan (https://github.com/pnpm/pnpm/issues/12952, RFC https://github.com/pnpm/rfcs/pull/18) in both stacks. TypeScript: new @pnpm/releasing.versioning package (intent reader/writer for changesets-compatible .changeset/*.md files with the additive 'none' decline; the committed per-package consumed-intents ledger .changeset/ledger.yaml; the release-plan assembler with direct bumps, dependent propagation via materialized workspace: ranges under real semver semantics, fixed groups, ignore, maxBump enforced against the real version distance, --filter narrowing, per-package release lanes with cumulative stable-target escalation and graduation; the applier with format-preserving version-only manifest updates, changelog composition in repository storage mode, ledger append, and intent GC with the prerelease retention exemption). CLI: pnpm change / change status; bare pnpm version -r (--dry-run, the new root pnpm lane command manages versioning.lanes through the format-preserving workspace manifest writer. The versioning key is typed on PnpmSettings/Config and validated by the workspace manifest reader. Rust: new pacquet-versioning crate mirroring the engine (same file formats, error codes, messages, and output), versioning settings on WorkspaceSettings/Config, and new change/version/lane commands in pacquet-cli. The npm-style pnpm version <bump> forms remain unported in pacquet (they did not exist there before) and error with a clear message. The two deploy install futures are boxed because Config grew past clippy's large-future threshold. Both stacks' engine test suites mirror each other one for one; integration tests drive the real binaries through record -> plan -> release -> lane -> graduation. |
||
|
Alessio AttilioandZoltan Kochan
|
411bbe89ff |
feat(registry-access): implement team command in both TypeScript and Rust stacks (#12789)
The team command communicates with the registry through the standard npm team API endpoints. The scope:team format is parsed to separate the organization scope from the team name. For mutation subcommands (create, destroy, add, rm) the registry URL is resolved per scope from the registries map with an optional --registry override, the auth header is resolved from the configured credentials honoring scoped credentials, and the request is sent with retry support and bounded response reads. When an OTP is in play, the Rust side restricts redirects to the configured registry origins so the npm-otp header cannot leak to another host; the TypeScript fetch layer already strips it on cross-host redirects. The ls subcommand dispatches to listing teams within an org when given @scope and to listing members of a specific team when given @scope:team. Output supports three modes, the default human-readable listing, --parseable which emits newline-delimited names, and --json which emits structured arrays.
On the TypeScript side the command is registered in pnpm/src/cmd/index.ts and removed from the notImplemented list. On the Rust side it is added as a CliCommand variant, routed in dispatch, and dispatched in dispatch_query. Both sides include comprehensive tests covering all subcommands, error paths for 401 403 404 and 409 responses, empty results, and the three output formats.
pnpr serves the npm team API from each hosted registry's config-declared teams: GET /-/org/{scope}/team and GET /-/team/{scope}/{team}/user list teams and members, gated by the registry-level access with denials masked as not-found, while team mutations answer an explicit 403 since pnpr teams are config-managed.
@pnpm/cli.parse-cli-args no longer stops option parsing at an escape word (create, exec, test) that appears as another command's parameter, which previously made pnpm team create drop a trailing --registry option.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
|
||
|
|
a8ad82d4dd |
fix: register pn alias in generated completions (#12861)
Register the pn short alias in generated shell completion scripts. pnpm exposes pn as a binary alias, but completion generation only registered pnpm with the supported shells. This meant zsh completions generated by pnpm completion zsh registered pnpm only, so pn did not receive the same completion function. Post-process the tabtab-generated scripts to register pn alongside pnpm for bash, fish, pwsh, and zsh, and cover each shell in the completion generator tests. Fixes pnpm/pnpm#11955. --------- Co-authored-by: ychampion <ychampion@users.noreply.github.com> Co-authored-by: Zoltan Kochan <zoltankochan@gmail.com> |
||
|
Zoltan Kochan
|
de1371daa9 |
feat(pacquet): add Node API bindings for the Rust engine (#12822)
Add `pacquet-napi`, a napi-rs cdylib crate, plus its `@pnpm/napi` npm wrapper, exposing pacquet's programmatic engine surface to Node.js so programmatic pnpm consumers can drive the Rust engine instead of the TypeScript pnpm packages. Bit is the reference consumer. Exports: install (in-memory importers, single and multi-importer workspaces, a synchronous readPackage hook per resolved dependency manifest, build-script approval, and depsRequiringBuild), rebuild, resolveDependency, pack, parseBareSpecifier, engineVersion, and auth via authHeaderByUri. The install runs on a dedicated 32 MiB-stack worker thread with its own tokio runtime; the napi async fn awaits the result over a oneshot channel, so pacquet's borrowed State never crosses the FFI boundary. A reporter bridge forwards the engine's wire-compatible log events to a JS callback, and errors carry pnpm's ERR_PNPM_* code and hint through a structured envelope. Engine-core changes are minimal and inert for existing callers: PackageManifest::from_value, and two Option fields on Install (pnpmfile_hook_override and workspace_projects_override) defaulted to None everywhere. A new napi-release profile sets panic = "unwind" since the workspace release profile uses panic = "abort". getPeerDependencyIssues is stubbed pending pacquet's own peer-issue renderer. Distribution follows the @pnpm/exe.* model via scripts/generate-packages.mjs. |
||
|
|
322f88f4f1 |
fix: avoid mutating unrelated deps on failed optional updates (#11373)
updateProjectManifest previously reconstructed the pairing between each resolved direct dependency and the wanted dependency it came from — first by alias, then by specifier shape, then by array position. Every one of those is fragile: a failed optional dependency drops out of directDependencies and shifts a positional pairing (#11267), and an aliasless selector that resolves to an alias already in the manifest matches the stale aliased entry instead of the request. Carry the originating wanted dependency on each resolved direct dependency (PkgAddressOrLinkBase.wantedDependency -> ResolvedDirectDependency.wantedDependency), set where the resolver builds the result and already has it in scope, and have updateProjectManifest read rdd.wantedDependency directly. This removes all the alias/position/specifier heuristics (and the earlier normalizeGitHubBareSpecifier band-aid) and fixes both correctness gaps structurally. Adds unit tests for the failed-optional (#11267), alias-collision re-add, and aliasless-optional-failure cases. Closes #11267 --------- Co-authored-by: cyphercodes <cyphercodes@users.noreply.github.com> Co-authored-by: Hermes Agent <hermes@example.invalid> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
|
d3f68e2aa4 |
fix(audit): compute reachable vulnerabilities with Tarjan SCC (#12467)
`pnpm audit` enumerates the install paths to every vulnerable package. The reachability-based pruning added in 11.5.1 (pnpm/pnpm#12087) lets the walker skip subtrees that reach no unsaturated finding by precomputing, per node, the set of vulnerabilities reachable from it. That getter only memoised acyclic subtrees: a node whose subtree contained a cycle was `complete === false`, and so was every ancestor up to the importer roots. None of them were cached, so their reachable set was recomputed on every query. Real dependency graphs commonly contain cycles, and a single cycle high in the graph makes a large fraction of nodes non-memoisable, yielding an O(N^2) walk. This matched the report in pnpm/pnpm#12212 exactly (CPU-bound, identical audit output across versions). Reachability is now computed with Tarjan's strongly-connected-components algorithm. Every node is scanned once; all members of an SCC reach the same set of vulnerabilities and share one set, finalised in reverse-topological order. Cyclic graphs are handled in O(N + E). The reachable set is used only to prune, so it must never under-approximate (that would hide a real finding). Tarjan yields the exact set for every node, so no finding can be dropped, and the path-recording logic is unchanged. The getter returns ReadonlySet<string> so the shared sets cannot be mutated by callers, and a missing memo entry (an impossible-by-construction state) throws rather than silently returning an empty set. A regression test asserts the read-count growth ratio between two cycle sizes (L=200 and L=400) is sub-quadratic: the fix scales ~2x (linear), the previous code ~4x (quadratic). Asserting the ratio cancels the per-node constant, so the test is not brittle to constant-factor changes. Closes pnpm/pnpm#12212. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
c86559c9bc | docs: add new silver sponsor (#12473) | ||
|
Hiten ShahandZoltan Kochan
|
1b02b47669 |
fix: remove macOS Gatekeeper quarantine xattr from native binaries (#11095)
Fixes #11056 ## Problem On macOS, pnpm imports files from its content-addressable store into `node_modules` via copy, reflink/clone, or hardlink. All three preserve extended attributes, including `com.apple.quarantine`. If a store blob carries that xattr — e.g. it was first written under a Gatekeeper-enabled app such as a Git client (`LSFileQuarantineEnabled=YES`) — the quarantine propagates into `node_modules`. Gatekeeper then blocks ad-hoc-signed native binaries (`.node`, `.dylib`, `.so`) from loading, even though pnpm has already verified each file's integrity against `pnpm-lock.yaml`. ## Solution After importing a package from the store, strip `com.apple.quarantine` from its native binaries — mirroring Homebrew's behaviour of dropping quarantine from downloads after checksum verification. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Abdullah AlaqeelandZoltan Kochan
|
61969fbddf |
fix(deps-status): detect lockfile-only changes (#12106)
## Summary Fixes `pnpm install` with `optimisticRepeatInstall` incorrectly returning `Already up to date` when `pnpm-lock.yaml` changed but project manifests did not. Fixes #12100. ## Root Cause `checkDepsStatus` used modified manifest mtimes as the only signal for whether it needed to validate dependency status. If no manifest was newer than `workspaceState.lastValidatedTimestamp`, it returned `upToDate: true` before checking whether the wanted lockfile had changed. That skipped lockfile validation for workflows like: - `git checkout HEAD~1 -- pnpm-lock.yaml` - restoring only `pnpm-lock.yaml` from a stash - external tools rewriting the lockfile without touching manifests ## Changes - Check wanted lockfile mtimes before taking the optimistic fast path. - If any wanted lockfile is missing or newer than the workspace state timestamp, validate all projects instead of only modified manifests. - Add a regression test proving a lockfile-only change does not skip wanted-lockfile validation. - Add a patch changeset for `@pnpm/deps.status` and `pnpm`. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
TruffleandZoltan Kochan
|
6d35338691 |
fix: detect changes inside file: dependencies on repeat install (pacquet + pnpm) (#12317)
## Summary - `pnpm install` reports "Already up to date" after edits inside a `file:` dependency's directory or after repacking a `file:` tarball. This is a v11 regression from the `optimisticRepeatInstall` default flip in pnpm/pnpm#11158. Fixes pnpm/pnpm#11795. - `checkDepsStatus` gains a `treatLocalFileDepsAsOutdated` option: when set, any project manifest declaring a local file dependency makes the check report not up to date. `installDeps` sets it on the optimistic fast path, so projects with local file dependencies always run a real install, which refetches those dependencies (the v10 behavior). - The predicate covers `file:` specs, path-prefixed specs (`./`, `../`, `~/`, absolute POSIX paths, and Windows drive paths including drive-relative ones like `C:dir`, matching the local resolver's `isFilespec`), and bare tarball file names (`vendor/pkg.tgz`). It is deliberately narrower than the local resolver's bare-path matching: a bare `user/repo` is statically indistinguishable from a git shorthand at this layer, and matching it would kill the fast path for every project with git dependencies, so protocol-carrying and URL specs stay on the fast path. - `pnpm.overrides` entries are scanned with the same predicate: an override mapping to a local file spec redirects every matching dependency in the graph to that directory, so it has the same blind spot as a direct local file dependency. Registry and `link:` overrides keep the fast path. - The option is caller-scoped on purpose. `verifyDepsBeforeRun` also consumes `checkDepsStatus`, and treating `file:` deps as always stale there would force a reinstall before every `pnpm run`. Its behavior is unchanged, and a regression test pins that. - pacquet port in the same commit: `check_optimistic_repeat_install` bails unconditionally on `file:` specifiers, because its only caller is the install command, the one consumer that sets the flag upstream. `link:` specifiers are excluded on both sides: they are symlinked, so changes inside them flow through without a reinstall. ## Why Both branches of `checkDepsStatus` are blind to content changes inside a `file:` dependency. The workspace branch exits early with `upToDate: true` when no project manifest's mtime moved, without ever reaching `linkedPackagesAreUpToDate`. The non-workspace branch exits at the manifest-vs-lockfile mtime gate the same way. Editing a source file inside a `file:` dependency bumps neither, so the fast path can never see it; the fix has to bail before those gates rather than refine them. This is the fix shape (a) I proposed in my diagnosis on the issue thread ([comment](https://github.com/pnpm/pnpm/issues/11795#issuecomment-4504177744)): the cost is a full resolution on repeat installs only for projects that declare `file:` dependencies, which is exactly what v10 did. The manifest-only comparison in `@pnpm/lockfile.verification` (`allProjectsAreUpToDate`) is intentional for the install-proper path and asserted by its tests, so this PR leaves it untouched. ## Checks - `pnpm --filter @pnpm/deps.status test test/checkDepsStatus.test.ts` (31 passed, 13 new) - `pnpm --filter @pnpm/deps.status run compile` and `pnpm --filter @pnpm/installing.commands run compile` (tsgo + eslint clean) - `cargo test -p pacquet-package-manager optimistic_repeat_install` (51 passed, 7 new; run in a rust:1.95.0 container) - `cargo fmt --check -p pacquet-package-manager` - `RUSTDOCFLAGS="-D warnings" cargo doc -p pacquet-package-manager --no-deps` --- Written by an agent (Claude Code, claude-fable-5). --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Victor Sumner
|
61810aa684 |
feat: add --frozen-store for installs against a read-only store (#12190)
## What
Adds an opt-in `frozenStore` / `--frozen-store` setting (default `false`) that lets `pnpm install --offline --frozen-lockfile` run against a package store that lives on a **read-only filesystem** — a Nix store, a read-only bind mount, an OCI layer.
## Why
A normal install fails against such a store **not** because it writes package content, but because it unconditionally:
1. opens the SQLite `index.db` in WAL mode, which needs to create `-shm`/`-wal` sidecars in the store directory; and
2. writes a project-registry entry under the store.
Both fail with `attempt to write a readonly database` / `EROFS` on a read-only store directory, even when the store is complete and the lockfile is frozen. This blocks any deployment that wants an immutable, content-addressed store.
## How
When `frozenStore` is enabled, pnpm opens `index.db` through the SQLite **`immutable=1`** URI — which tells SQLite the file cannot change underneath it, so it bypasses the WAL/`-shm` sidecar machinery entirely and reads the raw file with zero sidecar creation — and suppresses every store-write path. Pair it with `--offline --frozen-lockfile` against a fully-populated store. It is incompatible with two settings that would write into the store, and each throws a clear config-conflict error before any network or store access: **`--force`** (which bypasses the no-write-on-hit skip → `ERR_PNPM_CONFIG_CONFLICT_FROZEN_STORE_WITH_FORCE`) and a configured **pnpr server** (which would fetch and write packages into the store → `ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR`).
> Plain `SQLITE_OPEN_READ_ONLY` is **not** sufficient: opening a WAL-mode db read-only still tries to create the `-shm` sidecar, which fails on a read-only *directory*. `immutable=1` is the load-bearing piece.
> **Node.js requirement:** `node:sqlite` only passes `SQLITE_OPEN_URI` to SQLite (so the `immutable=1` query is honored rather than treated as part of a literal filename) starting in **v22.15.0** (22.x line), **v23.11.0**, and every **v24+**. pnpm's `engines` floor is `>=22.13`, so on a runtime older than that the frozen open is detected up front and fails with a clear `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` instead of SQLite's cryptic "unable to open database file". (pacquet uses rusqlite with an explicit `SQLITE_OPEN_URI` flag, so it has no such floor.)
> The `immutable=1` URI path is also percent-encoded (`%`→`%25`, `?`→`%3f`, `#`→`%23`, in that order, leaving `/` literal) so a store path containing those characters doesn't truncate the path or inject a spurious query parameter — applied identically in both stacks.
### Build backstop under the global virtual store
Under the global virtual store (default), a package's directory lives **inside** the store (`{storeDir}/links/...`). Applying a patch or running an allowlisted lifecycle script writes into that directory — so on a frozen store it would crash mid-build with a raw `EROFS`. A fully-seeded store never reaches the build step (patched/built packages are imported from the side-effects cache and filtered out by the `isBuilt` gate), so any residual build candidate means the seed is **missing that package's build output**.
`buildModules` now refuses up front with `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` and actionable guidance ("rebuild the seed with their scripts enabled, or remove them from `onlyBuiltDependencies`") instead of failing cryptically once a script starts. The check is gated on the global virtual store — under the isolated linker, slot directories live in the writable project-local store, so builds there are fine. Non-allowlisted scripts never run, so they are not treated as a blocking write.
Bin-linking has its own read-only-store edge under the global virtual store. On a **warm** checkout (the project's `.bin/<name>` already points at the seed target) `linkBin` returns before touching the store, so it is write-free. But on a **fresh** checkout it (re)creates the bin and calls `fixBin`, whose `chmod` targets the bin's **source file inside the store** (`{storeDir}/links/...`) — which is refused with `EPERM`/`EACCES` on a read-only store, even though a complete seed already ships that bin executable (so the `chmod` is redundant). `@pnpm/bins.linker` now wraps that call in `ensureExecutable`: it swallows the refusal when the target is already executable and rethrows otherwise, so bin-linking is write-free against a frozen store on a cold checkout too, while a genuinely non-executable bin (a broken seed) still surfaces as an error.
The blocking predicate distinguishes the two write kinds: a **patch** is applied regardless of `ignoreScripts`, so a patched package is always blocked; a **lifecycle script** is suppressed under `ignoreScripts`, so an allowlisted build-requiring package is *not* blocked when scripts are off (it would write nothing). This avoids falsely rejecting a valid `--ignore-scripts` frozen install. **Optional dependencies are exempt**: a build or patch failure on an optional dependency is non-fatal at runtime, so a seed missing an optional package's build output skips that build (emitting the `skipped-optional-dependency` log) instead of blocking the install — in both stacks.
### Both stacks (parity rule)
**TypeScript pnpm CLI**
- Config plumbing (`@pnpm/config.reader`): `frozen-store` type, config-file key, default, `Config.frozenStore`.
- The read-only open branch (`@pnpm/store.index`): `immutable=1`, read-only statements, throwing mutators.
- Wiring through `@pnpm/store.controller` and `@pnpm/store.connection-manager` to the sole `StoreIndex` construction site.
- Gating the project-registry write (`@pnpm/installing.context`).
- The `--force` / pnpr-server conflict guards (`@pnpm/installing.deps-installer`) and CLI surface (`@pnpm/installing.commands`).
- **After-install rebuild** (`@pnpm/building.after-install`): the post-install rebuild opens its `StoreIndex` immutably under the flag, so re-reading the store for a rebuild never attempts a writable open against the frozen store.
- **Worker fix:** `@pnpm/worker` opens its *own* writable `StoreIndex` on every `readPkgFromCafs` cache hit, so a pure read crashed on a frozen store. `frozenStore` is threaded through to `getStoreIndex` and keyed into its connection cache.
- **Build backstop** (`@pnpm/building.during-install`): `buildModules` throws `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` for a GVS slot that would build/patch on a frozen store, honoring `ignoreScripts`; threaded from `@pnpm/installing.deps-installer`.
- **Bin-linking on a read-only store** (`@pnpm/bins.linker`): `linkBin` wraps the `fixBin` chmod in `ensureExecutable`, which tolerates `EPERM`/`EACCES` when the bin's store-resident source is already executable (a complete seed) and rethrows otherwise — so a fresh checkout against a frozen store links bins without crashing on the redundant chmod. Catch-on-failure keeps the writable hot path at zero added syscalls.
**Rust pacquet**
- A dedicated `open_immutable` / `shared_immutable_in` opens via `immutable=1`, selected only under the flag. Plain `open_readonly` keeps the ordinary `SQLITE_OPEN_READ_ONLY` open (WAL locking intact) because normal installs read the index while the same process's `StoreIndexWriter` writes it concurrently — an immutable connection skips all locking and change detection, so a concurrent writer would make those reads undefined.
- `--frozen-store` CLI flag + `frozenStore` workspace-yaml setting.
- The store-index writer is replaced with a drain-and-drop stub (`spawn_disabled`) and `init_store_dir_best_effort` is skipped under the flag.
- **Build backstop:** `build_modules` returns `BuildModulesError::FrozenStoreNeedsBuild` (`ERR_PNPM_FROZEN_STORE_NEEDS_BUILD`) under the same GVS + frozen-store condition, threaded from `config.frozen_store`. The gate keys off `should_run_scripts` (which already folds the allow-build policy), so it is correct without an explicit ignore-scripts branch — pacquet has no configurable ignore-scripts mode yet.
pacquet already separated read-only index access (`shared_readonly_in`, or `shared_immutable_in` under the flag) from writes, so it never had the worker-conflation bug; the flag makes the "no writes attempted" contract explicit and gates the remaining best-effort write attempts.
## Testing
- **TS:** `store.index` frozen-mode-on-`0555`-directory test (reads work, writes throw `ERR_PNPM_FROZEN_STORE_WRITE`) plus a path-with-`?` open test — both gated on the runtime's immutable-URI support, with a complementary test asserting `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` fires where that support is absent (the CI Node 22.13.0 path); `config.reader` round-trip; `deps-installer` `--force` and pnpr-server conflict guards; `worker`/`package-requester` unit tests. End-to-end on a `chmod -R 0555` store: install succeeds, `node_modules` materializes, no `-shm`/`-wal`/`-journal` sidecars; negative control without the flag fails as expected; incomplete store → clean offline error.
- **pacquet:** `open_immutable_reads_wal_db_on_readonly_directory` unit test plus the `immutable_sqlite_uri` encoding test and a path-with-`?` open test; yaml + CLI fold tests; **integration test** `frozen_store_installs_against_a_read_only_store` — primes a store, `chmod 0555` the tree, runs `install --frozen-lockfile --frozen-store --offline`, asserts success + materialized `node_modules` + zero sidecars. Confirmed load-bearing by reverting the `immutable=1` fix (test then fails).
- **Build backstop (both stacks):** `building/during-install` unit tests — approved-build-not-cached and patched-not-cached refuse; cached, non-allowlisted, and non-GVS cases pass through; **approved-build-under-`ignoreScripts` passes through while patched-under-`ignoreScripts` still refuses** — and the matching pacquet `build_modules` tests (`frozen_store_gvs_patch_not_seeded_refuses` + GVS-off / frozen-off controls). Each confirmed load-bearing by disabling the relevant guard and watching the corresponding test fail.
- **Bin-linking (`bins/linker`):** `ensureExecutable` tests with `fixBin` mocked to reject with `EPERM` — an already-executable bin source resolves (and `fixBin` is asserted called, so it isn't the warm skip-guard passing), a non-executable one rethrows `EPERM`. Confirmed end-to-end by running the built `linkBins` against a real `chflags uchg`-immutable store: the executable-seed case resolves and links the bin, the non-executable-seed control throws `EPERM`.
A changeset is included with `"pnpm": minor` and `"@pnpm/bins.linker": patch` (the read-only-store bin-linking fix).
|
||
|
Zoltan Kochan
|
52be454d57 |
fix: infer missing platform fields of optional dependencies from the package name (#12312)
* fix: infer missing platform fields of optional deps from the package name Some registries strip the os/cpu/libc fields (or just libc) from the version objects of the packuments they serve. Resolution then saw every platform-specific optional dependency as platform-unrestricted, so pnpm downloaded and installed the binaries of every platform regardless of supportedArchitectures, and wrote lockfile entries without the platform fields, which broke installs from that lockfile on every machine. Platform-specific binary packages encode their platform in the package name (e.g. @nx/nx-win32-arm64-msvc), so packageIsInstallable now fills the missing platform fields of an optional dependency from the name's tokens. Since every install path decides installability through that check before fetching, foreign-platform binaries are skipped without even downloading them, in fresh resolution and in headless installs with both node linkers alike. A package that declares no platform fields at all is treated as platform-specific only when an operating system is recognized in its name, so a generic name segment (such as 'arm' on its own) never gets a package skipped. Fixes https://github.com/pnpm/pnpm/issues/11702 Fixes https://github.com/pnpm/pnpm/issues/9940 * chore: add platform name tokens to the cspell dictionary * fix(package-is-installable): infer missing platform fields of optional deps from the package name Port of pnpm commit https://github.com/pnpm/pnpm/commit/34875b2d7c (PR https://github.com/pnpm/pnpm/pull/12312). Some registries strip the os/cpu/libc fields (or just libc) from the version objects of the packuments they serve, and lockfile entries written from such metadata lack the fields too, so every platform's binaries were installed regardless of supportedArchitectures. Platform-specific binary packages encode their platform in the package name (e.g. @nx/nx-win32-arm64-msvc), so the installability check now fills the missing platform fields of an optional dependency from the name's tokens: infer_platform_from_package_name + inferred_platform in pacquet-package-is-installable, applied inside package_is_installable (hoisted linker) and in compute_skipped_snapshots (isolated linker, with the check cache keyed by the snapshot's optional flag since the verdicts can differ). The any_installability_constraint fast path now also considers optional snapshots whose names infer a platform their metadata row does not declare, so the inference is reachable on lockfiles without any declared constraint. Same guard rails as upstream: declared fields always win (each field is filled only when missing — a missing libc alone is inferred, disambiguating -gnu vs -musl), and a package declaring no platform fields at all engages the inference only when an operating-system token is recognized in its name, so a generic name segment such as 'arm' on its own never gets a package skipped. Fixes https://github.com/pnpm/pnpm/issues/11702 Fixes https://github.com/pnpm/pnpm/issues/9940 * test: shut the metadata-stripping proxy down cleanly and forward the request method |
||
|
Zoltan Kochan
|
5aed1200ea |
feat: add musl binaries for pacquet and pnpr (#12316)
Summary: - add Linux musl binary package selection to the pacquet and pnpr npm shims - generate linux-x64-musl and linux-arm64-musl native npm packages with libc metadata - build musl Rust release targets for both pacquet and pnpr - update package docs and cspell entries for the touched workflow files |
||
|
Zoltan Kochan
|
3d50680eda |
fix(security): verify Node.js runtime SHASUMS OpenPGP signature (#12295)
Follow-up to #12292 (which verifies the **package-manager** binary). This closes the same class of gap for the **Node.js runtime**. When a repository requests a Node.js runtime — `devEngines.runtime: node@X` (with `onFail: download`, the default) or `useNodeVersion` — pnpm downloads and then executes a Node binary (it's used to run lifecycle / `run` / `exec` scripts). The download **mirror is repository-configurable** via `node-mirror:<channel>` (`nodeDownloadMirrors`) in project `.npmrc`, and the integrity comes from `SHASUMS256.txt` fetched **from that same mirror**. That's a circular check: a malicious mirror serves a tampered `node` tarball **and** a matching `SHASUMS256.txt`, the sha256 check passes, and pnpm runs the binary. Drive-by on a normal command in a cloned repo. ## Fix pnpm now fetches `SHASUMS256.txt.sig` and verifies its **detached OpenPGP signature** against the **Node.js release team's public keys, embedded in the pnpm CLI**, before trusting the hashes. A mirror that serves a tampered binary cannot also produce a valid signature, so verification fails. Any faithful mirror (one that proxies the real signed SHASUMS) keeps working. - `@pnpm/crypto.shasums-file`: new `fetchVerifiedNodeShasums` / `fetchVerifiedNodeShasumsFile` verify the signature via `openpgp` against the embedded keys. - The keys live in a generated file (`src/nodeReleaseKeys.ts`, 28 keys) mirrored from the canonical `nodejs/release-keys` list. `crypto/shasums-file/scripts/update-node-release-keys.mjs` keeps them current (`pnpm check:node-release-keys` / `--update`), and the **create-release-pr** workflow runs the check as a gate so a new release signer can't silently break verification. - `@pnpm/engine.runtime.node-resolver` verifies the **configurable-mirror** SHASUMS. The hardcoded `unofficial-builds.nodejs.org` musl mirror is **not** repo-configurable and is signed by a different key, so it stays trusted over TLS. ## Scope - **Pre-release channels (rc, nightly, …) are not verified** — Node only signs the `release` channel (no `SHASUMS256.txt.sig` exists for them, even on nodejs.org), so they remain unverifiable. Verification is gated on the `release` channel. - **Bun / Deno are unaffected** — their download/SHASUMS URLs are hardcoded to canonical GitHub (`github.com/oven-sh/bun`, `api.github.com/repos/denoland/deno`), not mirror-configurable, so a repo can't redirect them. - **Pacquet parity:** `pacquet/crates/engine-runtime-node-resolver` has the same mirror-configurable SHASUMS logic and needs the equivalent Rust port — tracked as a follow-up (per the repo's parity rule, opening the TS side first). |
||
|
Zoltan Kochan
|
b4cc602b6d | docs: update the list of sponsors (#12265) | ||
|
Zoltan Kochan
|
70554b8677 |
feat(pnpr): config-selectable networked-SQLite auth backend (#12199 phase 3) (#12206)
## What Implements the **auth half** of [#12199](https://github.com/pnpm/pnpm/issues/12199) (phase 3) — making pnpr's remaining per-instance state pluggable so the registry can run as stateless, horizontally-scaled replicas. ### Auth records behind config-selected backends Users + tokens now sit behind narrow async `UserBackend` / `TokenBackend` traits, built once at startup into `Arc<dyn …>` handles (the same build-once pattern #12198 used for the hosted store). Three implementations: - **Local** (default) — today's htpasswd file + SQLite token DB, or in-memory when no file is configured. Unchanged behavior. - **Networked SQLite (libsql / Turso)** — `LibsqlAuth` stores **both** records in one shared database, so several stateless replicas observe a consistent set of users and tokens. The `tokens` table DDL is shared verbatim with the local backend (a DB can migrate between them); users — which the local backend keeps in htpasswd — move into a `users` table. Selected via a new top-level YAML block: ```yaml backend: libsql: url: ${PNPR_LIBSQL_URL} authToken: ${PNPR_LIBSQL_TOKEN} # optional embedded replica for local-fast hot-path reads: replicaPath: ./auth-replica.db syncIntervalSecs: 60 ``` When the block is absent, auth stays on local disk exactly as before. ### Embedded-replica read acceleration Token lookups are on the request hot path, so against a remote primary every read would be a network round-trip. With `replicaPath` set, `LibsqlAuth` builds a libsql **embedded replica**: reads hit a local file that libsql keeps current in the background; writes go to the primary. `syncIntervalSecs` is the freshness knob that bounds token-revocation lag. ### Async access path `identify` / `enforce_access` are now async (a networked lookup is async). `enforce_access` is split into an async `resolve_identity` + a sync `authorize`, so the search endpoint resolves the caller once and authorizes each candidate synchronously (no async-in-`retain`). ### Concurrent-publish guard (cross-cutting follow-up from the issue) Closes the same-instance lost-update window in the three read-modify-write packument flows (publish, dist-tag change, partial-unpublish): a striped per-package lock serializes same-package writers on one instance while letting different packages proceed in parallel. The **cross-replica** half (S3 `If-Match` / ETag CAS) is documented in-code as the remaining piece — the issue files it under "fix when we get there," and it belongs with the multi-writer S3 publish work, not this auth branch. ## Tests All green — `cargo test -p pnpr`: - **176 lib unit tests** incl. new `LibsqlAuth` tests (run against an in-memory libsql DB — same driver + SQL, no server) and `backend.libsql` config-parsing tests (incl. the replica options). - New `concurrent_publishes_of_distinct_versions_all_survive` integration test for the publish guard. - Existing auth_persistence / auth_user_endpoints / auth_publish / server / s3_backend suites pass. - Clean under `cargo fmt`, `clippy`, `RUSTDOCFLAGS=-D warnings cargo doc`, **Dylint perfectionist**, and `taplo`. ## Docs `backend.libsql` (incl. embedded replica) documented in the bundled `config.yaml` and the `pnpr` npm README, mirroring how the S3 backend was documented in #12198. |
||
|
Ruben NogueiraandZoltan Kochan
|
4e740d5562 |
fix(gvs): run dependency build scripts under the global virtual store (#11987)
* fix: unavailable dep * fix(gvs): re-link lifecycle-script bins into the GVS projection The post-lifecycle bin re-link pass used the classic virtualStoreDir path even under the global virtual store, so bins created by the build scripts this fix runs were never re-linked into the GVS projection. Use the same pkgModulesDir helper as the rest of the rebuild path. Also thread enableGlobalVirtualStore through the (recursive) rebuild command opts explicitly, and list pnpm in the changeset. * chore: add prebuild to cspell dictionary * test(gvs): cover rebuilding a shared dependency across workspace projects Adds a multi-project recursive rebuild test under the global virtual store with per-project lockfiles (sharedWorkspaceLockfile: false), which routes through recursiveRebuild's per-project concurrent branch. Both projects depend on the same package, deduped into one shared GVS projection, so the concurrent passes select the same projection directory and exercise the per-projection build lock. Asserts the projection is deduped to one directory and built exactly once. * test(gvs): reword comment to satisfy spellcheck * docs: trim verbose comments to 2-3 lines --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
8e5e764037 |
feat(pnpr): store hosted packages in an S3-compatible object store (#12198)
## What
Lets pnpr store its **hosted** packages (the ones published to it, plus static-served content) in an **S3-compatible object store** instead of a local directory. Because the same code targets any S3-compatible endpoint, this also covers **Cloudflare R2**, MinIO, Backblaze B2, Wasabi, etc.
The local `tokio::fs` path remains the default — nothing changes unless you add the new `s3:` config block.
## Why
The hosted store is pnpr's source of truth: durable, must be backed up, and can't be regenerated. That's exactly what belongs in object storage:
- The provider handles durability/replication, so there's no single-node volume to back up.
- Multiple **stateless pnpr replicas** can share one hosted store.
- R2 is the S3 API, so a configurable `endpoint` gets it (and the other S3-compatibles) for free.
The disposable proxy cache and the install-accelerator SQLite stores deliberately **stay on local disk** — they're ephemeral, latency-sensitive, and streamed/locked in filesystem-shaped ways.
## How
- New `s3.rs` module: `S3Settings` (the YAML `s3:` block), a client builder (`object_store` crate; AWS-env credentials with explicit override, plus R2/MinIO/path-style/HTTP knobs), and an `S3Store` adapter (packument get/put, streaming tarball get, staged upload, delete, prefix-scoped list).
- `storage.rs`: a `HostedStore { Fs | S3 }` backend enum routes the hosted ops; the `cached` store stays fs-only. Publish stages the decoded+verified tarball to local scratch, then finalize either renames (fs) or uploads (S3). `open_tarball` now returns a streaming response body so S3 reads stream straight through.
- `config.rs`: parses `s3:` and builds the client once at config-load time (the only fallible step), so `Storage` construction stays infallible.
- `search.rs`: local search now lists package names through the storage abstraction, so it works against a bucket too.
- Documented (commented) in the bundled `config.yaml`.
### Example: Cloudflare R2
```yaml
storage: ./storage # still backs the local proxy cache + upload staging
s3:
bucket: my-pnpr-packages
region: auto
endpoint: https://<account-id>.r2.cloudflarestorage.com
accessKeyId: ${PNPR_S3_ACCESS_KEY_ID}
secretAccessKey: ${PNPR_S3_SECRET_ACCESS_KEY}
```
|
||
|
Zoltan Kochan
|
33921c8019 |
fix(publish): normalize string repository field to object form (#12109)
* fix(publish): normalize string repository field to object form
Some registries (e.g. Gitea) reject a string `repository` field with a
500 Internal Server Error, since they decode it into an object-typed
struct. npm normalizes a string repository into { type, url } before
publishing; pnpm did not, so `pnpm publish` failed where npm succeeded.
Close #12099
* chore: add Codeberg to cspell dictionary
|
||
|
Zoltan Kochan
|
c12681f68d |
docs(registry): flesh out @pnpm/pnpr README (#11972)
* docs(registry): flesh out @pnpm/pnpr README Document install, default behavior, CLI flags, RUST_LOG, and a minimal verdaccio-shaped YAML config example. * docs(registry): reword pnpr intro Drop the "runs locally / verdaccio-like" framing — pnpr is hostable and not scoped to a local-dev role. * chore(cspell): add packuments, refetched |
||
|
Zoltan Kochan
|
0f5f338a89 | chore: add pnpr to cspell | ||
|
Zoltan Kochan
|
a23956e3ab |
fix(config/reader): pin unscoped per-registry settings to their source's registry at load time (#11953)
* fix(config/reader): drop user-level default auth when workspace overrides registry
When a workspace `.npmrc` overrides `registry=` to a different value than the
user's `~/.npmrc` or `~/.config/pnpm/auth.ini` would have set, do not bind
unscoped/default credentials (`_authToken`, `_auth`, `username`/`_password`)
from the user-level config to the workspace-selected registry. The previous
behavior leaked user-trusted credentials to whatever registry an untrusted
workspace `.npmrc` pointed at. Reported by JUNYI LIU.
* chore(cspell): allow JUNYI in changeset and tests
* fix(config/reader): also defend when pnpm-workspace.yaml overrides registry
Move the rebind defense to after all config layers (CLI, env vars,
pnpm-workspace.yaml, .npmrc) have settled. Compare the final resolved
default registry against what the user-level config alone would produce,
and skip the check entirely if the user requested a registry via CLI/env
themselves.
* feat(config/reader): deprecate unscoped authentication credentials
Emit a per-file warning whenever an .npmrc or auth.ini contains an
unscoped auth value (_authToken, _auth, username, _password,
tokenHelper). URL-scoped tokens have been npm's recommended pattern
since npm@9, and unscoped credentials are slated for removal in a
future major. The warning fires independently of whether the rebind
defense rejects the credentials, so users see the deprecation even when
their setup happens to be safe today.
* refactor(config/reader): rescope unscoped credentials at load time instead of detecting rebinds post-merge
Each .npmrc / auth.ini / CLI source's unscoped credential keys
(_authToken, _auth, username, _password, tokenHelper) are rewritten to
their URL-scoped equivalent during load, using the same source's
registry= value (or the npmjs default if it declares none). A later
layer overriding registry= can no longer rebind a credential to its own
registry — the credential is already pinned to the URL its author
intended.
This removes the post-merge source-tracking defense and replaces it
with the simpler per-source normalization. Each rescope emits a
deprecation warning so users migrate to writing the URL-scoped form
directly.
* refactor(network/auth-header): drop empty-string default-registry slot
After load-time rescoping, no source can populate configByUri[''] —
every credential is either URL-scoped from the start or rewritten to
the URL-scoped form during the .npmrc / auth.ini / CLI parse. The
runtime fallback that re-keyed configByUri[''] onto the merged default
registry, and the publish-side fallback that read it, are both dead
code.
Removed:
- empty-string handling in getAuthHeadersFromCreds, including its
defaultRegistry parameter
- defaultRegistry parameter from createGetAuthHeaderByURI
- the corresponding dedicated unit test
- the configByUri['']?.creds fallback in publishPackedPkg.ts
- empty-key assertions in config/reader tests
Updated all ~16 call sites of createGetAuthHeaderByURI to drop the now
unused second argument.
* feat(config/reader): extend per-source rescoping to client TLS cert/key
The same trust-boundary issue that affected unscoped credentials applies
to client TLS settings: an unscoped cert=/key= would be presented to
whatever registry the merged config settles on, even if a later layer
(workspace .npmrc, pnpm-workspace.yaml, CLI flag) overrode it. The
existing rescope helper now also rewrites unscoped `cert` and `key`
to their URL-scoped form, pinning them to the registry their author
named in the same source.
`ca`/`cafile` are intentionally left unscoped: they're trust anchors,
not credentials, and corporate MITM-proxy setups depend on them
applying to every HTTPS request. The default-registry override can't
weaponize an unscoped CA — the attacker would need a cert signed by it.
`certfile`/`keyfile` (file-path variants) are not rescoped either:
`certfile` isn't read unscoped by pnpm today (asymmetric vs. `keyfile`
in NPM_AUTH_SETTINGS), and supporting only one of them would be
confusing. Users wanting the path form can write it URL-scoped
directly.
* chore(config/reader): remove dead unscoped `keyfile` allowlist entry
`keyfile` was listed in NPM_AUTH_SETTINGS so unscoped `keyfile=<path>`
passed the .npmrc filter and ended up in authConfig — but nothing in
the codebase ever read it from there. The dispatcher uses `opts.key`
(inline PEM) and `configByUri[host].tls.key` (URL-scoped path/inline
content), neither of which is populated from unscoped `keyfile=`.
`certfile` was already absent from the allowlist for the same reason,
so this also removes the asymmetry between the two file-path variants.
URL-scoped `//host/:certfile=...` and `//host/:keyfile=...` continue
to work via `tryParseSslKey` and are unaffected.
* test(network/auth-header): drop test for removed default-registry slot
This test exercised the configByUri[''] re-keying path that was
removed in the rescope-at-load refactor. With createGetAuthHeaderByURI
no longer accepting a defaultRegistry parameter and unscoped
credentials no longer reaching the merged config, the scenario the
test described is structurally unreachable.
* fix(config/reader): handle empty/invalid registry value in rescope
Two CI fixes:
1. When a source's `registry=` resolves to an empty string (e.g. an
unresolved `${ENV_VAR}` placeholder), `new URL(...)` inside
`nerfDart` throws. Guard the call with try/catch: drop the
unscoped per-registry keys (a bare token has nowhere safe to bind)
and emit a warning naming the offending source.
2. Update `.npmrc does not load pnpm settings` to expect the rescoped
form of unscoped `_authToken`/`username` in `authConfig` — they
now appear as `//registry.npmjs.org/:_authToken` etc. since the
test's .npmrc declares no `registry=` of its own.
* chore(cspell): allow "rescoping"
* test(installing/deps-installer): drop "legacy way" auth test
This test passed credentials via the configByUri[''] empty-string slot,
which the auth-header layer re-keyed to the merged default registry at
request time. That slot was removed in the rescope-at-load refactor —
credentials are now always URL-scoped before they reach configByUri,
so the empty-key entry is unreachable from any code path.
The scenario the test covered (basicAuth via username/password) is
already exercised by the existing "installing a package that need
authentication, using password" test using the URL-scoped form.
|
||
|
Alessio AttilioandZoltan Kochan
|
d7da112eea |
feat(pkg): implement native pnpm pkg command (#11512)
Implements `pnpm pkg` natively with `get`, `set`, `delete`, and `fix` subcommands. Workspace usage follows pnpm conventions: use `-r` / `--recursive` for all selected workspace projects, and `--filter` to narrow the selected project graph. This does not add npm-style `--workspace` or `--workspaces` flags. The PR also extends `@pnpm/object.property-path` with safe set/delete helpers used by the command. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Jovi De CroockandZoltan Kochan
|
508e6d800b |
feat: add pnpm stage command (#11863)
* feat: add npm stage command
* fix: correct stage command edge cases
* fix: handle stage error paths
* refactor: address stage command review feedback
- Type stageId via OtpPublishResponse so publishPackedPkg no longer needs a cast.
- Hoist fetchFromRegistry + auth header into a per-subcommand StageContext.
- Send npm-auth-type: web on all stage requests, not just approve/reject.
- Consolidate stageRequest / stageRequestWithOtp / stageJsonRequest into (context, params) form.
* fix: trigger stage OTP flow on web-auth challenges
The registry responds to stage approve/reject with 401 and a body of
`{ authUrl, doneUrl }` when the user must complete a browser-based
authentication, but `www-authenticate` does not contain "otp" in that
case. The previous check missed this and surfaced the response as a
generic STAGE_REGISTRY_ERROR. Detect web-auth responses by body shape so
withOtpHandling can drive the polling flow.
* test: cover stage approve web-auth detection paths
Add tests that lock in the OTP-trigger detection on the stage command:
- 401 with `{authUrl, doneUrl}` enters the web-auth flow, exercised here
via the full polling-completion path (registry returns a token, the
retry request carries it as npm-otp).
- 401 with web-auth body but no TTY surfaces as OTP_NON_INTERACTIVE.
- 401 without any OTP signals stays a STAGE_REGISTRY_ERROR so we don't
over-trigger the OTP flow on unrelated unauthorized responses.
* fix: keep web-auth and OTP wrapper on stage publish
Calling `context.publish()` directly for staged publishes bypassed
`publishWithOtpHandling`, so users without a preconfigured token had no
path through the browser-based authentication flow on `pnpm stage
publish`. Route the staged publish through the same wrapper as the
regular publish; `OtpPublishResponse.stageId` carries the registry's
identifier when set.
* refactor: split stage into per-subcommand files and lift tarball helpers
The 631-line `stage.ts` carried six subcommands, tarball parsing,
auth/request plumbing, error class, OTP detection, and rendering helpers
in one file. Reorganized to match the existing `publish/` folder layout:
- `stage/{index,help,publish,list,view,approve,reject,download,context,
request,parsing,rendering,errors,types}.ts` — one concept per file.
- `tarball/{publishSummary,summarizeTarball}.ts` — shared between
`publish` and `stage` instead of duplicated. `PublishSummary` and
`extractBundledDependencies` now live with the tarball helper rather
than inside the publish subfolder, so other commands can reuse them
without reaching into `publish/`.
Behavior unchanged. Also dropped `StageRegistryError`'s redundant
`statusCode` field (was identical to `status`) to bring it in line with
`FailedToPublishError`.
* chore: add TOTP and unparseable to cspell dictionary
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
|
||
|
Khải
|
fd564a43b9 |
ci(benchmark): consolidate (#11741)
Unify the two install-benchmark stacks (`benchmarks/bench.sh` and `pacquet/tasks/integrated-benchmark/`) into one shared Rust orchestrator. Scenario, fixture, workspace-manifest, install-script, and report generation live in one place so changes propagate to both stacks. Scenario sets per workflow are unchanged: `benchmark.yml` keeps measuring the same 6 scenarios; `pacquet-integrated-benchmark.yml` keeps measuring the same 2. Both verdaccio and live-npm registry modes are preserved; neither is removed in favor of the other. All six scenarios accept both `pacquet@<rev>` and `pnpm@<rev>` targets. |
||
|
Zoltan Kochan
|
963861cac1 |
perf(npm-resolver): layer abbreviated meta + attestation before full metadata in the minimumReleaseAge gate (#11704)
Follow-up to #11691 — item 2 from #11687, plus a related shortcut. ## What When the `minimumReleaseAge` lockfile verification gate needs to know when a version was published, it used to fetch a multi-MB full metadata document per package just to read one timestamp. This PR replaces that single-step path with a four-layer lookup that pays the cheapest viable source first: 1. **Abbreviated metadata's `modified` field** — the resolver already fetches this for resolution. If the package as a whole hasn't been modified within the policy cutoff, every version it contains is at least that old; return `modified` as a conservative upper-bound and skip the rest of the chain. 2. **Local `FULL_META_DIR` mirror** — exact per-version times if a previous verification populated it. 3. **npm attestation endpoint** (`/-/npm/v1/attestations/<name>@<version>`) — a tens-of-KB Sigstore bundle whose Rekor inclusion time stands in for publish time. Wins on cold cache when the package was published with provenance. 4. **Full metadata fetch** — last resort. ## Why The verification cache from #11691 made repeat installs against an unchanged lockfile effectively free. The remaining cost is the *first* verification on a fresh CI runner with no restored cache — particularly `pnpm install --frozen-lockfile`, where every locked package's publish timestamp has to be confirmed. Fetching the full metadata document for each package is wasteful when: - The resolver has usually already cached abbreviated metadata, whose `modified` field alone is enough to clear stable packages (the common case). - For recently-modified packages, the per-version attestation endpoint is orders of magnitude smaller than full metadata. ## How ### Abbreviated `modified` shortcut `fetchFullMetadataCached` is refactored to share an internal helper with a new `fetchAbbreviatedMetadataCached`. Both do conditional GETs against their respective on-disk mirrors. On a non-frozen install the abbreviated mirror is already populated by the resolver, so the shortcut hits the local cache at headers-only cost. On `--frozen-lockfile` the fetch is still cheaper than full metadata. If `Date.parse(modified) < cutoff`, return `modified` — it's an upper bound on every version's publish time in this package, and the verifier's `published < cutoff` check passes trivially. ### Attestation endpoint `fetchAttestationPublishedAt` (new module) hits `/-/npm/v1/attestations/<name>@<version>`, parses the response, and reads the earliest `bundle.verificationMaterial.tlogEntries[].integratedTime` across the attestation bundles. That's the Rekor inclusion time — a couple of seconds after publish, well within tolerance for a policy that operates in minutes/hours/days. Returns `undefined` on 404 / network error / malformed body so the caller falls back. ### Per-install dedup The lookup carries a `PublishedAtLookupContext` with four memo maps: abbreviated meta per (registry, name), local full meta per (registry, name), full meta network fetch per (registry, name), final published-at per (registry, name, version). Verifying many versions of the same package only pays the disk/network costs once. ## Trust model **No Sigstore signature verification on the attestation path.** The trust model is identical to reading the registry's `time` field on the full metadata document — we already trust the registry to serve correct publish timestamps for the gate's purpose. The win is purely bandwidth. Full Sigstore verification (Fulcio cert chain + Rekor inclusion proof) would harden the timestamp against a compromised registry. It pulls in the `sigstore` npm package and the TUF root — a separate dependency-surface discussion, parked as future work. ## Tests - **13 unit tests** in `resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts`: ISO timestamp extraction, URL construction (scoped + unscoped), 404 / 5xx / network error / malformed JSON / missing fields → undefined, earliest-of-multiple-attestations, defensive number-as-integratedTime, auth header forwarding, trailing-slash normalization. - Existing `minimumReleaseAge` + `verifyLockfileResolutions` integration suites (45 tests) still pass — the fallback chain preserves end-to-end behavior when the new shortcuts don't apply. |
||
|
Zoltan Kochan
|
5dc8be8a42 |
fix(graph-hasher): resolve GVS engine per-snapshot for runtime-pinned deps (#11693)
Closes #11690. A dependency that declares `engines.runtime` in its manifest carries the desugared `dependencies.node: 'runtime:<version>'` pin in the lockfile, and pnpm's bin linker spawns that dep's lifecycle scripts through the pinned Node downloaded into `<pkgDir>/node_modules/node/`. The GVS hash and the side-effects-cache key prefix were still anchored to the install-wide runtime — so the pinning snapshot's slot encoded the wrong Node major, and a reinstall on the same host could read the cached side-effects under a key whose `<platform>;<arch>;node<major>` triple disagreed with the Node the build actually ran on. Per-snapshot resolution now matches what `bins/linker` already does on a per-package basis: a snapshot's own pin wins; the install-wide value (from #11689's `findRuntimeNodeVersion`) is the fallback. ### TypeScript - `deps/graph-hasher/src/index.ts:72-77` — adds `readSnapshotRuntimePin(children)`: pulls the bare Node version from a graph node's `children.node` entry when that points at a `node@runtime:<version>` snapshot. Factors out a small `extractRuntimeNodeVersion(snapshotKey)` parser shared with `findRuntimeNodeVersion`. - `deps/graph-hasher/src/index.ts:115-116,245-246` — `calcDepState` and `calcGraphNodeHash` consult `readSnapshotRuntimePin(graph[depPath].children)` first and only fall back to the install-wide `nodeVersion` parameter when the snapshot doesn't pin its own Node. No caller changes required — install-wide fallback continues to be computed via `findRuntimeNodeVersion(Object.keys(graph))` at each call site. - **Refactor (separate commit):** `findRuntimeNodeVersion` moved from `@pnpm/engine.runtime.system-node-version` to `@pnpm/deps.graph-hasher` (along with the new `readSnapshotRuntimePin`). `system-node-version` is about probing the *host* Node — `getSystemNodeVersion`, `engineName`. The lockfile-shape parsers fit better next to the package that actually composes the engine string. Every caller already depended on graph-hasher, so no new deps; six packages drop the now-unused dependency on `system-node-version`. ### Pacquet - `pacquet/crates/package-manager/src/install_frozen_lockfile.rs:1309-1345` — new `find_own_runtime_node_major(snapshot)` reads a snapshot's `dependencies` for a `node` entry with `Prefix::Runtime`, returning the bare major. - `pacquet/crates/package-manager/src/virtual_store_layout.rs:178-205` — `VirtualStoreLayout::new` resolves engine per-snapshot inside the hash loop via `engine_name(own_major, None, None)` when the snapshot pins, otherwise inherits the install-wide `engine` argument. ### Migration Snapshots of dependencies that declare their own `engines.runtime` re-hash under that dep's pinned Node instead of the install-wide value. Old slots become prune-eligible on next install. |
||
|
Zoltan Kochan
|
763ddf1c99 |
chore(pacquet): wire pacquet workflows into monorepo (#11635)
* chore(pacquet): wire pacquet workflows into monorepo
Move Cargo workspace, Rust toolchain configs, justfile, composite actions,
and 7 workflow files out of `pacquet/` and up to the repo root so:
- cargo / just / taplo run from repo root, the way the rest of the
monorepo's tooling does
- GitHub Actions actually discovers the workflows (it only reads
`.github/workflows/` at the repo root)
Workflows are prefixed with `pacquet-` and renamed to "Pacquet ..." so
they don't collide with the existing pnpm CI. Path filters are scoped
to `pacquet/**` so they don't trigger on every commit. The cargo entry
from pacquet's standalone `dependabot.yml` is folded into the root one;
pacquet's `CODEOWNERS` and `pull_request_template.md` are dropped because
the root copies supersede them.
Path rewrites:
- `Cargo.toml` workspace members → `pacquet/crates/*`, `pacquet/tasks/*`
- all path-deps in `[workspace.dependencies]` → `pacquet/...`
- `justfile` recipes (`install`, `install-hooks`) point at `pacquet/...`
- `.taplo.toml` include globs → `pacquet/crates/*/*.toml`, `pacquet/tasks/*/*.toml`
- `pacquet/npm/pacquet/scripts/generate-packages.mjs` REPO_ROOT walks one
more level up
- workflow `paths:` filters, `hashFiles(...)`, and shell paths all updated
Verified: `cargo metadata` resolves the workspace, `cargo fmt --check`
clean, `taplo format --check` picks up all 26 Cargo.tomls, `actionlint`
reports no new issues (the `type:`-on-input warnings on the rustup action
predate this move).
* chore(pacquet): drop pnpm version pin from pacquet CI workflows
The monorepo's root `package.json` declares `pnpm@11.1.1` under
`packageManager`, which conflicts with the workflows' explicit
`version: 11.0.0-rc.5` and trips `pnpm/action-setup` ERR_PNPM_BAD_PM_VERSION.
The pin was a pacquet-era workaround for the v9 lockfile while pnpm 11
was still pre-release. Stable 11.x writes v9 too, so let action-setup
read the version from `packageManager` like every other workflow in
this repo does.
* chore(pacquet): use pnpm/setup matching the rest of the monorepo
Replaces `pnpm/action-setup@v6` with the same `pnpm/setup@b1cac3...`
SHA the rest of pnpm/pnpm uses (release.yml, test.yml, ci.yml,
benchmark.yml, audit.yml). Reads pnpm version from `packageManager`
in root package.json, and skips the implicit `pnpm install` since
pacquet does its own scoped install via `just install` (which only
touches `pacquet/tasks/registry-mock/`).
The release workflow now also installs Node via the same action
(`runtime: node@22`) instead of via `pnpm runtime -g set node 22`,
since pnpm/setup handles runtimes in one step.
* chore(pacquet): tighten permissions and Dependabot cooldown
Address zizmor warnings on the pacquet CI changes:
- `dependabot.yml`: the cargo entry I added in the previous commit
inherited from pacquet's standalone repo and is missing the
`cooldown: default-days: 7` the github-actions entry uses. Add it
so cargo and github-actions debounce updates consistently.
- `pacquet-ci.yml`, `pacquet-codecov.yml`, `pacquet-cargo-unused.yml`
lacked a top-level `permissions:` block, so GITHUB_TOKEN inherited
the repo default. Declare `contents: read` — every job in these
workflows only reads the repo and runs local checks.
The other four pacquet workflows already declare permissions
explicitly (integrated-benchmark/comment, micro-benchmark, release).
* chore(pacquet): add "reimagining" to cspell dictionary
cspell at the repo root scans all `**/README.md` and was rejecting
`pacquet/README.md` and `pacquet/npm/pacquet/README.md`, which describe
pacquet as "not a reimagining of pnpm." Add the word to the existing
allow-list rather than rewording two READMEs imported from a separate
repo.
* fix(pacquet): prefix workspace-relative paths with pacquet/
Two Rust source files looked up paths off the cargo workspace root
(\`cargo locate-project --workspace\`), which now resolves to the
monorepo root rather than the pacquet directory. Add the \`pacquet/\`
prefix:
- \`tasks/registry-mock/src/dirs.rs\` — \`registry_mock()\` was
pointing the node launcher at \`<repo>/tasks/registry-mock/launch.mjs\`
instead of \`<repo>/pacquet/tasks/registry-mock/launch.mjs\`, which
failed every Pacquet CI test job ("Cannot find module ...launch.mjs").
- \`tasks/micro-benchmark/src/main.rs\` — same idea for the
fixtures folder.
|
||
|
Zoltan Kochan
|
39e42665b3 |
fix(git-resolver): avoid encoded slash in GitLab tarball URL (#11551)
* fix(git-resolver): avoid encoded slash in GitLab tarball URL hosted-git-info's default GitLab tarball URL routes through `/api/v4/projects/<user>%2F<project>/...`. The `%2F` survives into the virtual store directory name (depPathToFilename only escapes raw `/`, not `%`), and Node refuses to import any module whose path contains an encoded slash. The same URL is also intermittently rejected by GitLab with a 406. Override the GitLab tarballtemplate to the `/-/archive/` URL, which works for both public and private repos and contains no encoded slashes. Closes #11533 * test: avoid cspell-flagged words * test: keep existing gitlab assertions, only add new ones Restore the skipped tests' original API-URL assertions; they document the old expected shape and weren't running anyway. Add the new `/-/archive/` URL to the pick-fetcher fixture as an additional case so both shapes are exercised. |
||
|
Zoltan Kochan
|
6836547017 |
feat(default-reporter): add pnpm-render bin to render NDJSON from stdin (#11530)
- Adds a `pnpm-render` bin to `@pnpm/cli.default-reporter` that reads pnpm-shaped NDJSON from stdin and pipes it through the default reporter, so external tools that emit `pnpm:*` log records can reuse pnpm's renderer. - Optional first positional arg sets the command name (defaults to `install`), e.g. `pnpm-render add` for piping output from `pacquet add`. ## Motivation [Pacquet](https://github.com/pnpm/pacquet) emits pnpm-shaped `--reporter=ndjson` output for forward-compatibility with pnpm's renderer, but there was no way to actually render it. With this bin: ```sh pacquet install --reporter=ndjson 2>&1 >/dev/null | pnpm-render ``` (pacquet writes NDJSON to stderr, so the redirect is needed.) |
||
|
|
6ac06cbed4 |
feat(audit): add registry signature verification (#11405)
* feat(audit): add registry signature verification * chore: add registry signature terms to cspell * chore: sort cspell registry terms * refactor(audit): use repo concurrency and error helpers * refactor(audit): use registry fetch helper for signatures * refactor(audit): share audit command context * fix(audit): respect scoped registries for signatures * fix(audit): handle missing signature metadata gracefully * docs(audit): document signature verification * test(audit): avoid signature spellcheck false positives * chore(audit): add scoped registry project reference * refactor(audit): clarify signature verification fetching * style(audit): align signature verifier formatting * fix(audit): validate signature metadata shape and report cleanly * fix(audit): handle crypto.verify throws on malformed registry keys A registry returning malformed PEM key material made verifier.verify throw synchronously, rejecting the Promise.all and crashing the whole audit run. Treat any verify failure as an invalid signature for that single package. * refactor(audit): extract parseJsonResponse helper Both fetchRegistryKeys and fetchPackument repeated the same JSON.parse + PnpmError wrapping pattern. Collapse into a single helper. * refactor(audit): split signature verification into its own package Move verifySignatures from @pnpm/deps.compliance.audit into a new @pnpm/deps.compliance.signatures package. Vulnerability auditing and signature verification are conceptually distinct trust subsystems, and sigstore provenance verification is in scope for a future change — keeping all signature work in its own package avoids growing the audit module into two unrelated concerns. * docs(audit): drop signature verification section The signature verification implementation moved to @pnpm/deps.compliance.signatures; that package's README documents the behavior. The audit package no longer needs to mention it. * refactor(signatures): move package to deps/security Place the new signature verification package under deps/security/ rather than deps/compliance/. Compliance is a fuzzy fit for tamper detection; security is the right home, and sigstore provenance verification (future scope) will live alongside it. Existing audit/license/sbom packages stay where they are — this only changes where the new package lands. --------- Co-authored-by: Colin Fristoe <47856231+ctfristoe@users.noreply.github.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Charlie CroomandAmp
|
3b12eb27de |
fix(@pnpm/exe): preserve relative symlinks when packaging dist/ (#11399)
* fix(@pnpm/exe): preserve relative symlinks when packaging dist/ The standalone executable build copies dist/ via fs.cpSync(...) without verbatimSymlinks: true, which causes Node to resolve relative symlinks into absolute paths at the source filesystem location. On the GitHub Actions runner this rewrites .bin symlinks to /home/runner/work/pnpm/... targets that ship verbatim in the release tarballs. Adding verbatimSymlinks: true preserves the relative symlink targets so the archived links remain valid at any extraction location. Fixes #11398. 🤖 Generated with [Amp](https://ampcode.com) Amp-Thread-ID: https://ampcode.com/threads/T-019dda79-b947-742f-8711-b6f83bcda9ff Co-authored-by: Amp <amp@ampcode.com> * chore: add 'unextractable' to cspell.json Per https://github.com/pnpm/pnpm/pull/11399#issuecomment-4348220789 🤖 Generated with [Amp](https://ampcode.com) Amp-Thread-ID: https://ampcode.com/threads/T-019dda79-b947-742f-8711-b6f83bcda9ff Co-authored-by: Amp <amp@ampcode.com> --------- Co-authored-by: Amp <amp@ampcode.com> |
||
|
Igor SavinandZoltan Kochan
|
b61e268d57 |
feat: add support for github prefix and named registries (#11324)
This is consistent with #9358, but implements support for the GitHub Packages npm registry and, more broadly, for vlt-style https://docs.vlt.sh/cli/registries for any registry. This PR adds a built-in gh: specifier that resolves against the GitHub Packages npm registry, plus a namedRegistries config key so a project can map its own aliases to arbitrary registries. A project can mix public npm packages and private GitHub Packages (or self-hosted) ones without applying a scope-wide registry override to every @scope/* package. - pnpm add gh:@acme/private writes "@acme/private": "gh:^1.0.0" and resolves from https://npm.pkg.github.com/. - pnpm add gh:@acme/private@^1.0.0 (with or without an alias) is also supported. Aliased form writes "my-alias": "gh:@acme/private@^1.0.0". - Auth comes from the existing per-URL .npmrc mechanism, e.g. //npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}. No new auth surface. - @github is intentionally not defaulted to https://npm.pkg.github.com/ - hardcoding that would hijack installs of the public @github/* packages on npmjs.org (e.g. @github/relative-time-element) for users without a scope-wide override. Use gh: to install from GitHub Packages, or configure @github:registry=... yourself if that's really what you want. - Additional named registries (a self-hosted proxy, GitHub Enterprise Server, etc.) can be configured in pnpm-workspace.yaml: ```yml namedRegistries: gh: https://npm.pkg.github.example.com/ # optional: overrides the built-in `gh` alias for GHES work: https://npm.work.example.com/ ``` - Then work:@corp/lib@^2.0.0 resolves against https://npm.work.example.com/, and the built-in gh alias can be redirected to a GHES host. - Env-var substitution (${VAR}) is supported in namedRegistries values, mirroring the .npmrc convention. - Reserved alias names (npm, jsr, github, workspace, catalog, file, git, http, https, link, patch, and related git host shorthands) cannot be redefined as user-named registries - the resolver throws ERR_PNPM_RESERVED_NAMED_REGISTRY_ALIAS at startup rather than silently shadowing another protocol. Malformed URLs throw ERR_PNPM_INVALID_NAMED_REGISTRY_URL at startup too, instead of failing as a confusing 404 during resolution. - On publish, createExportableManifest strips any named-registry prefix (both the built-in gh: and any user-configured alias) so npm and yarn consumers can still resolve the dependency via their own scope-registry configuration - mirroring the user-facing requirement when installing such a dep without the prefix. The prefix is gh: rather than github: because github: is reserved by npm-package-arg / hosted-git-info as a git host shorthand (e.g. github:owner/repo) - reusing it would be a deviation from the specs used by the npm CLI. gh: is shorter, matches vlt's convention, and cannot collide with any existing npm scheme. Unlike jsr:, gh: (and any other named-registry alias) does not rewrite the package name - gh:@acme/foo resolves @acme/foo from the GitHub Packages registry as-is. This also means npm/yarn consumers see the original name after the prefix is stripped on publish. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
eb7e6ae8ab |
fix(exe): restore @pnpm/exe startup on Node.js v25.7+ (#11330)
## Summary `@pnpm/exe@11.0.0-rc.4` aborts on every invocation with: ``` node::sea::(anonymous namespace)::SeaDeserializer::Read() at ../src/node_sea.cc:174 Assertion failed: (format_value) <= (static_cast<uint8_t>(ModuleFormat::kModule)) ``` Two independent Node.js v25.7+ SEA regressions are responsible, both surfaced by the rc.4 bump of the embedded runtime from 25.6.1 to 25.9.0. This PR fixes both and adds a prepublish smoke test so a broken binary can't reach npm again. ## Root cause **1. SEA blob format changed in Node.js v25.7.0** ([nodejs/node#61813](https://github.com/nodejs/node/pull/61813) added ESM-entry-point support and inserted a new `ModuleFormat` byte into the blob header). SEA blobs carry no version marker, so a blob written by one Node.js version can only be deserialized by a matching one. In rc.4, the CI host Node.js (25.6.1, pre-change) wrote the blob and it was embedded in a 25.9.0 runtime (post-change) — the deserializer reads a misaligned byte as `format_value`, exceeds `kModule`, `CHECK_LE` fires, `SIGABRT`. `resolveBuilderBinary()` was preferring `process.execPath` whenever the running Node supported `--build-sea`, never checking that its version matched the embedded runtime. **2. Node.js v25.7+ replaces the ambient `require` and `import()` inside a CJS SEA entry with embedder hooks** that only resolve built-in module names. The `pnpm.cjs` shim loaded `dist/pnpm.mjs` via `await import(pathToFileURL(...).href)`, which after the fix to (1) reached the CJS entry and then blew up with: ``` ERR_UNKNOWN_BUILTIN_MODULE: No such built-in module: file:///.../dist/pnpm.mjs at loadBuiltinModuleForEmbedder at importModuleDynamicallyForEmbedder ``` ## Changes - **`releasing/commands/src/pack-app/packApp.ts`** — `resolveBuilderBinary` now takes the resolved target runtime version and only reuses `process.execPath` when `process.version` exactly matches; otherwise it downloads a host-arch Node of the target version via the existing `ensureNodeRuntime` path. Added `PACK_APP_RUNTIME_TOO_OLD` for runtimes older than v25.5 (no `--build-sea`). Removed the now-unused `DEFAULT_BUILDER_SPEC` and the stale `fetch`/`nodeDownloadMirrors` args on the builder resolver. Help text / examples refreshed to drop `node@22` / `node@lts` references that would now be rejected. - **`pnpm/pnpm.cjs`** — loads `dist/pnpm.mjs` through `Module.createRequire(process.execPath)` instead of `await import(fileURL)`. `createRequire` returns a regular CJS loader that bypasses the SEA embedder hooks, and the pnpm bundle has no top-level await so synchronous `require` of ESM (Node 22+) loads it cleanly. No build-time paths are baked in — `process.execPath` is evaluated at runtime, verified by relocation-testing the darwin-arm64 SEA under `/tmp/`. - **`pnpm/artifacts/verify-binary.mjs`** (new) + `prepublishOnly` on every platform artifact — replaces the existence-only `test -f pnpm` gate with: 1. A **relocation-sensitivity check**: run the binary without `dist/` staged and confirm the failure mentions a path derived from `process.execPath`, not a build-time constant. Catches any future regression of (2). 2. A **smoke test**: stage a `dist → ../exe/dist` symlink (using `symlink-dir` so Windows junctions are handled transparently), exec `./pnpm -v`, assert the output is a SemVer 2 string. - Cross-platform targets (darwin/win32 artifacts on a Linux CI, or a libc mismatch) skip the exec with a log line and fall back to existence-only, so a musl artifact published from a glibc host still goes through. - Real `dist/` dirs (developer layout) are preserved; stale symlinks from aborted runs are replaced; created symlinks are cleaned up on exit. - **`pnpm/artifacts/exe/test/setup.test.ts`** — new `pnpm -v` execution test gated on both the platform binary and the staged bundle being present, so ordinary `pn compile` test runs skip cleanly instead of failing on a missing `dist/`. |