## Summary
Adds an opt-in **pnpm agent** server that resolves dependencies server-side and streams only the files missing from the client's content-addressable store.
- **`@pnpm/agent.server`** — multi-process HTTP server (Node.js `cluster`) with SQLite-backed metadata and file caches
- **`@pnpm/agent.client`** — streams an NDJSON response, dispatches worker threads to fetch files while the server is still resolving
- **New config**: `agent` in `pnpm-workspace.yaml` (opt-in)
## How it works
1. Client reads integrity hashes from its local store index
2. Sends `POST /v1/install` with dependencies + store integrities
3. Server resolves the dependency tree using pnpm's `install({ lockfileOnly: true })`, with a SQLite-backed `PackageMetaCache` for fast repeat resolution
4. As each package resolves, a wrapped `storeController.requestPackage` looks up its files and immediately streams digests the client is missing (NDJSON `D` lines)
5. Client reads the stream line by line; digest batches fill up and dispatch worker threads to `POST /v1/files` — file downloads overlap with server-side resolution
6. After resolution, server sends index entries (`I` lines) and lockfile (`L` line)
7. Client writes index entries to store, then runs headless install with a wrapped `fetchPackage` that calls `readPkgFromCafs` with `verifyStoreIntegrity: false` (files are trusted from the agent)
8. `/v1/files` response is gzip-streamed (274MB → ~80MB) — server pipes through `createGzip`, worker pipes through `createGunzip`, parsing and writing files to CAFS as data arrives
## Performance
1351-package project, cold local store, warm server (localhost):
| Scenario | Time |
|----------|------|
| Vanilla pnpm install (cold OS cache) | ~48s |
| Vanilla pnpm install (warm OS cache) | ~34s |
| With pnpm agent (consistent) | **~33s** |
### Key optimizations
1. **SQLite metadata cache** — server-side resolution drops from ~3.4s to ~0.9s
2. **SQLite file store** — consistent read performance regardless of OS file cache state
3. **Streaming `/v1/install`** — file digests stream during resolution, downloads start before resolution finishes
4. **Gzip-streamed `/v1/files`** — whole-stream gzip (274MB → ~80MB), significant savings on remote servers
5. **Worker-thread streaming HTTP** — workers pipe gzip → parse → write to CAFS as data arrives, no buffering
6. **No rehashing** — server-provided digests used directly, skipping 33K SHA-512 computations
7. **No re-verification** — wrapped `fetchPackage` calls `readPkgFromCafs` with `verifyStoreIntegrity: false`
8. **Direct `writeFileSync` with `wx`** — no stat + temp + rename
9. **Pre-packed msgpack** — server sends raw store index buffers, client writes directly to SQLite
10. **WAL checkpoint** — ensures store index entries written by agent are visible to headless install's worker threads
## Usage
Start the server:
```bash
node agent/server/lib/bin.js
```
Configure in `pnpm-workspace.yaml`:
```yaml
agent: http://localhost:4873
```
The legacy `/-/npm/v1/security/audits{,/quick}` endpoints have been retired by npmjs.org. This PR rewires the audit client to the replacement `/-/npm/v1/security/advisories/bulk` endpoint.
The new endpoint is not a drop-in rename — the request and response contracts are both different:
- **Request**: a flat `{ pkgName: [versions] }` map. `lockfileToAuditRequest` walks the lockfile once and builds the POST body directly; there is no more nested `AuditTree`.
- **Response**: only `id`, `url`, `title`, `severity`, `vulnerable_versions`, and `cwe` per advisory. Everything else the old endpoint returned is computed locally:
- `findings[].paths` are walked from the lockfile (skipped entirely when the response is empty; the second walk intentionally avoids `@pnpm/lockfile.walker`'s global dedup so alternate install chains to the same shared dep aren't dropped).
- `metadata.vulnerabilities` counts advisories per severity.
- `metadata.dependencies` / `devDependencies` / `optionalDependencies` / `totalDependencies` come from a classified lockfile walk; the classifier respects `--prod`/`--dev` include flags when deciding whether a subgraph is reachable non-optionally.
- `patched_versions` is inferred from the vulnerable range for common `<X.Y.Z` / `<=X.Y.Z` shapes so `audit --fix` can still produce usable overrides; left `undefined` when inference fails.
- `github_advisory_id` is parsed from the advisory URL and canonicalized to the github.com form (uppercase `GHSA-` prefix, lowercase suffix).
- `info` severity is now supported end-to-end (severity type, `--audit-level`, filters, colors).
## Breaking changes (v11)
- Private registries that do not implement `/advisories/bulk` now fail with `AuditEndpointNotExistsError`.
- CVE-based filtering is replaced with GHSA-based filtering, since the bulk endpoint does not return CVE identifiers:
- `auditConfig.ignoreCves` → `auditConfig.ignoreGhsas` (the old key is no longer recognized).
- `pnpm audit --ignore <id>` and `--ignore-unfixable` now read and write GHSAs.
- Migration: replace each `CVE-YYYY-NNNNN` in `auditConfig.ignoreCves` with the matching `GHSA-xxxx-xxxx-xxxx` (visible in the `More info` column of `pnpm audit` output) under `auditConfig.ignoreGhsas`.
- `--ignore-unfixable` now only targets advisories whose patched range couldn't be inferred — the only "no fix available" signal the bulk endpoint provides.
- `AuditReport` and `AuditAdvisory` are trimmed to just the fields the audit client actually populates:
- `AuditReport`: `advisories` + `metadata` only (`actions` and `muted` removed).
- `AuditAdvisory`: `findings`, `id`, `title`, `module_name`, `vulnerable_versions`, `patched_versions?`, `severity`, `cwe`, `github_advisory_id`, `url`. Dropped: `cves`, `created`, `updated`, `deleted`, `access`, `overview`, `recommendation`, `references`, `found_by`, `reported_by`, `metadata`.
- `AuditAction`, `AuditResolution`, `AuditActionRecommendation` removed (no consumers).
## Hardening
- Response body validated: non-object / malformed JSON / non-array package buckets all surface as `ERR_PNPM_AUDIT_BAD_RESPONSE` with a body excerpt. Advisory `id` must be a finite number and `severity` must be a known value before being indexed.
- Name-keyed records use `Object.create(null)` so a hostile/unusual package name can't trigger prototype pollution.
- GHSA ids canonicalized on both read and write so casing drift between config and registry doesn't mask ignores.
- `findings[].paths` are deduped and capped per (name, version) to keep pathologically shared graphs from blowing up memory.
## Internals
- `AuditTree` / `AuditNode` / `lockfileToAuditTree` removed. `lockfileToAuditIndex.ts` exports `lockfileToAuditRequest` (flat POST body + counts) and `buildAuditPathIndex` (only invoked when the response has advisories).
- `AuditAdvisory.findings` is now `AuditFinding[]` (was an unintended 1-tuple).
- Top-level test fixtures regenerated from real `registry.npmjs.org` responses; synthetic `update-*` fixtures converted in place to bulk shape.
---------
Co-authored-by: John van Leeuwen <john.van.leeuwen@priva.com>
Co-authored-by: Zoltan Kochan <z@kochan.io>
Proxy settings (httpProxy, httpsProxy, noProxy), local-address,
strict-ssl, and git-shallow-hosts are now written to config.yaml
(global) or pnpm-workspace.yaml (local) instead of auth.ini/.npmrc.
They are still readable from .npmrc for easier migration from npm CLI.
The canonical YAML key names (httpProxy, httpsProxy, noProxy) match
Yarn Berry's naming convention.
- Add httpProxy, httpsProxy, noProxy to PnpmSettings type
- Add http-proxy to pnpmTypes and pnpmConfigFileKeys
- Separate network keys from auth keys in config routing
- Add isNpmrcReadableKey for backward-compatible .npmrc reading
Major cleanup of the config system after migrating settings from `.npmrc` to `pnpm-workspace.yaml`.
### Config reader simplification
- Remove `checkUnknownSetting` (dead code, always `false`)
- Trim `npmConfigTypes` from ~127 to ~67 keys (remove unused npm config keys)
- Replace `rcOptions` iteration over all type keys with direct construction from defaults + auth overlay
- Remove `rcOptionsTypes` parameter from `getConfig()` and its assembly chain
### Rename `rawConfig` to `authConfig`
- `rawConfig` was a confusing mix of auth data and general settings
- Non-auth settings are already on the typed `Config` object — stop duplicating them in `rawConfig`
- Rename `rawConfig` → `authConfig` across the codebase to clarify it only contains auth/registry data from `.npmrc`
### Remove `rawConfig` from non-auth consumers
- **Lifecycle hooks**: replace `rawConfig: object` with `userAgent?: string` — only user-agent was read
- **Fetchers**: remove unused `rawConfig` from git fetcher, binary fetcher, tarball fetcher, prepare-package
- **Update command**: use `opts.production/dev/optional` instead of `rawConfig.*`
- **`pnpm init`**: accept typed init properties instead of parsing `rawConfig`
### Add `nodeDownloadMirrors` setting
- New `nodeDownloadMirrors?: Record<string, string>` on `PnpmSettings` and `Config`
- Replaces the `node-mirror:<channel>` pattern that was stored in `rawConfig`
- Configured in `pnpm-workspace.yaml`:
```yaml
nodeDownloadMirrors:
release: https://my-mirror.example.com/download/release/
```
- Remove unused `rawConfig` from deno-resolver and bun-resolver
### Refactor `pnpm config get/list`
- New `configToRecord()` builds display data from typed Config properties on the fly
- Excludes sensitive internals (`authInfos`, `sslConfigs`, etc.)
- Non-types keys (e.g., `package-extensions`) resolve through `configToRecord` instead of direct property access
- Delete `processConfig.ts` (replaced by `configToRecord.ts`)
### Pre-push hook improvement
- Add `compile-only` (`tsgo --build`) to pre-push hook to catch type errors before push
Replace the unmaintained @pnpm/npm-conf package with a purpose-built
module that reads only auth/registry-related settings from .npmrc files
using read-ini-file + @pnpm/config.env-replace (both already deps).
All non-registry settings (hoist-pattern, node-linker, etc.) are now
only read from pnpm-workspace.yaml, CLI options, or environment
variables. Registry-related settings (auth tokens, registry URLs,
SSL certs, proxy settings) continue to be read from .npmrc for
migration compatibility, and can also be set in pnpm-workspace.yaml.
New modules:
- loadNpmrcFiles.ts: reads .npmrc from standard locations, filters to
auth/registry keys, returns structured layers
- npmConfigTypes.ts: inlined npm config type definitions
- npmDefaults.ts: inlined npm defaults (registry, unsafe-perm, etc.)
The metadata cache files now use a two-line NDJSON format:
- Line 1: cache headers (etag, modified, cachedAt) ~100 bytes
- Line 2: raw registry metadata JSON (unchanged)
This allows loadMetaHeaders to read only the first 1 KB of the file
to extract conditional-request headers (etag, modified), avoiding
the cost of reading and parsing multi-MB metadata files when the
registry returns 200 and the old metadata would be discarded.
Also moves cache directories to v11/ namespace (v11/metadata,
v11/metadata-full, v11/metadata-full-filtered) since the format
is not backwards compatible.
Replace node-fetch with native undici for HTTP requests throughout pnpm.
Key changes:
- Replace node-fetch with undici's fetch() and dispatcher system
- Replace @pnpm/network.agent with a new dispatcher module in @pnpm/network.fetch
- Cache dispatchers via LRU cache keyed by connection parameters
- Handle proxies via undici ProxyAgent instead of http/https-proxy-agent
- Convert test mocking from nock to undici MockAgent where applicable
- Add minimatch@9 override to fix ESM incompatibility with brace-expansion
* fix: show error cause when failing to read metadata
* fix: correct changeset package name and add cause assertion tests
- Fix changeset to reference @pnpm/resolving.npm-resolver (not @pnpm/npm-resolver)
- Add PnpmError cause unit tests in @pnpm/error
- Fix npm-resolver tests to actually verify cause on thrown errors
(.toThrow() only checks message, not cause/hint/code properties)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Since v11 uses a new store version, all runtime packages (node, deno, bun)
have a generated package.json with bin fields. The hardcoded switch block
in the linker is no longer needed.
Also moves getNodeBinsForCurrentOS, getDenoBinLocationForCurrentOS, and
getBunBinLocationForCurrentOS out of @pnpm/constants into their respective
resolver packages, since each is only used in one place.
* fix: ensure PNPM_HOME/bin is in PATH during pnpm setup
When upgrading from old pnpm (global bin = PNPM_HOME) to new pnpm
(global bin = PNPM_HOME/bin), `pnpm setup` would fail because the
spawned `pnpm add -g` checks that the global bin dir is in PATH.
Prepend PNPM_HOME/bin to PATH in the spawned process env so the
check passes during the transition.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update pnpm to v11 beta 2
* chore: update pnpm to v11 beta 2
* chore: update pnpm to v11 beta 2
* chore: update pnpm to v11 beta 2
* fix: lint
* refactor: rename _-prefixed scripts to .-prefixed scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update root package.json to use .test instead of _test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ci: update action-setup
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: reduce noisy warnings in test output
- Suppress ExperimentalWarning and DEP0169 via --disable-warning in NODE_OPTIONS
- Fix MaxListenersExceededWarning by raising limit in StoreIndex when adding exit listeners
- Update meta-updater to generate the new _test scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: stop streaming pnpm subprocess output during CLI tests
Buffer stdout/stderr from execPnpm instead of writing to the parent
process in real time. Output is still included in the error message on
failure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: pipe all subprocess output in CLI tests
Use stdio: 'pipe' for all pnpm/pnpx spawn helpers so subprocess output
is buffered instead of printed. Output is still included in error
messages on failure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove duplicate @pnpm/installing.env-installer in pnpm/package.json
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: use pipe stdio in dlx and errorHandler tests
Replace stdio: 'inherit' and [null, 'pipe', 'inherit'] with 'pipe' to
prevent subprocess output from leaking into test output.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip maxListeners adjustment when set to unlimited (0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: rename workspace.sort-packages and workspace.pkgs-graph
- workspace.sort-packages -> workspace.projects-sorter
- workspace.pkgs-graph -> workspace.projects-graph
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: rename packages/ to core/ and pkg-manifest.read-package-json to reader
- Rename packages/ directory to core/ for clarity
- Rename pkg-manifest/read-package-json to pkg-manifest/reader (@pnpm/pkg-manifest.reader)
- Update all tsconfig, package.json, and lockfile references
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: consolidate runtime resolvers under engine/runtime domain
- Remove unused @pnpm/engine.runtime.node.fetcher package
- Rename engine/runtime/node.resolver to node-resolver (dash convention)
- Move resolving/bun-resolver to engine/runtime/bun-resolver
- Move resolving/deno-resolver to engine/runtime/deno-resolver
- Update all package names, tsconfig paths, and lockfile references
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update lockfile after removing node.fetcher
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: sort tsconfig references and package.json deps alphabetically
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: auto-fix import sorting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update __typings__ paths in tsconfig.lint.json for moved resolvers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove deno-resolver from deps of bun-resolver
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
