## Summary
Adds an opt-in **pnpm agent** server that resolves dependencies server-side and streams only the files missing from the client's content-addressable store.
- **`@pnpm/agent.server`** — multi-process HTTP server (Node.js `cluster`) with SQLite-backed metadata and file caches
- **`@pnpm/agent.client`** — streams an NDJSON response, dispatches worker threads to fetch files while the server is still resolving
- **New config**: `agent` in `pnpm-workspace.yaml` (opt-in)
## How it works
1. Client reads integrity hashes from its local store index
2. Sends `POST /v1/install` with dependencies + store integrities
3. Server resolves the dependency tree using pnpm's `install({ lockfileOnly: true })`, with a SQLite-backed `PackageMetaCache` for fast repeat resolution
4. As each package resolves, a wrapped `storeController.requestPackage` looks up its files and immediately streams digests the client is missing (NDJSON `D` lines)
5. Client reads the stream line by line; digest batches fill up and dispatch worker threads to `POST /v1/files` — file downloads overlap with server-side resolution
6. After resolution, server sends index entries (`I` lines) and lockfile (`L` line)
7. Client writes index entries to store, then runs headless install with a wrapped `fetchPackage` that calls `readPkgFromCafs` with `verifyStoreIntegrity: false` (files are trusted from the agent)
8. `/v1/files` response is gzip-streamed (274MB → ~80MB) — server pipes through `createGzip`, worker pipes through `createGunzip`, parsing and writing files to CAFS as data arrives
## Performance
1351-package project, cold local store, warm server (localhost):
| Scenario | Time |
|----------|------|
| Vanilla pnpm install (cold OS cache) | ~48s |
| Vanilla pnpm install (warm OS cache) | ~34s |
| With pnpm agent (consistent) | **~33s** |
### Key optimizations
1. **SQLite metadata cache** — server-side resolution drops from ~3.4s to ~0.9s
2. **SQLite file store** — consistent read performance regardless of OS file cache state
3. **Streaming `/v1/install`** — file digests stream during resolution, downloads start before resolution finishes
4. **Gzip-streamed `/v1/files`** — whole-stream gzip (274MB → ~80MB), significant savings on remote servers
5. **Worker-thread streaming HTTP** — workers pipe gzip → parse → write to CAFS as data arrives, no buffering
6. **No rehashing** — server-provided digests used directly, skipping 33K SHA-512 computations
7. **No re-verification** — wrapped `fetchPackage` calls `readPkgFromCafs` with `verifyStoreIntegrity: false`
8. **Direct `writeFileSync` with `wx`** — no stat + temp + rename
9. **Pre-packed msgpack** — server sends raw store index buffers, client writes directly to SQLite
10. **WAL checkpoint** — ensures store index entries written by agent are visible to headless install's worker threads
## Usage
Start the server:
```bash
node agent/server/lib/bin.js
```
Configure in `pnpm-workspace.yaml`:
```yaml
agent: http://localhost:4873
```
* test: ensure prerelease weighting is correct
* fix: use higher weight for package versions already in lockfile
* test: remove fundamentally incompatible test
* fix(test): use undici MockAgent instead of nock for HTTP mocking
nock only patches Node's built-in http/https modules, but pnpm uses
undici for HTTP requests. Replace nock with @pnpm/testing.mock-agent
(which wraps undici's MockAgent) so the regression test actually
intercepts registry metadata requests.
* fix(benchmarks): show errors from store populate step
The populate step redirected both stdout and stderr to /dev/null,
hiding the actual error when pnpm install fails during benchmarks.
* fix(benchmarks): replace deprecated packages in benchmark fixture
The old fixture used deprecated babel 6, gulp, and other legacy
packages whose transitive dependencies (e.g. es-abstract) are missing
the "time" field in registry metadata, causing ERR_PNPM_MISSING_TIME
with time-based resolution mode.
Replace with modern equivalents (babel 7, webpack 5, MUI, Redux
Toolkit, etc.) that maintain a similar dependency tree size (~1300
packages) while using well-maintained packages with proper registry
metadata.
* fix(benchmarks): drop eslint plugins that pull in es-abstract
eslint-plugin-react, eslint-plugin-import, and eslint-plugin-jsx-a11y
transitively depend on es-abstract, whose registry metadata lacks the
"time" field. Replace them with eslint-plugin-prettier to avoid
ERR_PNPM_MISSING_TIME with time-based resolution.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Replace node-fetch with native undici for HTTP requests throughout pnpm.
Key changes:
- Replace node-fetch with undici's fetch() and dispatcher system
- Replace @pnpm/network.agent with a new dispatcher module in @pnpm/network.fetch
- Cache dispatchers via LRU cache keyed by connection parameters
- Handle proxies via undici ProxyAgent instead of http/https-proxy-agent
- Convert test mocking from nock to undici MockAgent where applicable
- Add minimatch@9 override to fix ESM incompatibility with brace-expansion
* fix: ensure PNPM_HOME/bin is in PATH during pnpm setup
When upgrading from old pnpm (global bin = PNPM_HOME) to new pnpm
(global bin = PNPM_HOME/bin), `pnpm setup` would fail because the
spawned `pnpm add -g` checks that the global bin dir is in PATH.
Prepend PNPM_HOME/bin to PATH in the spawned process env so the
check passes during the transition.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update pnpm to v11 beta 2
* chore: update pnpm to v11 beta 2
* chore: update pnpm to v11 beta 2
* chore: update pnpm to v11 beta 2
* fix: lint
* refactor: rename _-prefixed scripts to .-prefixed scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update root package.json to use .test instead of _test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ci: update action-setup
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the @pnpm/fs.msgpack-file package which was never imported in
source code (only in its own tests). Also remove the deprecated
lockfile-directory CLI option alias — users should use lockfile-dir.
* chore: reduce noisy warnings in test output
- Suppress ExperimentalWarning and DEP0169 via --disable-warning in NODE_OPTIONS
- Fix MaxListenersExceededWarning by raising limit in StoreIndex when adding exit listeners
- Update meta-updater to generate the new _test scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: stop streaming pnpm subprocess output during CLI tests
Buffer stdout/stderr from execPnpm instead of writing to the parent
process in real time. Output is still included in the error message on
failure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: pipe all subprocess output in CLI tests
Use stdio: 'pipe' for all pnpm/pnpx spawn helpers so subprocess output
is buffered instead of printed. Output is still included in error
messages on failure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove duplicate @pnpm/installing.env-installer in pnpm/package.json
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: use pipe stdio in dlx and errorHandler tests
Replace stdio: 'inherit' and [null, 'pipe', 'inherit'] with 'pipe' to
prevent subprocess output from leaking into test output.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip maxListeners adjustment when set to unlimited (0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: rename workspace.sort-packages and workspace.pkgs-graph
- workspace.sort-packages -> workspace.projects-sorter
- workspace.pkgs-graph -> workspace.projects-graph
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: rename packages/ to core/ and pkg-manifest.read-package-json to reader
- Rename packages/ directory to core/ for clarity
- Rename pkg-manifest/read-package-json to pkg-manifest/reader (@pnpm/pkg-manifest.reader)
- Update all tsconfig, package.json, and lockfile references
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: consolidate runtime resolvers under engine/runtime domain
- Remove unused @pnpm/engine.runtime.node.fetcher package
- Rename engine/runtime/node.resolver to node-resolver (dash convention)
- Move resolving/bun-resolver to engine/runtime/bun-resolver
- Move resolving/deno-resolver to engine/runtime/deno-resolver
- Update all package names, tsconfig paths, and lockfile references
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update lockfile after removing node.fetcher
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: sort tsconfig references and package.json deps alphabetically
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: auto-fix import sorting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update __typings__ paths in tsconfig.lint.json for moved resolvers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove deno-resolver from deps of bun-resolver
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
