mirror of https://github.com/pnpm/pnpm.git synced 2026-07-23 22:22:51 -04:00

Files

Zoltan Kochan 3b6cc772c6 feat: pnpm agent — server-side resolution for faster installs

Add an opt-in pnpm agent server that resolves dependencies server-side
and streams only the files missing from the client's store.

Server (@pnpm/agent.server):
- Multi-process HTTP server (Node.js cluster, 9 workers)
- SQLite-backed metadata cache — resolution in ~1s vs ~3.4s with .jsonl
- Streaming NDJSON /v1/install — file digests emitted as packages resolve
- Gzip-compressed streaming /v1/files — no buffering on server or worker
- Binary protocol with server-provided digests (no client rehashing)

Client (@pnpm/agent.client):
- Streaming NDJSON parser dispatches worker batches during resolution
- Worker-thread streaming HTTP + gzip decompress + CAFS writes
- Pre-packed msgpack store index entries written directly to SQLite
- Pipelined headless install via wrapped store controller

Config: `agent: "http://host:port"` in pnpm-workspace.yaml

2026-04-16 22:40:43 +02:00

3.0 KiB

Raw Permalink Blame History

@pnpm/agent.server

A pnpm agent server that resolves dependencies server-side and streams only the files missing from the client's content-addressable store.

How it works

Client sends POST /v1/install with dependencies, an optional existing lockfile, and the integrity hashes of packages already in its store.
Server resolves the full dependency tree using pnpm's own resolution engine.
Server computes which file digests the client is missing — at the individual file level, not just the package level.
Server streams a binary response: JSON metadata (lockfile + per-package file indexes) followed by the raw content of missing files.

This eliminates sequential metadata round-trips (the server resolves in one shot) and avoids downloading files that already exist in the client's store from other packages.

Starting the server

From the command line

# Build first
pnpm --filter @pnpm/agent.server run compile

# Run with defaults (port 4873, upstream https://registry.npmjs.org/)
node lib/bin.js

# Or configure via environment variables
PORT=4000 \
PNPM_AGENT_STORE_DIR=./my-store \
PNPM_AGENT_CACHE_DIR=./my-cache \
PNPM_AGENT_UPSTREAM=https://registry.npmjs.org/ \
node lib/bin.js

Environment variables

Variable	Default	Description
`PORT`	`4873`	Port to listen on
`PNPM_AGENT_STORE_DIR`	`./store`	Directory for the server's content-addressable store
`PNPM_AGENT_CACHE_DIR`	`./cache`	Directory for package metadata cache
`PNPM_AGENT_UPSTREAM`	`https://registry.npmjs.org/`	Upstream npm registry to resolve from

Programmatic usage

import { createRegistryServer } from '@pnpm/agent.server'

const server = await createRegistryServer({
  storeDir: '/var/lib/pnpm-agent/store',
  cacheDir: '/var/lib/pnpm-agent/cache',
  registries: { default: 'https://registry.npmjs.org/' },
})

server.listen(4000, () => {
  console.log('pnpm agent listening on port 4000')
})

Quick start

Terminal 1 — start the server:

cd registry/server
pnpm run compile
node lib/bin.js
# pnpm agent server listening on http://localhost:4873

Terminal 2 — use it from any project:

cd my-project

Add to pnpm-workspace.yaml:

pnpmRegistry: http://localhost:4873

Then run:

pnpm install

That's it. pnpm will resolve dependencies on the server, download only the files missing from your local store, and link node_modules as usual. Remove the pnpmRegistry line to go back to normal behavior.

API

`POST /v1/install`

Request body (JSON):

{
  "dependencies": { "react": "^19.0.0" },
  "devDependencies": { "typescript": "^5.0.0" },
  "overrides": {},
  "lockfile": null,
  "storeIntegrities": ["sha512-abc...", "sha512-def..."]
}

Response (binary, Content-Type: application/x-pnpm-install):

[4 bytes: JSON metadata length]
[N bytes: JSON metadata — lockfile, package file indexes, stats]
[file entries: 64B digest + 4B size + 1B mode + content, repeated]
[64 zero bytes: end marker]

3.0 KiB Raw Permalink Blame History