diff --git a/registry/client/README.md b/registry/client/README.md new file mode 100644 index 0000000000..6906c71b74 --- /dev/null +++ b/registry/client/README.md @@ -0,0 +1,43 @@ +# @pnpm/registry.client + +Client library for the pnpm registry server. Reads the local store state, sends it to the server, and writes the received files into the content-addressable store. + +## How it works + +1. Reads integrity hashes from the local store index (`index.db`). +2. Sends `POST /v1/install` to the pnpm registry server with the project's dependencies and the store integrities. +3. Decodes the binary streaming response — JSON metadata followed by raw file entries. +4. Writes each received file directly to the local CAFS (`files/{hash[:2]}/{hash[2:]}`). +5. Writes store index entries for all new packages in a single SQLite transaction. +6. Returns the resolved lockfile for use with pnpm's headless install (linking phase). + +## Usage + +This package is used internally by pnpm when the `pnpm-registry` config option is set. It is not intended to be called directly, but can be used programmatically: + +```typescript +import { fetchFromPnpmRegistry } from '@pnpm/registry.client' +import { StoreIndex } from '@pnpm/store.index' + +const storeIndex = new StoreIndex('/path/to/store') + +const { lockfile, stats } = await fetchFromPnpmRegistry({ + registryUrl: 'http://localhost:4000', + storeDir: '/path/to/store', + storeIndex, + dependencies: { react: '^19.0.0' }, + devDependencies: { typescript: '^5.0.0' }, +}) + +console.log(`Resolved ${stats.totalPackages} packages`) +console.log(`${stats.alreadyInStore} cached, ${stats.filesToDownload} files downloaded`) +// lockfile is ready for headless install +``` + +## Configuration + +Set in `.npmrc` to enable automatically during `pnpm install`: + +```ini +pnpm-registry=http://localhost:4000 +``` diff --git a/registry/server/README.md b/registry/server/README.md new file mode 100644 index 0000000000..f2b921e593 --- /dev/null +++ b/registry/server/README.md @@ -0,0 +1,91 @@ +# @pnpm/registry.server + +A pnpm registry server that resolves dependencies server-side and streams only the files missing from the client's content-addressable store. + +## How it works + +1. Client sends `POST /v1/install` with dependencies, an optional existing lockfile, and the integrity hashes of packages already in its store. +2. Server resolves the full dependency tree using pnpm's own resolution engine. +3. Server computes which file digests the client is missing — at the individual file level, not just the package level. +4. Server streams a binary response: JSON metadata (lockfile + per-package file indexes) followed by the raw content of missing files. + +This eliminates sequential metadata round-trips (the server resolves in one shot) and avoids downloading files that already exist in the client's store from other packages. + +## Starting the server + +### From the command line + +```bash +# Build first +pnpm --filter @pnpm/registry.server run compile + +# Run with defaults (port 4873, upstream https://registry.npmjs.org/) +node lib/bin.js + +# Or configure via environment variables +PORT=4000 \ +PNPM_REGISTRY_STORE_DIR=./my-store \ +PNPM_REGISTRY_CACHE_DIR=./my-cache \ +PNPM_REGISTRY_UPSTREAM=https://registry.npmjs.org/ \ +node lib/bin.js +``` + +### Environment variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `PORT` | `4873` | Port to listen on | +| `PNPM_REGISTRY_STORE_DIR` | `./store` | Directory for the server's content-addressable store | +| `PNPM_REGISTRY_CACHE_DIR` | `./cache` | Directory for package metadata cache | +| `PNPM_REGISTRY_UPSTREAM` | `https://registry.npmjs.org/` | Upstream npm registry to resolve from | + +### Programmatic usage + +```typescript +import { createRegistryServer } from '@pnpm/registry.server' + +const server = await createRegistryServer({ + storeDir: '/var/lib/pnpm-registry/store', + cacheDir: '/var/lib/pnpm-registry/cache', + registries: { default: 'https://registry.npmjs.org/' }, +}) + +server.listen(4000, () => { + console.log('pnpm-registry listening on port 4000') +}) +``` + +## Configuring pnpm to use the server + +Add to `.npmrc`: + +```ini +pnpm-registry=http://localhost:4000 +``` + +Then `pnpm install` will use the registry server for resolution and fetching instead of the normal flow. + +## API + +### `POST /v1/install` + +**Request body** (JSON): + +```json +{ + "dependencies": { "react": "^19.0.0" }, + "devDependencies": { "typescript": "^5.0.0" }, + "overrides": {}, + "lockfile": null, + "storeIntegrities": ["sha512-abc...", "sha512-def..."] +} +``` + +**Response** (binary, `Content-Type: application/x-pnpm-install`): + +``` +[4 bytes: JSON metadata length] +[N bytes: JSON metadata — lockfile, package file indexes, stats] +[file entries: 64B digest + 4B size + 1B mode + content, repeated] +[64 zero bytes: end marker] +``` diff --git a/registry/server/package.json b/registry/server/package.json index c9b6361564..6dd8527e71 100644 --- a/registry/server/package.json +++ b/registry/server/package.json @@ -17,6 +17,9 @@ "type": "module", "main": "lib/index.js", "types": "lib/index.d.ts", + "bin": { + "pnpm-registry": "./lib/bin.js" + }, "exports": { ".": "./lib/index.js" }, diff --git a/registry/server/src/bin.ts b/registry/server/src/bin.ts new file mode 100644 index 0000000000..5c86cd08c4 --- /dev/null +++ b/registry/server/src/bin.ts @@ -0,0 +1,27 @@ +import { createRegistryServer } from './createRegistryServer.js' + +const port = parseInt(process.env['PORT'] ?? '4873', 10) +const storeDir = process.env['PNPM_REGISTRY_STORE_DIR'] ?? './store' +const cacheDir = process.env['PNPM_REGISTRY_CACHE_DIR'] ?? './cache' +const upstream = process.env['PNPM_REGISTRY_UPSTREAM'] ?? 'https://registry.npmjs.org/' + +async function main (): Promise { + const server = await createRegistryServer({ + storeDir, + cacheDir, + registries: { default: upstream }, + port, + }) + + server.listen(port, () => { + console.log(`pnpm-registry server listening on http://localhost:${port}`) + console.log(` store: ${storeDir}`) + console.log(` cache: ${cacheDir}`) + console.log(` upstream: ${upstream}`) + }) +} + +main().catch((err) => { + console.error(err) + process.exit(1) +})