Every token-authenticated request awaited an UPDATE of last_used_at before
the handler ran. With the libsql backend that is a round trip to the primary
even when an embedded replica answers the lookup locally, so a replica
removed almost none of the latency for authenticated traffic.
LastUsedWriter in pnpr-auth now owns the write. It skips the write while the
stored time, which the authenticating lookup already read, is under a minute
old; claims the interval per token in memory so a burst of concurrent
requests from one token writes once even before the first write lands or the
replica syncs it back; and runs the write in a spawned task the request never
waits for. Both middleware sites, bearer tokens and OCI tokens' parents, go
through it. GET /-/npm/v1/tokens therefore reports a last-used time accurate
to within a minute.
Related to https://github.com/pnpm/tasks/issues/194
Part of https://github.com/pnpm/tasks/issues/191.
Wrong claims:
- cli: drop the nonexistent `add-user --password` flag and document the
terminal/stdin password read; `oci-gc --min-age` and `cache-gc --max-age`
are the flags (the `-secs` forms are aliases); `--from` defaults to the
`storage` key, not an internal field; `info` and `http` log the same
access records; only global and server options have PNPR_* variables.
- configuration: `packumentTtl` defaults to 5m; the minimal config now
sets `auth.htpasswd.file`, and the auth section says accounts live in
memory without it or a backend; grouped hosted registries default their
org to `<ecosystem>~<name>`; the handshake is served for any surface;
the pipeline surface also satisfies "something must be served"; add
`package.update_maintainers` to the audit list.
- tls-and-reverse-proxy: remove the invalid `publicUrl:` YAML key and the
nonexistent `rate_limiting` setting.
- backup-and-restore: `pnpr verify` hashes by default (`--check-hashes`
does not exist, `--skip-hashes` skips only the hash
comparison); `migrate-storage
--verify` compares bytes.
- webhooks: audit events do not cover publishes or token creation; the
example uses a real action and the `actor` field.
- cli: the `RUST_LOG` example keeps `pnpr::audit=info`.
- install-acceleration: the `--public-url` host is exempt from the
private-address refusal.
- shared-side-effects-cache: `pnpm pipeline` publishes workspace-task
artifacts.
Missing facts: `routes.allowedPrivateNetworks`, the upstream `search` key
and canonical `maxAge`, built-in `npm.jsr.io` route, relative path
resolution, `log` defaults and `type`, `${VAR:-default}` substitution and
the full secret-field list, `oci` defaults, `artifacts.quota` bounds,
`migrate-storage` destination rules, `--osv false`, `--packument-ttl`
default, S3 `forcePathStyle`/`allowHttp` defaults, grouped org layout,
OIDC key table, container image section, `backend:` ignoring the htpasswd
and token files.
Duplicates and links: health probe bodies, reload procedure, and the
resolver allowlist now link to their one home; broken `#secret` anchor
fixed; links into the pnpm docs use absolute pnpm.io URLs.
Stale history: drop "pnpr v1" and the per-paragraph "Since alpha.N" /
"Since pnpm 12.x" prefixes, and the contributor-only extensibility text.
Also corrects the `as_filter_directive` doc comment, which claimed `http`
quiets the rest of the crate.
The libsql backend ran every operation on one shared connection. On a
remote database that connection is a Hrana stream, and each response
carries the baton for the stream's next request. An operation dropped
while its request was in flight (a client that disconnected, the auth
timeout) never received the new baton, so the connection kept the spent
one and the server refused every later request with "Received a baton
that has already been used". From then on every login and every
authenticated request failed with a 500 until pnpr restarted. One shared
stream also put concurrent operations into each other's transactions.
Operations now check a connection out of a pool and run alone on it. A
connection returns to the pool only after its operation succeeded outside
a transaction; any other is closed and replaced on a later checkout. This
is what sqlx does for the PostgreSQL and MySQL backends. Pooled
connections keep their HTTP client, so reuse costs no new TCP or TLS
handshake, which a connection per operation would have paid on every
token lookup. backend.libsql.maxConnections sizes the pool, defaulting to
10 like sqlx.
* docs: lint prose with Vale
Add Vale with an in-repo style, .vale/styles/pnpm, that turns the
writing rules in AGENTS.md and the Google developer documentation style
guide into lint rules. The stock Vale packages (Google, Microsoft,
RedHat, write-good) were too noisy to block on, for example 1,300
passive-voice hits in the v12 docs alone, so the rules are written for
this repo, following the GitLab model. Each one links the guideline it
enforces.
Levels:
- error, fails CI: dashes, plain-word substitutions, condescending
words, "please", "note that", hedges, puffery, optional plurals,
exclamation marks, ellipses, curly quotes, generic link text, product
spellings, sentence-case headings, repeated words, double spaces,
implementation history in comments; in changesets, bold or italic
emphasis, internal names and release-line names.
- warning: future tense, sentences over 30 words, "instead of" and
"so that" tails in changesets.
- suggestion: passive voice.
.vale.ini scopes the rules. Docs get every rule. Contributor Markdown
and code comments keep dashes and headings as warnings until their
roughly 8,100 em dashes are rewritten. Released changesets stay in
.changeset/ and are not rewritten, so lint-prose.mjs lints only the
changesets added since PROSE_BASE (the PR base in CI).
The prose workflow installs a pinned, checksum-verified Vale release and
runs `pnpm lint:prose`. The rest of this change rewrites the existing
docs, READMEs, contributor Markdown, and code comments to pass the
error-level rules. Only prose and comment text changed. Renamed
headings keep their anchors.
Spelling stays with cspell, which checks identifiers that Vale does not
see.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: keep prose lint from flattening meaning
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Cloudflare removes zstd from Accept-Encoding before a Worker runs, and
from cf.clientAcceptEncoding too, so pnpr behind the recipe's Worker never
sent zstd. The Worker now asks pnpr for "zstd, gzip", the value pnpm sends
on archive requests, and returns the coded body with encodeBody "manual".
Cloudflare passes a zstd body to a client that accepts it and decodes it
for one that does not. Verified on a trial deployment: a tar for pnpm
arrives zstd-coded, metadata for a gzip client is still gzip-coded, and a
gzip-only client gets the tar uncompressed.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Enable libsql's `sync` feature so the embedded replica
(`backend.libsql.replicaPath`) can follow a Turso Cloud primary.
The replica was built with libsql's `replication` feature only, whose
protocol Turso Cloud stopped serving: pnpr failed to start with
"Write delegation: Unimplemented ... deprecated version of sync". With
the `sync` feature, `Builder::new_remote_replica` defaults to
`SyncProtocol::Auto`: it probes the primary's `/info` route and uses the
newer sync protocol (bootstrap from `/export`, frames from `/sync`,
writes over Hrana) when the primary answers, and keeps the replication
protocol otherwise. A self-hosted `sqld` answers 404 on `/info`, so it
is unaffected.
The new test runs a mock primary that serves those routes in front of a
local libsql database and checks that the replica bootstraps from the
export, reads locally, and writes to the primary. Without the feature
it fails the way the issue describes.
`connect` also creates the replica's parent directory, as the local
stores do for their files, and with `syncInterval: 0` it runs the
initial sync under the startup deadline, since libsql would otherwise
bootstrap inside `Database::connect` with no bound.
Closespnpm/tasks#183.
The registries page presented the pnpr layout as the layout of every pnpr
registry, but a pnpr .tgz base advertises the npm layout, which the
undeclared default already reconstructs; serverType: pnpr belongs on a tar
base. The pnpr configuration page now says tarballs: tar may go on the main
router and what that costs other clients. The Cloudflare recipe's Worker
waits through a cold start instead of failing it, and the page notes that
the edge rewrites Accept-Encoding.
The libsql auth backend classified a duplicate row only through SQLite
result codes. Turso's remote protocol reports it as a Hrana stream error
whose message carries the code name, so the counter seed on a second
start against the same database failed instead of being tolerated, and
pnpr exited at startup. A taken username reported the same way answered
500 instead of the login or name-taken path.
Parse the Hrana message the way the lock-conflict check already does.
Related to https://github.com/pnpm/tasks/issues/183.
Precalculate package binary entrypoints and shebang runtimes during tar
streaming in pnpr (dist.pnpr.bins) and during cold tarball extraction in
pnpm so .bin command shims can be linked without reading package.json,
walking directories.bin, or opening bin scripts on disk.
- Add PackageBin, ScriptRuntime, parse_shebang_from_bytes,
resolve_package_bins, and verify_package_bins in pnpm-package-manifest
with strict path normalization and shell-safe runtime validation.
- Extend inspect_tar in pnpm-tarball to compute bins and shebang runtimes
in the same streaming pass as requiresBuild when requiresBuild is false.
- Emit dist.pnpr.bins in pnpr packument responses and cache it in
pnpr-storage TarMetadataCache.
- Propagate cold-extracted manifests and precalculated bins through
CreateVirtualStoreOutput into LinkVirtualStoreBins and direct-dep bin
linking, and verify warm-cache dist.pnpr.bins against the store index
manifest and CAS file list before skipping shebang reads.
Closespnpm/tasks#178
Compute and persist per-tarball requiresBuild in pnpr when publishing or
decoding upstream tarballs, project it onto served packuments and
manifests under versions[v].dist.pnpr.requiresBuild and
POST /-/pnpr/v0/resolve package frames, and consume it in pnpm v12 to:
- overlap remote side-effects artifact resolution with cold tarball
downloads when cold snapshots already have a verified requiresBuild,
- skip unnecessary node --version probes and BuildModules passes when no
package requires a build, and
- skip redundant background extraction scheduling and CAS package.json
reads during early materialization.
Closes https://github.com/pnpm/tasks/issues/177.
Written by an agent (Antigravity, Gemini 4 Argon).
Complete Section B deliverables (B4, B5, B6, B7) for pnpr production readiness.
- B5: Fail fast at startup when secret-bearing configuration fields (secret,
secrets.cache, secrets.oci, s3.secretAccessKey, auth.scim.token,
clientSecret, authToken, or upstream tokens) contain unresolved \${VAR}
references rather than falling back to empty strings or per-process keys.
Support independent configuration of secrets.cache and secrets.oci, and
derive domain-separated HMAC-SHA256 subkeys from root secret.
- B4: Enforce per-ecosystem validation rules rejecting \`teamsManagedBy: api\`
on non-npm registries and \`unpublish\` rules on Cargo and PyPI registries.
- B6: Set \`publish = false\` on \`pnpr/crates/pnpr/Cargo.toml\` to guard against
accidental crates.io publication.
- B7: Document the v1 extensibility model: compile-time trait implementations
(\`UserBackend\`, \`TokenBackend\`), no dynamic plugin runtime.
Relates to pnpm/tasks#131.
Complete production-readiness requirements for pnpr Section A.
- Persisted-document format versions: introduce formatVersion to packuments,
OCI images, crate documents, and state journals, rejecting documents written
by newer format versions.
- CLI surface grammar: support `pnpr serve` subcommand, and add parse_cli_interval
for duration units (s/m/h/d) on --packument-ttl, --min-age, and --max-age.
- Strict config validation: reject unknown fields, disallow removed configuration
blocks, and normalize field naming across config and CLI flags.
- Client protocol versioning: add capability handshake at /-/pnpr, v0 resolve
and publish endpoints, policy echo in done frames, and structured error codes.
- Distribution improvements: add Node.js fallback wrapper, fix engines range,
and package tini in pnpr Docker container.
Relates to pnpm/tasks#131.
The hosted store shared one key space between packages, orgs and blobs:
a registry without an org stored packages at the store root, an org's
registry stored them in `<org>/`, a package's blobs sat beside its
document, and a nested image repository sat inside its parent's
directory. So an org collided with an unscoped package of the same name
(an unpublish of that package removed the org's whole namespace), the
implicit `<ecosystem>~<name>` org of a grouped registry was a valid npm
name, and OCI GC and verify needed ad-hoc exclusion lists. On S3,
object_store percent-encodes `~` in keys while the upload sweep compared
raw prefixes, so grouped OCI registries were never swept. This is part
of section A4 of pnpm/tasks#131.
The new layout (pnpr_storage::layout), the same on disk and in a bucket:
layout.json {"layout": 1}
default/packages/<name>/document.json
default/packages/<name>/blobs/<file>
orgs/<org>/packages/...
Every name, org and file name is one segment: `[a-z0-9._-]` as is,
every other byte as `!` and two hex digits. That keeps nested names
siblings, separates names that differ only in case on case-insensitive
filesystems, and leaves nothing for object_store to escape, so a key is
byte for byte the local path. Per-namespace records stay inside their
namespace; records every namespace shares stay at the root. Listing is
one directory read, so the `.package-index` workaround is gone, and the
blob inventory reports package and blob names instead of paths.
Startup refuses a store marked with another layout or an unmarked store
holding data, and marks an empty one. The proxy cache uses the same
package layout and is emptied once when its marker is missing.
`pnpr upgrade-storage` converts an earlier store on disk or in a bucket,
telling org directories from packages by the configured orgs; it moves
blobs before documents so an interrupted run can resume, and refuses a
directory that is both an org and a package. export, verify and
migrate-storage check the layout, and migrate-storage marks its target.
The registry fixtures the pnpm test suites serve are built in the new
layout, and the cached build is keyed by the layout version.
lockfile.integrityTarget: tar makes the integrity of every package tarball
entry the hash of its uncompressed tar, recorded as settings.integrityTarget.
The lockfile then survives a registry recompressing its tarballs, switching
to tar resources, or being replaced by a mirror that does either. A lockfile
recorded under another target is stale: a frozen install fails with
ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, and any other install re-resolves with
update-checksums semantics so no hash of the old kind carries forward. The
setting is opt-in and the lockfile version is unchanged.
The tarball crate now reports both hashes of every extraction: the resource
after content decoding, and the tar a .tgz decodes to. A pin names one of
them (IntegrityKind::Resource / Tar), or either at resolution, where a locked
or published integrity may be of either kind (ResourceOrTar, sha512 only).
Under a tar pin every gzip member is decoded and nothing may follow the tar.
The resource pin keeps today's behaviour, extracting the first member and
tolerating trailing data, but then records no tar.
Store rows extracted from a .tgz record tarIntegrity and get an alias under
the tar key, in a table pnpm v11 never reads. Store lookups follow aliases,
so a tar-pinned frozen install is served offline by a store that .tgz
installs populated, and resolution takes a version's tar hash from its row
before downloading anything.
Registry revisions now always pin their tar, revision 0 included, and are
fetched with a tar pin, so the digest route can use any content coding.
Revision metadata must list revision 0 and the selected revision, and a
digest-route URL without a revision is rejected. pnpr keys its digest
references by tar hash on every registry format, serves the route as a
negotiated tar resource, and projects revision 0 explicitly. pnpm v11 cannot
extract tar resources, so it now rejects revisions with
ERR_PNPM_REGISTRY_REVISIONS_UNSUPPORTED.
Implements pnpm/rfcs text/0000-tar-integrity-for-every-source.md and
pnpm/rfcs#42.
pnpr served npm at the server root when npm was the only configured
ecosystem and moved it under /npm/ as soon as a second ecosystem was
added. Adding an ecosystem therefore broke every .npmrc registry URL,
every tarball URL already written into a lockfile, named-registry URLs,
and OIDC workload publish paths, and /npm meant the package `npm` on one
server and the ecosystem prefix on another. This settles section A1 of
pnpm/tasks#131: every ecosystem is always prefixed.
- Ecosystem::base_path is now independent of the configuration. The
per-configuration Registries::base_path and is_only_ecosystem are gone,
and the routes, the resolver's tarball URLs, OIDC workload paths and
pnpr's own-origin route classification all use it.
- The default image registry still answers at /v2/ on the host root; an
image client cannot be pointed at a prefix. The registry directory keeps
its `prefixed` and `namedPrefixed` flags for the web UI.
- The read-only token allowlist matched the npm audit endpoints at the
root only, so read-only tokens were already refused for audits on
multi-ecosystem servers. It now matches them under the npm prefix.
- /metrics no longer switches between Prometheus metrics and the npm
package `metrics` on the Accept header; the package is /npm/metrics.
- The pnpm v11 and v12 test harnesses, the registry-mock crate and the
integrated benchmark address the fixture registry at /npm/ and keep the
server origin for the pnpr protocol endpoints. The benchmark asks each
revision mock's registry directory whether npm is prefixed, so it can
still compare against a pnpr revision from before this change.
A user a SCIM client provisioned has no password account until they sign
in, so anyone could take the name first through npm adduser and keep it.
Registration now treats every username in the SCIM directory as reserved,
like admin and OIDC-bound names. Provisioned users sign in through OIDC
or get their account from an admin. A password account that exists when
its user is provisioned keeps working.
A login that creates an account checks the name again before issuing a
token, so a provisioning that lands mid-login takes the account back, as
does a directory that can no longer be read.
Raised in review of pnpm/pnpm#17046.
A `.tgz` registry now serves a hosted version's blob, or an upstream
version's cached `.tgz`, under the canonical `<basename>-<version>.tgz`
the way the filename route does, so a warm hit loads no packument. A cold
upstream request still reads the requested version's dist, because the
filename route binds a filename to whichever version's `dist.tarball`
names it.
Without a fresh cached packument, the route reads that dist from the
upstream's `<name>/<version>` manifest, a few KB instead of the whole
packument. A manifest of another version, as an upstream returns for a
dist-tag, or one without a valid `dist.integrity` falls back to the
packument, as does an upstream that serves no manifest. An unreachable
upstream gets the stale cached packument at once.
A version that is not a safe path segment is not found, since the
canonical filename is built from it.
A tar registry's response needs the version's `.tgz` integrity. A dist
read from an upstream is kept in a per-replica cache. One read from a
cached packument is tagged with that packument's stamp (mtime and size)
and answers only while the packument is fresh and unchanged, so a
refetched packument retires its dists at once. One read from a version
manifest answers for the packument TTL. Authorization runs before every
lookup. Hosted dists and `cache: false` upstreams are not cached.
A large packument took 3-8 ms to read and parse per request (react,
`@types/node`, typescript). Warm requests for those versions now take
0.06-0.8 ms.
Closespnpm/tasks#151
Close gaps left in reserved-username registration, from the review of
pnpm/pnpm#17046:
- A login that saw a reserved account, and then created it because an
admin removed the account in between, now takes back the account and
its token and answers 403. Previously the caller's password stayed
stored under the admin's name. The check runs before a token is issued.
The account is deleted only while it still accepts the login's
password, so an admin's concurrent rewrite stays, and the name's tokens
are revoked afterwards.
- `pnpr add-user` revokes tokens left under the username before it
creates the account, as the admin API already did. The create-or-update
step and token revocation move to `AuthState` so both share them.
- Document `pnpr add-user`, including that it must run while pnpr is
stopped when users live in an htpasswd file, and stop telling admins to
self-register the first admin account.
Implements pnpm/rfcs#39 in pnpm v12 and pnpr.
pnpm:
- `serverType: pnpr` rebuilds tarball URLs as `<registry>/<name>@<version>`.
- Tarball requests go through HTTP clients that leave Content-Encoding in
place. pnpm decodes zstd (8 MiB window, RFC 9659) and gzip itself, because
reqwest's built-in decoder allows a 128 MiB window and cannot be configured.
- The decoded body is sniffed: the gzip magic means a .tgz, anything else a
tar. The hash covers the decoded resource on both paths.
- A decoded-output budget bounds what a response makes pnpm decode before its
hash is checked: file payload up to `maxTarballUnpackedSize` (4 GiB),
tightened by `dist.unpackedSize` for tars only; 256 MiB of non-payload;
at most 10 KiB of zeros after the end-of-archive marker; empty bodies fail.
- A missing `dist.tarball` is rebuilt from the declared layout, and fails
with ERR_PNPM_MISSING_TARBALL_URL without one. A missing `dist.integrity`
is learned by the existing resolve-time archive read.
- `--update-checksums` re-resolves every locked package.
pnpr:
- `tarballs: tar` makes a registry's metadata describe tars and its
`<name>@<version>` route serve them. Every registry serves that route.
- Tar hashes are derived at publish or on the first decode and stored in a
reserved `.tar-integrity` namespace of the shared hosted backend.
- A cached upstream .tgz is re-checked as its tar streams; a mismatch fails
the response and evicts the copy.
- gzip clients get the stored .tgz. zstd clients get a level-19 variant from
the local cache once a bounded worker made one, and a level-3 stream until
then. Decoding stops at 4.25 GiB, and copies kept for the worker are
coalesced per version and drawn from a 1 GiB quota.
* feat(pnpr): provide minimal production default configuration and separate test fixtures
Replace the bundled test configuration in `pnpr/crates/config/config.yaml`
with a minimal production-safe default configuration (Section A3 of
https://github.com/pnpm/tasks/issues/131):
- Configure public npmjs upstream proxy and hosted local registry at
`/~local/` with default registry set to npmjs.
- Generate a secure 32-byte CSPRNG random secret by default when secret is
omitted, and disable registration (`max_users: -1`).
- Clean up unused Verdaccio legacy keys (`plugins`, `middlewares`, `web`)
and remove test fixtures from production default configuration.
- Move test mock configuration and fixture scopes to `pnpr/.fixtures/config.yaml`.
- Move test presets (`REGISTRY_MOCK_LOCAL_PATTERNS`, `registry_mock_rules`,
`registry_mock_graph`, and `MockConfigPresets` trait with `Config::proxy`
and `Config::static_serve`) from `pnpr-config` into `pnpr-fixtures`.
- Update Jest global setup and `pnpm-registry-mock` to load test configuration
from `pnpr/.fixtures/config.yaml`.
- Update documentation in `pnpr/docs/configuration.md` and `pnpr/docs/quick-start.md`.
Related to https://github.com/pnpm/tasks/issues/131
* fix: drop obsolete quick-xml advisories from deny.toml
quick-xml was upgraded to 0.41.0, resolving RUSTSEC-2026-0194 and
RUSTSEC-2026-0195, so cargo-deny no longer encounters them.
* fix: ignore unreachable zip advisories in deny.toml
* fix(pnpr-config): require authentication to access private packages on local registry
* docs(pnpr): document reverse-proxy and TLS setup, webhooks, lifecycle, and pnprServer
Document production deployment and operational architecture for pnpr and
pnpm (Section C13 and Section F of https://github.com/pnpm/tasks/issues/131):
- Add Reverse proxy and TLS deployment guide (pnpr/docs/tls-and-reverse-proxy.md)
with architecture rationale and configuration recipes for Nginx, Caddy,
Traefik, and AWS ALB.
- Add Webhooks and event notifications guide (pnpr/docs/webhooks.md)
documenting the decoupled event architecture, structured audit logs
(pnpr::audit), and object storage bucket notifications.
- Document signals, readiness probe state transitions, process lifecycle, and
standard exit codes in pnpr/docs/cli.md.
- Document S3 commit journal lifecycle, multi-replica considerations, and
disaster recovery caveats in pnpr/docs/storage.md.
- Document the pnprServer workspace setting reference in
pnpm/docs/settings/dependency-resolution.md and pnpm11/docs/settings/dependency-resolution.md.
- Update sidebars and quick start navigation.
Related to https://github.com/pnpm/tasks/issues/131
* docs(pnpr): clarify pnprServer offloads resolution and streams package metadata
Clarify that pnprServer offloads dependency resolution rather than file
fetching: the server streams back resolved package metadata frames as it walks
the graph so the client can prefetch tarballs from registries in parallel.
* docs(pnpr): address review feedback on Vector remap and pnprServer tarball fetch boundary
- Add remap transform in Vector example in pnpr/docs/webhooks.md to parse
raw JSON from Docker container logs before filtering audit events.
- Clarify direct public tarball fetching and server-side private upstream
credential boundaries in pnpm/docs/settings/dependency-resolution.md and
pnpm11/docs/settings/dependency-resolution.md.
* docs(pnpr): correct probe endpoints, shutdown lifecycle, and log shippers
* docs(pnpr): correct S3 commit journal staging paths and write-ahead sequence
* docs(pnpr): clarify proxy hop coverage and orphan pruning scope
* docs(pnpr): distinguish X-Real-IP from hop traversal and document ALB target port restriction
Add maintenance subcommands and migration tooling for pnpr (Section C12 of
https://github.com/pnpm/tasks/issues/131):
- Implement `pnpr config check` (`pnpr config-check`) to validate configuration
and report diagnostic details on routes, upstreams, auth, storage, and surfaces.
- Implement `pnpr verify` to audit hosted package documents against storage
blobs, checking for missing files, corrupted checksums, and unreferenced orphans.
- Implement `pnpr export --out <dir>` to dump packages, documents, and blobs,
with optional `--include-auth` support.
- Implement `pnpr migrate-storage` to migrate hosted packages and blobs from local
filesystem storage to an S3-compatible object store.
- Support Apache/Verdaccio `{SHA}` format in `pnpr-auth`, automatically upgrading
authenticated users to bcrypt upon first login, and provide `pnpr import-htpasswd`.
- Add backup and disaster recovery documentation in `pnpr/docs/backup-and-restore.md`
and update CLI documentation.
Related to https://github.com/pnpm/tasks/issues/131
Support npm login web flow (`/-/v1/login`) so `npm login` and `pnpm login`
authenticate via browser SSO when an OIDC login provider is configured.
- Implement `POST /-/v1/login` and scoped `POST /~{registry}/-/v1/login`:
return 404 when no OIDC login providers are configured to allow fallback
to CouchDB auth; return 202 Accepted with `loginUrl` and `doneUrl` when enabled.
- Implement `GET /-/v1/login/start`: validate session and redirect to single
provider login or render an HTML picker for multiple providers.
- Implement `GET /-/v1/login/poll/{session}`: return 202 with `Retry-After: 1`
while pending, and 200 with `{ "token": "<token>" }` once authenticated, consuming session.
- Update OIDC login flow to support `return=cli&session={session}` and fulfill
CLI web login session upon completion.
- Export `bundled_root_certs` in `pnpm-network` and configure OIDC reqwest client
with root certificates.
Related to https://github.com/pnpm/tasks/issues/131
Add ETag generation, Vary: Accept, and Cache-Control headers to packument
responses, and return 304 Not Modified when an incoming If-None-Match or
If-Modified-Since header matches the current packument state.
- Compute strong ETag over rendered packument response body
- Return 304 Not Modified with empty body preserving ETag, Vary, and Cache-Control
- Support If-None-Match (with weak comparison per RFC 9110) taking precedence over If-Modified-Since
- Emit Vary: Accept on packument responses, preserving Accept and appending Authorization when private_no_cache applies
- Set Cache-Control: public, max-age={ttl} on public packuments from upstream maxage or config packument_ttl
- Maintain Cache-Control: private, no-store on private or caller-scoped responses
- Extract OSV filtering helpers into osv_filtering.rs and server lifecycle into lifecycle.rs to keep source files within line limits
Related to https://github.com/pnpm/tasks/issues/131
Emit structured audit log events under target `pnpr::audit` at info level
for administrative and destructive actions:
- User management (create, password update, delete, token revoke, register)
- Rule override updates and deletions
- Team creation, destruction, member add/remove, and package access grants
- SCIM user provisioning, replacement, patching, and deprovisioning
- Package and tarball unpublishing, dist-tag set/remove, and OCI deletions
All audit events inherit active request span context (request_id,
client_ip, method, uri) through tracing spans.
Ref: https://github.com/pnpm/tasks/issues/131
* feat(pnpr): implement cache eviction, lru size cap, and orphan namespace pruning
Add configurable cache budget and TTL retention policies for the
upstream proxy cache, automated orphan namespace pruning on startup,
periodic runtime cache eviction, and a new `pnpr cache-gc` CLI command.
Closespnpm/tasks#131 (Section C5).
* test(registry-mock): poll for the live registry child during pending startup
Poll for the child process until it appears with its command line or the
timeout elapses, avoiding a race when sysinfo samples the process table
before the spawned child has completed execve.
* test(registry-mock): record spawned child PID via oneshot channel
* test(pnpr): set accessed time alongside modified time in cache eviction tests
* fix(pnpr): address PR review findings on cache eviction
- Reject unknown fields on cache configuration table
- Count only successfully removed files in eviction reports and size accounting
- Log and continue orphan namespace pruning if individual entry removal fails
- Prevent symlink traversal during orphan namespace pruning
- Skip pruning ~upstreams/ when secret is not explicitly configured to protect sibling caches
* fix(pnpr): preserve in-flight tmp files and check mtime before LRU eviction
- Ignore active temporary files (.tmp*) unless older than 1 hour threshold
- Prevent in-flight tmp files from inclusion in LRU candidates or totals
- Check file effective time before LRU deletion to avoid removing replaced files
- Add test verifying in-flight tmp files are preserved during eviction
* feat(pnpr): metrics, request IDs, and enriched access logging
Expose Prometheus metrics endpoint at `GET /-/metrics` and `GET /metrics`
emitting standard text exposition with in-flight gauges, request counters,
duration histograms, and cache statistics.
Forward and validate incoming `x-request-id` headers or generate fresh
UUID v4s, recording them on response headers, tracing spans, and enriched
access logs under `pnpr::access` alongside user identity, client IP,
user agent, latency, and bytes.
Resolvespnpm/tasks#131 (Section C4).
* fix(pnpr): normalize metrics methods and clarify scraper docs
Normalize request methods to standard HTTP methods or "OTHER" and key
by static strings to eliminate allocations and bound cardinality.
Clarify scraper access policy and byte accounting in documentation.
* fix(pnpr): clarify metrics endpoint doc for npm accept headers
Clarify that requests to GET /metrics carrying npm packument Accept
headers receive the metrics package packument, as noted by CodeRabbit.
Add pre-buffer authorization for package writes, upstream request coalescing,
login rate limiting, and request timeouts to pnpr.
- Pre-buffer authorization: package mutations (publish, update, dist-tags) check
caller authentication and write permissions before reading the request body.
- Upstream coalescing (singleflight): deduplicate concurrent in-flight fetches for
the same package/namespace so cache misses do not hammer upstream registries.
- Rate limiting and concurrency limits: throttle failed login attempts by IP and
username with 429 Too Many Requests and Retry-After headers, and limit concurrent
login verifications and package publish buffering.
- Request timeouts: add configurable requestTimeout layer to pnpr HTTP service.
Ref: https://github.com/pnpm/tasks/issues/131
Adds trusted-proxy configuration and client IP resolution to pnpr.
When behind a reverse proxy, load balancer, or CDN, requests arrive from the
proxy's IP. To accurately record client IPs in access logs and enforce IP/CIDR
restrictions on authentication tokens, pnpr needs to inspect forwarding headers
(`Forwarded`, `X-Forwarded-For`, `X-Real-IP`).
However, forwarding headers can be trivially spoofed if accepted blindly. This
change adds `trusted_proxies` configuration (`HttpConfig`) supporting CIDR
blocks, exact IPs, and `loopback`. If the direct TCP peer is not trusted, all
forwarding headers are ignored. If trusted, the configured `trusted_proxy_header`
(defaulting to `x-forwarded-for`) is inspected without falling back to other
headers. Hops are traversed right-to-left, skipping trusted intermediate
proxies until the first untrusted client IP is found.
Ref: https://github.com/pnpm/tasks/issues/131
Listen for Unix SIGTERM signals alongside SIGINT (Ctrl-C) to perform
graceful server shutdown in container environments such as Docker and
Kubernetes.
Implement readiness checking on storage backends (local filesystem and
S3), cache directories, and auth backends (htpasswd, SQLite, Libsql,
PostgreSQL, MySQL). Expose dedicated `/-/ready` and `/-/live` probes
mounted globally and under registry prefixes, and make `/-/ping` return
503 Service Unavailable when storage or authentication is unhealthy.
Refs: https://github.com/pnpm/tasks/issues/131
* docs(pnpr): remove stale duplicated docs and fix drifted claims
The crate README and OIDC.md under pnpr/crates/pnpr duplicated the website
docs and had drifted: a compilerCaches key that no longer exists, hardcoded
quota numbers, the old registry/ path. The pnpr.client README used the
pre-rename remoteSideEffectsCache spellings and claimed the artifact owner
must equal the username. Point both at pnpr/docs instead.
Also fix claims the code contradicts: the handshake is served when any
surface is enabled and advertises ecosystems, publish, and pipeline; the
resolve request field is registries, not namedRegistries; pnpm does not
call the handshake before an npm resolve; there is no config reload; the
packument TTL has no config key; and the --config help still described a
verdaccio-shaped file.
Install instructions now name the next dist-tag, because latest still
points at a July 2026 pre-alpha snapshot.
* docs(pnpr): address review on handshake, verify order, and root-mounted endpoints
* docs(pnpr): document the /npm/ registry bases on multi-ecosystem servers
The workspace builds libsql without its `tls` feature, so a
`backend.libsql` config panicked at startup with "The `tls` feature is
disabled, you must provide your own http connector", for every remote
URL including plain `http://`.
pnpr now gives libsql its own connector. It is built on the workspace's
rustls 0.23 with the aws-lc-rs provider, and it verifies servers against
the platform trust store through rustls-platform-verifier. Enabling
libsql's `tls` feature instead would add hyper-rustls 0.25 and a second
rustls release. The lockfile gains no new crates.
- The trust store is loaded only for URLs libsql dials over HTTPS, so an
`http://` database works on a host without CA certificates. The
`http://` check is case-insensitive.
- A connector built for HTTPS refuses plain-HTTP URIs, so a `base_url`
in a Hrana response cannot move the auth token out of TLS.
- An `authToken` with a non-loopback `http://` url fails at startup,
the HTTPS-or-loopback rule pnpr-client applies to its credentials. With a
token, the connector also refuses each later plain-HTTP request to a
non-loopback host.
Dependency side effects and `pnpm pipeline` task results now share one
artifact format, one trust model, and two transports.
`pnpm pipeline` stores cacheable task results as `workspace-task`
artifacts of the shared artifact protocol: P-256 signed envelopes whose
manifest mirrors a local cache entry (`meta.json` and `outputs/`). A
remote artifact is verified against `trustedKeys`, written into the
local cache, and restored through the same checks as a local entry.
Results of tasks that ran are published in the background when
`publish` is on, and the run waits for the uploads before it ends.
`ArtifactStore` in pnpm-pnpr-client is either a pnpr server or a server
that speaks the Turborepo Remote Cache API. Both answers go through the
same client-side verification. On the Turborepo API an artifact is the
signed envelope stored under a hash of the owner, the input key, and
the compatibility scope from `compatibility_scopes`, and each blob once
per owner under a hash of its integrity, mirroring pnpr's resolve and
blob routes. The API cannot create conditionally, so a later
publication replaces an earlier one. A failed per-key lookup is a miss. Both transports
refuse credentials for a server that is neither HTTPS nor loopback.
The remote side-effects cache now goes through `ArtifactStore`, so it
can use a Turborepo server too. pnpr-client's candidate filter now
permits `workspace-task` candidates, which the pnpr server already
accepted.
`ArtifactSigner` replaces the duplicated signing setup in deps-restorer
and the pipeline, and `decode_trusted_keys` moves into the protocol
crate.
The new `remoteCache` setting holds `url`, `team`, `token`, `org`, and
the signing fields, with `PNPM_REMOTE_CACHE_*` overrides. Every field
except `org` is machine-only and refused in pnpm-workspace.yaml with
ERR_PNPM_WORKSPACE_REMOTE_CACHE_TRUST: `token` is not URL-scoped, so a
repository that could name `url` could have the token sent anywhere.
`sideEffectsCache.remote` keeps `packages`; its signing fields remain a
fallback through `Config::remote_cache_settings()`. Only
`remoteCache.publish` publishes pipeline tasks, so a machine that
publishes dependency builds does not start uploading task outputs and
logs. The section is
boxed on `Config` to keep CLI futures under clippy's large_futures
limit.
Each artifact kind has its own size limits: a dependency build keeps
64 MiB per file and per artifact, and a task result may hold files of
up to 4 GiB and 16 GiB in all. Blobs travel between files and servers
as streams, so no result is held in memory. On pnpr,
`PUT /-/pnpr/v0/artifacts/blob` stores one blob ahead of its envelope
and commits it only once its bytes match the integrity, and `HEAD`
tells a publisher which blobs to skip. `artifacts.quota` makes pnpr's
storage bounds configurable.
pnpm-testing-utils gains an in-memory Turborepo Remote Cache server for
tests.
On pnpr, `artifacts.orgs` declares who may read and who may publish what
an organization owns: its signed artifacts, its compiler cache, and its
pipeline run records. It replaces the username-equals-organization rule
for artifacts, `artifacts.compilerCaches`, and `pipeline.workspaces`.
The shared-artifact store asks an `OrgAccess` policy the server builds
from it, and `CallerOrgs` replaces the duplicated authorize functions.
Pipeline run records are keyed `<org>/<workspace>/<run id>` under a new
`.pipeline-runs/v1` namespace; `pnpm pipeline --report` sends
`remoteCache.org`.
- `GET /-/pnpr/v0/sign-in` lists the OIDC providers that offer browser
sign-in, so the UI's sign-in screen knows which buttons to show.
- `/-/oidc/{provider}/login?return=ui` records in the sealed login
cookie that the sign-in ends in the web UI. The callback then puts a
one-time code in an HttpOnly `__Host-` cookie and redirects to
`/-/ui/sign-in/oidc` instead of showing the token. The redirect is
relative, so it works behind a path prefix. No URL carries the code
or the token, so a forwarded link cannot sign another browser in.
`return=ui` is refused when pnpr serves no web UI.
- `POST /-/oidc/handoff` trades the code in that cookie for the session
token and clears the cookie. A code works once, within a minute.
Codes live in memory next to the sessions they carry, so the existing
affinity requirement for multi-replica OIDC sign-in covers them.
- Browser sessions and handoff codes are grouped in `BrowserSessions`.
Password sign-in needs no server change: the UI uses npm's login
endpoint.
The bundled web UI moves to pnpr-web 0.2.5, which adds the sign-in
screen.
pnpr now serves its web UI at /-/ui/ from the same origin, so the UI
needs no CORS setup and no server URL. The UI ships as the separate
`@pnpm/pnpr-ui` package, an optional peer dependency of `@pnpm/pnpr`.
pnpr finds it next to its own package (npm and pnpm both link a resolved
peer beside the package that declares it), or reads it from `ui.dir`,
and `ui.enabled: false` turns it off.
- Serving: files from the UI directory, `index.html` for every other
path outside `assets/`, a 404 for a missing hashed asset. The page
gets a relative `<base>` so deep routes and path prefixes work without
an absolute Vite base. Responses carry a CSP that keeps scripts and
requests on this origin.
- The UI source stays on bit.cloud. `pnpr/ui` is a workspace of its own
that pins `@pnpm/pnpr.apps.pnpr-web` from bit.cloud's registry and
builds it into `pnpr/npm/pnpr-ui/dist`.
- Release: a `build-pnpr-ui` job builds the bundle. The verify and
publish jobs download it. The smoke test installs the UI beside the
wrapper and checks that pnpr serves it. `@pnpm/pnpr-ui` releases in a
fixed group with `@pnpm/pnpr`.
With `auth.scim.token`, pnpr serves the SCIM 2.0 `Users` endpoints and
discovery documents under `/-/pnpr/v0/scim/v2`, authenticated only by that
token.
An OIDC sign-in creates no account row and its session lives in one
replica's memory, so deleting an account is not enough to deprovision
anyone. The SCIM directory is therefore one record in the hosted store,
reloaded on the managed-state TTL like rosters and rules. Authentication
refuses any user identity the directory marks inactive, and so does the
npm login endpoint. Deactivating or deleting a user also revokes its
stored tokens, removes its password account, and drops this replica's
browser sessions for it.
A deleted user stays in the record as inactive and unlisted, so the
username keeps being refused until it is provisioned again. Provisioning
creates no password: the identity provider is the sign-in path.
Follow-up of pnpm/tasks#111.
`npm access grant`, `npm access revoke`, and `npm access list packages
<scope:team>` now work on a registry with `rulesManagedBy: api`. A grant
edits the access and publish lists of the package's own `packages:` key
through the same read-modify-write path as the rules admin API, and stores
only the lists it changes, so untouched lists keep following the YAML.
A package that only a scope pattern covers is refused, because editing the
pattern would grant the team every package in the scope.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A token kept authenticating after its account was gone: after a removal
whose token revocation failed, after a login that raced a removal, or
after an operator deleted the account from the htpasswd file by hand.
Authentication lookups now require a stored owner.
- The libsql, PostgreSQL, and MySQL backends join `users` in the
authentication lookups (`lookup`, `lookup_record`), so the check adds
no round trip on the request path.
- The local backend wraps its token store in `OwnedTokens`, which checks
the in-memory htpasswd map.
- Listing and revocation still see a removed account's tokens, so they
can be cleaned up.
Only a login issues tokens, so every legitimate token has a stored
owner. Tests that issued tokens for accounts they never stored now
store the account first.
The package rules admin API reports the version of a registry's stored
changes as `version` and `ETag`, and takes `If-Match` on `PUT` and
`DELETE`. A change based on a version another admin has since replaced
answers 412 and changes nothing; without `If-Match` the last change
still wins.
- Resetting the rules now stores the empty set of changes rather than
removing the record, so a reset goes through the same compare-and-set
as every other write and can be conditional too.
- The version is the SHA-256 of the stored record, `config` before
any change is stored. GET reads the rules view and the version from
one read of the stored record, so a replica that lags another's
write cannot pair a newer version with older rules.
- `If-Match` uses strong comparison and accepts a list of tags. An
unreadable header answers 400 rather than counting as absent.
- CORS allows `If-Match` and exposes `ETag`.
Follow-up of pnpm/tasks#111.
An OIDC login can map the groups its ID token lists to registry teams
under `auth.oidc[].login.groups`: a claim (default `groups`) and a list
of `{group, registry, team}` grants. A session whose claim lists a group
is a member of the granted team for as long as the session lasts.
- The memberships ride on the request identity, not the roster:
`Identity::User` carries `memberships`, and a `team:` token admits a
member of its roster or a credential that carries a membership of that
same roster's team. Removing someone from a group takes effect at their
next sign-in, and the grant never reaches the user's password logins
or tokens.
- The auth crate maps the claim to grants at sign-in and keeps them on
the session; the server resolves each grant to the registry's live
team directory per request.
- Startup rejects a grant that names no hosted registry, or a team a
config-managed roster does not declare.
Phase 4 of pnpm/tasks#111 (group mapping; SCIM is left out).
A hosted registry that sets `rulesManagedBy: api` lets the admins in
`auth.admins` replace its registry-level `access` and the `access`,
`publish`, and `unpublish` lists of the patterns its `packages:` map
declares, through `/-/pnpr/v0/admin/rules/{ecosystem}/{name}`, the pair
the registry directory lists. The patterns stay in config, because they
feed routing.
- `PackageRules` becomes a shared handle on a swappable `RuleTable`, so
a replaced table reaches every holder at once, including the resolver's
route context, which is built once per process.
- Changes are stored as a hosted-store record next to the team rosters.
`ManagedState` replaces `TeamRosters` and reloads both on one schedule
and lock; rosters load before rules, because a rule can name a team.
- Changes are validated strictly when written. When read back, a `team:`
reference to a team the roster lacks admits nobody, and changes that
cannot be read or applied close the registry until a read succeeds.
- `TeamsManagement` becomes `Management`, and `TeamConflict` becomes
`AdminConflict`, since both now cover more than teams.
- The registry directory reports `rulesManagedBy`.
Phase 3 of pnpm/tasks#111.
Admins listed in `auth.admins` can now list, create, re-key, and remove
accounts and revoke their tokens under `/-/pnpr/v0/admin/users`.
- `UserBackend` gains `list_users`, `create_user`, `set_password`, and
`delete_user`, implemented for the htpasswd store, libsql, PostgreSQL,
and MySQL. The htpasswd `UserStore` moves to its own module.
- Admin-created accounts ignore `max_users`, so an operator can keep
self-registration disabled and still add people. In the SQL backends
they still count against the `users` counter the cap reads, and a
removal takes one off it, so the cap stays exact.
- Removing an account revokes its tokens, deletes it, and revokes again,
so a failed revocation leaves the account in place.
- A login keeps its token only while the bcrypt hash it verified is
still stored; a registration only while the stored account accepts
the password that created it. A removal, a re-creation, or a password
change during the login revokes the token.
- An htpasswd edit runs whole on a blocking thread under a mutex and
becomes current only after its write succeeds, so writes stay ordered
and a failed write changes nothing.
- `require_admin` is shared by the team and account endpoints.
Phase 2 of pnpm/tasks#111.
A hosted registry that sets `teamsManagedBy: api` accepts `pnpm team`
create, destroy, add, and rm from the usernames in `auth.admins`. Other
registries keep answering `teams_config_managed`.
- `team:` tokens read a shared, swappable `TeamDirectory` per registry
instead of a member set copied at config load, so a roster change
reaches every rule that names the team at once.
- The roster lives in the hosted store under `.team-rosters/v0`, written
with compare-and-set and retried on conflict. The YAML `teams:` map
seeds it until the first change is stored.
- Each replica reloads the rosters once its copy is older than 10
seconds. Only the first load after startup holds up a request; later
reloads run in the background. A roster that cannot be read is served
empty, so its `team:` grants admit nobody, and the load is retried
after one second.
- Destroying a team a `packages:` rule names is refused with 409.
- Roster edits hold the reload lock, so a reload that read the store
before an edit cannot publish its older copy after it.
- Team write bodies are capped at 64 KiB.
- The registry directory reports `admin` and each hosted registry's
`teamsManagedBy`, so the web UI can offer edit controls.
- CORS allows PUT, POST, and DELETE. Credentials travel only in the
Authorization header, so allowing writes does not expose them to CSRF.
Phase 1 of pnpm/tasks#111.
Code blocks that hold only shell commands now use the `sh` language,
and blocks showing a file's contents carry a `title` naming the file,
across the pnpm v12, pnpm v11, and pnpr docs. Bold file-name labels
above code blocks are folded into the titles.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Quay redirects blob downloads to cdn01.quay.io and its sibling CDN hosts.
The OCI download allowlist only covered the Docker Hub and GHCR CDNs, so
the redirect was refused and every Quay layer pull through pnpr failed
with a 502. Allow cdn.quay.io and cdn01 to cdn06.quay.io for the
https://quay.io origin.
Closespnpm/pnpm#16548
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Upstream packuments may omit dist.integrity, leaving pnpm clients unable to
install those versions through pnpr.
When pnpr fetches a packument from a caching upstream, it downloads each
version that lacks a usable integrity (capped at 64 per fetch), verifies a
declared legacy shasum, caches the tarball, and persists the computed SHA-512
into the cached packument. Resolution and later tarball requests then use the
same integrity.
Pinning is per version: a version that cannot be downloaded, fails its
shasum, or has a malformed dist keeps its upstream metadata and is refused by
the tarball route, while the rest of the package stays installable. Pinning
runs only on an upstream fetch, not on cached reads, so unpinnable versions
cost one request per refresh. The downloads run without the striped package
lock, which hosted publishes share. Each tarball is synced and closed but
not promoted. Under the lock, pnpr re-reads the cached packument, writes a
pin only where that version still declares the same tarball and shasum
without a usable integrity, persists the packument, and only then renames
those tarballs into the cache; the rest are abandoned. A refresh that lands
meanwhile is kept, a published pin always describes the cached bytes, and
a cached tarball never lacks its published pin. If the packument write
fails, the staged tarballs are abandoned and the unpinned packument is
served. Packument refreshes and authoritative-404 purges take the same lock
for their cache writes, and pinning publishes nothing when the cached
packument is gone. Cache-disabled
upstreams are left unchanged because the computed pin cannot be stored.
Related to pnpm/tasks#24
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Zoltan Kochan