Complete production-readiness requirements for pnpr Section A.
- Persisted-document format versions: introduce formatVersion to packuments,
OCI images, crate documents, and state journals, rejecting documents written
by newer format versions.
- CLI surface grammar: support `pnpr serve` subcommand, and add parse_cli_interval
for duration units (s/m/h/d) on --packument-ttl, --min-age, and --max-age.
- Strict config validation: reject unknown fields, disallow removed configuration
blocks, and normalize field naming across config and CLI flags.
- Client protocol versioning: add capability handshake at /-/pnpr, v0 resolve
and publish endpoints, policy echo in done frames, and structured error codes.
- Distribution improvements: add Node.js fallback wrapper, fix engines range,
and package tini in pnpr Docker container.
Relates to pnpm/tasks#131.
pnpr-error depended on axum, bcrypt, object_store, reqwest, rusqlite,
serde_json, tokio and tracing on every non-wasm target. pnpm-config depends
on pnpr-registry for Ecosystem and PackagePattern, and pnpr-registry and
pnpr-package-name both depend on pnpr-error, so every consumer of pnpm-config
compiled and linked the registry server's HTTP, storage and auth crates.
- pnpr-error: those dependencies are optional and enabled by a new `server`
feature. The response module (IntoResponse, public_message, log_kind) and
the Upstream, Bcrypt, Sqlite, JoinError, ObjectStore and Json variants are
gated on it, replacing the cfg(not(target_family = "wasm")) gate.
- The pnpr crates that use those variants or the response rendering (auth,
osv, pipeline-runs, pnpr, search, shared-artifacts, storage, upstream)
enable `server`. pnpr-config only builds InvalidConfig and does not.
- pnpr-registry drops its unused pnpr-error dependency.
- pnpr-package-name tests compare Display output instead of
public_message(), so they pass without the server feature.
- Three workflow_run workflows carry their zizmor ignore directive on the
trigger line. zizmor did not apply the standalone comment, which failed the
zizmor job on fork PRs.
Closespnpm/tasks#134.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A user a SCIM client provisioned has no password account until they sign
in, so anyone could take the name first through npm adduser and keep it.
Registration now treats every username in the SCIM directory as reserved,
like admin and OIDC-bound names. Provisioned users sign in through OIDC
or get their account from an admin. A password account that exists when
its user is provisioned keeps working.
A login that creates an account checks the name again before issuing a
token, so a provisioning that lands mid-login takes the account back, as
does a directory that can no longer be read.
Raised in review of pnpm/pnpm#17046.
- Implement GET /-/user/org.couchdb.user:{name} returning CouchDB user document
- Record initial publisher in packument maintainers list and retain on subsequent publishes
- Support $owners token in package publish and unpublish access policies
- Support npm owner mutations via packument rev updates
- Store package and version tombstones under .tombstones/v0 to prevent squatting and version reuse
Part of https://github.com/pnpm/tasks/issues/131
Prevent self-registration from squatting configured admin usernames,
OIDC login, and workload bindings (Section B1 of
https://github.com/pnpm/tasks/issues/131):
- Reject self-registration attempts for reserved usernames with 403
Forbidden and a descriptive error message.
- Add `is_reserved_name` check on `IdentityConfig` covering `auth.admins`,
OIDC user logins, and OIDC workload bindings.
- Add `pnpr add-user <username>` CLI command for out-of-band admin
bootstrapping and password updates.
- Preserve normal CouchDB login for existing accounts matching reserved
names when provided with the correct password.
Part of https://github.com/pnpm/tasks/issues/131
Add pre-buffer authorization for package writes, upstream request coalescing,
login rate limiting, and request timeouts to pnpr.
- Pre-buffer authorization: package mutations (publish, update, dist-tags) check
caller authentication and write permissions before reading the request body.
- Upstream coalescing (singleflight): deduplicate concurrent in-flight fetches for
the same package/namespace so cache misses do not hammer upstream registries.
- Rate limiting and concurrency limits: throttle failed login attempts by IP and
username with 429 Too Many Requests and Retry-After headers, and limit concurrent
login verifications and package publish buffering.
- Request timeouts: add configurable requestTimeout layer to pnpr HTTP service.
Ref: https://github.com/pnpm/tasks/issues/131
Listen for Unix SIGTERM signals alongside SIGINT (Ctrl-C) to perform
graceful server shutdown in container environments such as Docker and
Kubernetes.
Implement readiness checking on storage backends (local filesystem and
S3), cache directories, and auth backends (htpasswd, SQLite, Libsql,
PostgreSQL, MySQL). Expose dedicated `/-/ready` and `/-/live` probes
mounted globally and under registry prefixes, and make `/-/ping` return
503 Service Unavailable when storage or authentication is unhealthy.
Refs: https://github.com/pnpm/tasks/issues/131
The package rules admin API reports the version of a registry's stored
changes as `version` and `ETag`, and takes `If-Match` on `PUT` and
`DELETE`. A change based on a version another admin has since replaced
answers 412 and changes nothing; without `If-Match` the last change
still wins.
- Resetting the rules now stores the empty set of changes rather than
removing the record, so a reset goes through the same compare-and-set
as every other write and can be conditional too.
- The version is the SHA-256 of the stored record, `config` before
any change is stored. GET reads the rules view and the version from
one read of the stored record, so a replica that lags another's
write cannot pair a newer version with older rules.
- `If-Match` uses strong comparison and accepts a list of tags. An
unreadable header answers 400 rather than counting as absent.
- CORS allows `If-Match` and exposes `ETag`.
Follow-up of pnpm/tasks#111.
A hosted registry that sets `rulesManagedBy: api` lets the admins in
`auth.admins` replace its registry-level `access` and the `access`,
`publish`, and `unpublish` lists of the patterns its `packages:` map
declares, through `/-/pnpr/v0/admin/rules/{ecosystem}/{name}`, the pair
the registry directory lists. The patterns stay in config, because they
feed routing.
- `PackageRules` becomes a shared handle on a swappable `RuleTable`, so
a replaced table reaches every holder at once, including the resolver's
route context, which is built once per process.
- Changes are stored as a hosted-store record next to the team rosters.
`ManagedState` replaces `TeamRosters` and reloads both on one schedule
and lock; rosters load before rules, because a rule can name a team.
- Changes are validated strictly when written. When read back, a `team:`
reference to a team the roster lacks admits nobody, and changes that
cannot be read or applied close the registry until a read succeeds.
- `TeamsManagement` becomes `Management`, and `TeamConflict` becomes
`AdminConflict`, since both now cover more than teams.
- The registry directory reports `rulesManagedBy`.
Phase 3 of pnpm/tasks#111.
A hosted registry that sets `teamsManagedBy: api` accepts `pnpm team`
create, destroy, add, and rm from the usernames in `auth.admins`. Other
registries keep answering `teams_config_managed`.
- `team:` tokens read a shared, swappable `TeamDirectory` per registry
instead of a member set copied at config load, so a roster change
reaches every rule that names the team at once.
- The roster lives in the hosted store under `.team-rosters/v0`, written
with compare-and-set and retried on conflict. The YAML `teams:` map
seeds it until the first change is stored.
- Each replica reloads the rosters once its copy is older than 10
seconds. Only the first load after startup holds up a request; later
reloads run in the background. A roster that cannot be read is served
empty, so its `team:` grants admit nobody, and the load is retried
after one second.
- Destroying a team a `packages:` rule names is refused with 409.
- Roster edits hold the reload lock, so a reload that read the store
before an edit cannot publish its older copy after it.
- Team write bodies are capped at 64 KiB.
- The registry directory reports `admin` and each hosted registry's
`teamsManagedBy`, so the web UI can offer edit controls.
- CORS allows PUT, POST, and DELETE. Credentials travel only in the
Authorization header, so allowing writes does not expose them to CSRF.
Phase 1 of pnpm/tasks#111.
Compile the shared Rust CLI for wasm32-wasip1-threads and supply filesystem,
network, process, terminal and cancellation capabilities through a Node host.
Select host implementations at compile time so native builds retain their
native dependency/features and execution paths.
Bundle the WASM runtime in the regular npm wrappers and select it from the
WebContainer runtime marker. Keep Node launchers usable without installation
scripts. Native installation replaces all four launchers with native bytes.
Regenerate shims after upgrades or dependency builds replace Node targets with
native binaries, including before dependent lifecycle scripts run. Preserve
collision winners and publish replacement shims with executable permissions.
Reject non-descendant permission templates and intermediate symlink escapes, validate before creating directories, and avoid path-based cleanup races.
Restore direct native aliases in both version-management implementations.
Retain process cleanup after failed stdin shutdown. Prevent cross-origin redirects from replaying login credentials in both CLI lines, and strip npm-otp from cross-origin WASM requests. Avoid repeated selected-snapshot and removed-bin scans.
Preserve resolution, lockfile, integrity and build-approval behavior. Use an
isolated WASM store with rollback journals and process leases. Patch WASI path
normalization for multiple parent segments and retain that patch in bundles.
Add pinned tooling, release packaging, real-browser workflow and selection
tests, documentation and a release note. Document host limits, including the
Corepack cold-download failure in WebContainer's streaming hash implementation.
Controlled native CLI cases showed no separated slowdown, and the latest
hoisted rebuild and repeat-install fixture improved. Retain the documented
microbenchmark uncertainty and host limitations.
Related to pnpm/tasks#63.
Pin pnpm's rustfmt fork and use a cached wrapper for local formatting,
CI, pre-push checks, and editor integration. The formatter has its own
dated nightly runtime while the workspace keeps its existing compiler.
Choose chain layout by call count and argument complexity, with a
40-column allowance for short expressions. Attach the first method when
the complete first line would otherwise end within the continuation
indentation. Preserve simple receivers,
cap intermediate leading accesses at column 80 and final accesses at
100, and separate fields and
await after wrapped methods. Apply the same rules in conditions.
Keep Max heuristics and compact struct literals, with an independent
35-column destructuring limit. Reformat pnpm and pnpr, extract only the
helpers and test modules needed for existing size limits, and fix macro
commas without relaxing lint rules.
Related to pnpm/pnpm#14562 and pnpm/pnpm#14862.
taplo padded TOML with spaces so that things line up in a column, which makes
unrelated lines show up in diffs. Two separate options did this.
`align_entries` padded the key of every entry so the `=` signs share a column.
In `[workspace.dependencies]` that meant padding every entry out to the width of
the longest crate name, so adding or renaming one crate reflowed the whole
block. `align_comments` did the same for trailing comments, so the clippy lint
tables in the root manifest repadded every comment whenever a lint name grew.
Set both to false and reformat the 107 files taplo covers. The diff is
whitespace only: `git diff -w` reports `.taplo.toml` as the only file that
differs. Verified with both the locally installed taplo 0.10.0 and the 0.8.1
that CI pins, so `pnpm ci:toml-fmt` passes on both.
Split Rust production modules and test suites across the repository to keep
production files within 400 code lines and test files within 800. Group
modules by responsibility and test scenarios while retaining public APIs
and existing test binaries.
Move Node release public keys into individual data assets and update their
generator. Preserve key bytes, fingerprints, and ordering. Update module
imports, documentation links, and snapshot locations after extraction.
Document the limits and verify them with the proposed upstream rule's
lexer without depending on an unmerged lint.
Related to pnpm/pnpm#14562.
* refactor: enforce 40-line production Rust functions
Adopt perfectionist::overly_long_function across the Rust workspace with
tests exempt. Refactor all existing violations without legacy suppressions
and place helper contracts at their declarations.
Related to pnpm/pnpm#14562.
* chore: omit release note for internal Rust refactor
Pin the perfectionist dylint library to a revision instead of a tag and
enable the four rules that revision adds: arbitrary_source_item_ordering,
core_instead_of_std, excessive_cognitive_complexity with a limit of 10,
and redundant_derive_more_forward_template.
The tree is made to pass them by refactoring rather than by exemptions:
no #[expect], no #[allow], and no raised limit. Oversized functions are
split into named steps, conditions that carried a meaning are given
names, three-way outcomes become enums, repeated parameter lists become
types that own the shared values, and the nearly sixty CLI config
overrides that repeated the same three shapes are driven from macro
tables.
cargo-dylint refuses a library built against a newer dylint_linting, so
CI moves to cargo-dylint/dylint-link 6.0.4 to match the pinned revision.
The 6.0.1 bootstrap failure the previous 6.0.0 pin worked around is fixed
there.
Related to https://github.com/pnpm/pnpm/issues/14562. That issue plans
seven size-and-shape rules; only excessive_cognitive_complexity has
landed upstream so far. It also proposed a per-site #[expect] for every
existing violation, which this PR deliberately does not do.
Staged publish records live in the hosted store, which every replica of an
S3-backed deployment shares, but an approval read the record and replayed the
held publish without ever writing the record back. Two approvals of one stage
reaching two replicas therefore both ran the publish, and a rejection that
landed between another replica's read and its publish did not stop it.
The approval now claims the record first: it rewrites it with the time the
approval started, conditional on the bytes it read. The object-store backend
puts that write under `If-Match` on the record's `ETag`; the local backend,
which one process owns, compares the bytes under a staged-write lock. A second
approval finds a record that is no longer the one it read and answers 409, or
404 once the winner has consumed the stage. The claim is released on every
failure path, and expires after ten minutes so a replica that died mid-approval
does not strand the stage.
Staged records are also written create-only now, so a record can never be
written over one another request owns.
Adds a multi-replica test suite that drives two routers over one object store
the way a load balancer spreads requests over two containers: concurrent
publishes of one package keep both versions, a stage created on one replica is
approved on another, two replicas approving one stage publish it once, and a
dist-tag written on one replica is served by the other.
Related to pnpm/pnpm#12199.
Add offline OCI blob collection and shared upload sessions for S3-backed
registries. These changes address the correctness and operations section
of pnpm/pnpm#14630.
Use immutable objects for request chunks and compare-and-set writes for
the session record. Only a successfully recorded chunk advances the
accepted offset. Completion assembles the chunks on local scratch for
digest verification, then uses the existing multipart blob path. Losing
a replica's scratch no longer loses an accepted upload. Conditionally freeze
the accepted chunks and digest before promotion, keeping failed promotion
retryable with the same digest. After promotion, session cleanup is best
effort; cleanup failures do not turn a committed blob into a failed response.
Closing or expiring a session invalidates stale writers before chunk deletion.
Keep garbage collection offline so no publisher can introduce a new
reference between the mark and delete phases. Walk retained manifests
and index children rather than tags alone, and validate their integrity
before deleting any candidate. Inventory blob files separately from the
request-path package listing to include unpublished, document-only, and nested
image repositories without making catalog requests enumerate blob populations.
Stream the inventory into temporary SQLite storage so a large registry does
not require its complete file listing and candidate set in memory.
Startup expiry and large-blob multipart transfer were already present
in the initial OCI implementation. Extend expiry to shared sessions and
reuse and test multipart cleanup for both promotion and session chunks.
Related to pnpm/pnpm#14630. This affects pnpr only; there is no pnpm CLI
implementation to mirror.
Centralize npm, Cargo, and Python package-name validation and normalization in `pnpr-package-name`. Make storage and cache keys canonical by construction, and keep journal recovery keyed by the recorded ecosystem.
This removes protocol-crate dependencies from `pnpr-config` and eliminates duplicate normalization in server and resolver call sites.
Related to pnpm/pnpm#14599.
The storage layer serves three ecosystems but was named for one. Rename its
vocabulary so a Cargo index file and a Simple API page are documents and a
`.crate` and a wheel are blobs.
`pnpr-storage` now exposes `read_hosted_document`, `open_hosted_blob`,
`reserve_hosted_blob`, `finalize_blob_slot` and their peers on both the
filesystem and the S3 backend, and a publish write conflict is
`RegistryError::DocumentWriteConflict`, logged as `document_write_conflict`.
`PackageName` keeps its name: it holds a package's name and never a version,
and a name is what a crate and a Python project have too. The npm surfaces
keep their own packument and tarball vocabulary, which is accurate there.
Separately, a name now rejects `?`, `#`, `%`, whitespace, and control
characters. A request path is percent-decoded before a handler parses a name
from it, and a name is interpolated into the upstream URL, so `GET /foo%23bar`
was authorized and cached as `foo#bar` while fetching `foo` — a package rule
that named `foo` never ran.
The journal manifest's `packument_file` and `tarballs` aliases had no
coverage, so a sealed entry that spells its keys that way now has a recovery
test.
Related to pnpm/pnpm#14599.
`PUT /-/pnpr/v0/publish` takes a batch whose entries each name their
ecosystem and carry what that surface's own publish endpoint takes, with
the binary parts base64-encoded. An entry that names none is an npm
publish document, so a body the npm batch endpoint accepts is already a
valid one.
The address is in pnpr's own namespace rather than beside the npm batch
endpoint, which is part of the npm surface and answers under `/npm/`
with it. `GET /-/pnpr` advertises the protocol as `publish: [0]`, and now
answers for a registry-only tier, which has a pnpr protocol of its own
for the first time.
Every entry is parsed, authorized and verified before any of them is
staged, every blob is staged before any of them is committed, and the
commit is the single journal transaction the publish flow already had:
a release that spans ecosystems becomes visible all at once. The
per-ecosystem publish endpoints keep their own behavior; they and the
batch now share one staging path, `StagedPublish`, which carries the
ecosystem of the package it staged.
Closes one checkbox of pnpm/pnpm#14599.
Add Cargo and Python registry protocols to pnpr. Concrete registries declare
an ecosystem, and routing filters mixed router sources by the requested
protocol. Ecosystem prefixes provide default and named registry endpoints
while preserving the original npm aliases.
Reuse hosted document and blob storage, package access rules, upstream
metadata caching, and verified artifact streaming across the new surfaces.
Keep upstream artifact cache identities bound to metadata checksums. Check
metadata before cache lookup so changed or removed entries take effect.
Honor package-specific Cargo privacy rules when advertising auth-required.
Require authentication before reading Python upload bodies. Bound multipart
boundaries and use the existing regex crate for delimiter searches, checking
boundary suffixes so binary lookalikes remain part of the uploaded file.
Reject Cargo archive traversal, links, oversized decompressed streams, and
manifests whose package identity differs from the publication metadata.
Reject package configuration keys that collide after normalization.
Apply route allowlists to Cargo and Python metadata and artifact fetches,
including redirects. Require secure same-origin destinations for configured
headers. Rebuild headers on every approved redirect so cross-origin metadata
and artifact downloads succeed without forwarding upstream credentials.
Share this redirect loop with the existing network metadata fetcher and retain
its request-budget guard while response bodies are read. Transfer permits to
streamed artifact responses so body consumption or cancellation releases them.
Keep one pnpr timeout budget across redirects and the final response body.
Stream through a bounded producer whose deadline runs independently of
downstream polling. Reject hosted Python blobs absent from publication metadata.
Abort publication on an immutable object-store conflict and remove the
conflicting staged file. The filesystem backend remains single-writer;
replicated deployments use the object-store backend. Journaled publication
across all registry surfaces remains tracked in pnpm/pnpm#14599.
Add protocol unit tests and HTTP regressions for publication, routing,
authentication, content negotiation, caching, and integrity validation.
Add an exact-origin CORS allowlist for separately hosted browser registry UIs.
Complete hosted search pagination and maintainer discovery, record authenticated publishers,
and serve npm-compatible organization package maps. Apply registry routing and access rules
before counting or returning local packages.
Add opt-in upstream search and organization discovery. Use only configured upstream credentials.
Treat upstream pages as a visible, deduplicated sequence so filtered entries neither underfill pages
nor leak through provider-reported totals. Derive publisher and organization permission metadata
from pnpr authorization rather than client or upstream service-account values.
Closespnpm/pnpm#14543
An artifact was stored under its own envelope digest, so a second build
for the same input joined the first as another variant rather than
colliding with it. Anyone who could sign could add a different artifact
for a dependency that already had one, and neither side would notice.
It is now stored under a slot naming what it is for — the input key and
the compatibility constraints — so one input admits one artifact per
platform, the same way a registry makes `name@version` mean one tarball.
A different envelope for a claimed slot is a 409; the identical one stays
idempotent.
Releasing a claimed slot is not something the API can do: a publishing
token is the organization, so if publication could replace, a stolen
token could swap the artifact for a dependency nobody has looked at in a
year.
Extracting config was blocked by a cycle: config reached into s3 for
S3Settings and build_s3_store, s3 into storage, and storage back into config.
Neither of those items belonged in s3. S3Settings is the deserialized YAML
s3: block — config's own data, filed under the backend that consumes it. And
build_s3_store builds an AmazonS3Builder from that block; it touches
object_store but nothing in our s3 module, so it sits just as well next to
the settings it reads. Moving both leaves config with no dependency on s3 at
all and the cycle gone, with no behavior change: the store is still built
once at config-load time, which is what makes constructing the storage layer
infallible.
config is then a leaf, so it becomes pnpr-config, carrying config.yaml (the
bundled default it include_str!s) with it.
Two items the boundary caught as reaching across: RedactedHeaders and
UpstreamConfig::with_defaults, both now pub. with_defaults also gained
#[must_use], the same public-API lint consequence the last extraction saw.
The suite is unchanged at 714, of which 144 now run under pnpr-config.
Related to pnpm/pnpm#14191.
pnpr was one 42k-line crate, though pnpr/AGENTS.md has always specified the
pnpr-<short-name> convention for registry-only crates and none existed.
Start at the bottom of the module graph, where the boundary is already
closed: error depends on nothing, package_name on error, registry on
package_name, policy on registry, and no member reaches back up.
Those four become pnpr-error, pnpr-package-name, pnpr-registry, and
pnpr-policy — one crate each, named for what it holds. The workspace has 77
Rust crates and not one named core, common, shared, or utils; such a name
describes a position in the graph rather than a subject, which is what lets
it collect anything. AGENTS.md now says so, so the next extraction doesn't
have to relearn it.
Consumers name pnpr_error:: and friends instead of crate::, so the layering
is checked by cargo rather than by convention. The backend features forward
to pnpr-error, which is the crate that names those backends.
Three things the crate boundary surfaced. AccessList::new becomes pub — the
one item reaching across. Three methods gained #[must_use], since pedantic
public-API lints apply where pub(crate) had exempted them. And every
first-party pnpr* crate needs a deny.toml exception plus a licenses.clarify
block, because their PolyForm license text scores below cargo-deny's
confidence threshold; without both, Cargo Deny fails the crate as unlicensed.
All four feature combinations build, cargo deny check passes, and the suite
is unchanged at 714 — 65 of which now run under the four new crates.
Related to pnpm/pnpm#14191.
Zoltan Kochan