mirror of
https://github.com/pnpm/pnpm.git
synced 2026-10-31 12:09:04 -04:00
docs-migration-runtime-setting
366
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zoltan Kochan
|
be2f02ffc8 |
fix(pnpr): emit the access record outside the request span (#17267)
With the default pretty format, every pnpr::access line printed each field twice and user three times. emit_access_log entered the per-request span before emitting, the span records the same fields, and the compact formatter appends the current span's fields after the event's. The JSON format hid the same duplication under a span object. The access event now fires outside the request span. It carries every documented field itself, so nothing is lost in either format. The span keeps request_id, method, uri, client_ip, user_agent, and user for the events emitted inside a request: the audit records inherit them, and the authentication layer records user before the handler runs. The span's status, latency_ms, and bytes slots were only filled after the handler returned, when no event inside the request could see them, so they are removed along with record_span_and_metrics. The subscriber construction moves from main.rs into pnpr::logging::log_subscriber so a test can run a request through the real pretty and JSON configurations and assert on the formatted line. The audit-log test captures values recorded on the span after it opens and checks the span's user against the audit actor. Closes pnpm/tasks#195 |
||
|
Zoltan Kochan
|
3bca7ffd2b |
fix(pnpr): record a token's last use off the request path, at most once a minute (#17266)
Every token-authenticated request awaited an UPDATE of last_used_at before the handler ran. With the libsql backend that is a round trip to the primary even when an embedded replica answers the lookup locally, so a replica removed almost none of the latency for authenticated traffic. LastUsedWriter in pnpr-auth now owns the write. It skips the write while the stored time, which the authenticating lookup already read, is under a minute old; claims the interval per token in memory so a burst of concurrent requests from one token writes once even before the first write lands or the replica syncs it back; and runs the write in a spawned task the request never waits for. Both middleware sites, bearer tokens and OCI tokens' parents, go through it. GET /-/npm/v1/tokens therefore reports a last-used time accurate to within a minute. Related to https://github.com/pnpm/tasks/issues/194 |
||
|
Zoltan Kochan
|
30f6d58a56 |
fix(pnpr): bound libsql auth reads, and a write's wait for a connection, by the auth deadline (#17252)
Only reads ran under the auth timeout, so a write that never got an answer held its pool slot indefinitely, and once every slot was held that way each lookup waited for one until it timed out. Putting writes under the deadline instead would report a write the database committed late as a timeout. The pool now has a read and a write entry point. A read runs under the deadline end to end. A write is bounded only while it waits for a connection, before anything is sent, and then runs to completion, the half of the sqlx model libsql can follow without a server-side statement timeout. A database that stops answering writes can still hold every connection, but the pool recovers once those writes return. Connections open on the blocking thread pool, with at most as many in flight as the pool has connections, so a replica's stalled background sync can neither hold a slot past the deadline nor pile up blocked threads. Also drops the claim that a pooled connection avoids new TCP or TLS handshakes. Measured against a local libsql server, one pooled connection served 50 token lookups but libsql opened 45 TCP connections for them; the old shared connection behaved the same. |
||
|
Zoltan Kochan
|
7fb2bfab4a |
docs(pnpr): correct configuration keys, defaults, and commands (#17245)
Part of https://github.com/pnpm/tasks/issues/191. Wrong claims: - cli: drop the nonexistent `add-user --password` flag and document the terminal/stdin password read; `oci-gc --min-age` and `cache-gc --max-age` are the flags (the `-secs` forms are aliases); `--from` defaults to the `storage` key, not an internal field; `info` and `http` log the same access records; only global and server options have PNPR_* variables. - configuration: `packumentTtl` defaults to 5m; the minimal config now sets `auth.htpasswd.file`, and the auth section says accounts live in memory without it or a backend; grouped hosted registries default their org to `<ecosystem>~<name>`; the handshake is served for any surface; the pipeline surface also satisfies "something must be served"; add `package.update_maintainers` to the audit list. - tls-and-reverse-proxy: remove the invalid `publicUrl:` YAML key and the nonexistent `rate_limiting` setting. - backup-and-restore: `pnpr verify` hashes by default (`--check-hashes` does not exist, `--skip-hashes` skips only the hash comparison); `migrate-storage --verify` compares bytes. - webhooks: audit events do not cover publishes or token creation; the example uses a real action and the `actor` field. - cli: the `RUST_LOG` example keeps `pnpr::audit=info`. - install-acceleration: the `--public-url` host is exempt from the private-address refusal. - shared-side-effects-cache: `pnpm pipeline` publishes workspace-task artifacts. Missing facts: `routes.allowedPrivateNetworks`, the upstream `search` key and canonical `maxAge`, built-in `npm.jsr.io` route, relative path resolution, `log` defaults and `type`, `${VAR:-default}` substitution and the full secret-field list, `oci` defaults, `artifacts.quota` bounds, `migrate-storage` destination rules, `--osv false`, `--packument-ttl` default, S3 `forcePathStyle`/`allowHttp` defaults, grouped org layout, OIDC key table, container image section, `backend:` ignoring the htpasswd and token files. Duplicates and links: health probe bodies, reload procedure, and the resolver allowlist now link to their one home; broken `#secret` anchor fixed; links into the pnpm docs use absolute pnpm.io URLs. Stale history: drop "pnpr v1" and the per-paragraph "Since alpha.N" / "Since pnpm 12.x" prefixes, and the contributor-only extensibility text. Also corrects the `as_filter_directive` doc comment, which claimed `http` quiets the rest of the crate. |
||
|
Zoltan Kochan
|
ae47bbf062 |
fix(pnpr): sync the embedded replica on a connection of its own (#17232)
libsql's read-your-writes pulls the primary's frames on the connection that issued the write and resets that connection's pager when the pull ends, so a read in flight on the same connection then fails with "file is not a database"; the schema setup at startup hit it deterministically, since it holds a count query open across the counter transaction. Turn read-your-writes off and pull through `Database::sync` after each write instead, which uses a fresh connection. A login that only checks a password does not pull. Schema setup takes the sync as a hook and runs it before each read that depends on a preceding write, and the login that follows a lost insert race syncs before reading the winner's row. TokenBackend gains revoke_all_for_user, so revoking every token of an account is one DELETE and one pull instead of one pull per token; the pull is skipped when nothing was deleted and the replica lists no token for the account. The test's mock primary now serves the frames of its own WAL, modelling a WAL restart as the next generation, so the test covers a write reaching the replica through background sync, a replica seeing its own write and bulk revocation, and schema setup against an empty primary. Related to pnpm/tasks#183. |
||
|
Zoltan Kochan
|
26047ac598 |
fix(pnpr): run libsql auth operations on a pool of connections (#17237)
The libsql backend ran every operation on one shared connection. On a remote database that connection is a Hrana stream, and each response carries the baton for the stream's next request. An operation dropped while its request was in flight (a client that disconnected, the auth timeout) never received the new baton, so the connection kept the spent one and the server refused every later request with "Received a baton that has already been used". From then on every login and every authenticated request failed with a 500 until pnpr restarted. One shared stream also put concurrent operations into each other's transactions. Operations now check a connection out of a pool and run alone on it. A connection returns to the pool only after its operation succeeded outside a transaction; any other is closed and replaced on a later checkout. This is what sqlx does for the PostgreSQL and MySQL backends. Pooled connections keep their HTTP client, so reuse costs no new TCP or TLS handshake, which a connection per operation would have paid on every token lookup. backend.libsql.maxConnections sizes the pool, defaulting to 10 like sqlx. |
||
|
Zoltan KochanandClaude Opus 5.5
|
6f95957063 |
docs: lint prose with Vale (#17238)
* docs: lint prose with Vale Add Vale with an in-repo style, .vale/styles/pnpm, that turns the writing rules in AGENTS.md and the Google developer documentation style guide into lint rules. The stock Vale packages (Google, Microsoft, RedHat, write-good) were too noisy to block on, for example 1,300 passive-voice hits in the v12 docs alone, so the rules are written for this repo, following the GitLab model. Each one links the guideline it enforces. Levels: - error, fails CI: dashes, plain-word substitutions, condescending words, "please", "note that", hedges, puffery, optional plurals, exclamation marks, ellipses, curly quotes, generic link text, product spellings, sentence-case headings, repeated words, double spaces, implementation history in comments; in changesets, bold or italic emphasis, internal names and release-line names. - warning: future tense, sentences over 30 words, "instead of" and "so that" tails in changesets. - suggestion: passive voice. .vale.ini scopes the rules. Docs get every rule. Contributor Markdown and code comments keep dashes and headings as warnings until their roughly 8,100 em dashes are rewritten. Released changesets stay in .changeset/ and are not rewritten, so lint-prose.mjs lints only the changesets added since PROSE_BASE (the PR base in CI). The prose workflow installs a pinned, checksum-verified Vale release and runs `pnpm lint:prose`. The rest of this change rewrites the existing docs, READMEs, contributor Markdown, and code comments to pass the error-level rules. Only prose and comment text changed. Renamed headings keep their anchors. Spelling stays with cspell, which checks identifiers that Vale does not see. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: keep prose lint from flattening meaning --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Zoltan KochanandClaude Opus 5.5
|
cb496f5427 |
docs(pnpr): get zstd through Cloudflare's edge (#17230)
Cloudflare removes zstd from Accept-Encoding before a Worker runs, and from cf.clientAcceptEncoding too, so pnpr behind the recipe's Worker never sent zstd. The Worker now asks pnpr for "zstd, gzip", the value pnpm sends on archive requests, and returns the coded body with encodeBody "manual". Cloudflare passes a zstd body to a client that accepts it and decodes it for one that does not. Verified on a trial deployment: a tar for pnpm arrives zstd-coded, metadata for a gzip client is still gzip-coded, and a gzip-only client gets the tar uncompressed. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
79b4dd2d78 |
fix(pnpr): follow a Turso Cloud primary from the embedded replica (#17225)
Enable libsql's `sync` feature so the embedded replica (`backend.libsql.replicaPath`) can follow a Turso Cloud primary. The replica was built with libsql's `replication` feature only, whose protocol Turso Cloud stopped serving: pnpr failed to start with "Write delegation: Unimplemented ... deprecated version of sync". With the `sync` feature, `Builder::new_remote_replica` defaults to `SyncProtocol::Auto`: it probes the primary's `/info` route and uses the newer sync protocol (bootstrap from `/export`, frames from `/sync`, writes over Hrana) when the primary answers, and keeps the replication protocol otherwise. A self-hosted `sqld` answers 404 on `/info`, so it is unaffected. The new test runs a mock primary that serves those routes in front of a local libsql database and checks that the replica bootstraps from the export, reads locally, and writes to the primary. Without the feature it fails the way the issue describes. `connect` also creates the replica's parent directory, as the local stores do for their files, and with `syncInterval: 0` it runs the initial sync under the startup deadline, since libsql would otherwise bootstrap inside `Database::connect` with no bound. Closes pnpm/tasks#183. |
||
|
Zoltan Kochan
|
81e6e9ec1a |
docs: say where serverType pnpr applies and what a tar main base costs (#17227)
The registries page presented the pnpr layout as the layout of every pnpr registry, but a pnpr .tgz base advertises the npm layout, which the undeclared default already reconstructs; serverType: pnpr belongs on a tar base. The pnpr configuration page now says tarballs: tar may go on the main router and what that costs other clients. The Cloudflare recipe's Worker waits through a cold start instead of failing it, and the page notes that the edge rewrites Accept-Encoding. |
||
|
Zoltan Kochan
|
f880d91124 |
fix(pnpr): recognise a unique violation reported by Turso (#17223)
The libsql auth backend classified a duplicate row only through SQLite result codes. Turso's remote protocol reports it as a Hrana stream error whose message carries the code name, so the counter seed on a second start against the same database failed instead of being tolerated, and pnpr exited at startup. A taken username reported the same way answered 500 instead of the login or name-taken path. Parse the Hrana message the way the lock-conflict check already does. Related to https://github.com/pnpm/tasks/issues/183. |
||
|
Khải
|
7e81d63bdb |
refactor: replace bare bools with named enums and structs (#16501)
Several facts travelled through the resolver, the lockfile's YAML emitter,
the default reporter and pnpr as bare `bool`s that could not be read at
the use site: a `true`/`false` literal selecting which of two behaviors a
function performs, a `bool` travelling in a tuple with no name, two flags
where one implies the other, and a `bool` return meaning "which of two
things happened". Each of those now has a name.
In the resolver:
- `PeerWalkMode { Discovery, Final }` replaces `Walker::new`'s
`discovery: bool` and the traversal field it was stored in, and
`ChildOutputs::push` takes that mode instead of its own
`collect_dep_paths: bool`.
- `PeerIdRecording { Record, Skip }` replaces the `record_peer_ids: bool`
of the final-path pass, parameter and field.
- `ChildrenRecording` travels back to the callers of the children
realisation next to the children, instead of the `bool` that
`into_children` flattened it to; its callers ask it
`stales_other_occurrences`.
- `PackageRegistration { Created(Arc<str>), Existing(Arc<str>) }`
replaces the `(Arc<str>, bool)` the two package registrations
returned.
- `DependencySpec { alias, bare_specifier, optional, injected }` replaces
the `(String, String, bool, bool)` tuple that lived under two aliases,
`WantedSpec` and `ChildSpec`, and was read positionally in six files. Its
fields are named after `WantedDependency`'s, the resolved form they feed.
`hoisted_dependency` builds the shape both hoist rounds want, with the
rationale for its two defaults in one place.
- `collect_deps` takes a `DependencyGroup` instead of a manifest key and
an `optional: bool`, and derives both from the group, so a caller
cannot pair a key with the wrong flag.
- `WorkspacePeerSettings` groups the three consecutive booleans of
`resolve_peers_workspace` into named fields.
- `NodeKind { Link, Leaf, Branch }` replaces the `is_link`/`is_leaf`
field pair, of which only three of the four combinations were states.
- `ReusableResolution { Registry, Git }` replaces the `bool` inside the
`Option<bool>` that `reusable_resolution_is_git` answered.
- `reuse.rs` calls the shared `register_peer_dep_names` instead of
keeping a verbatim copy of it.
In the lockfile's YAML emitter, four `bool`s threaded through every node
and were read positionally by ten functions: `CollectionStyle`,
`FirstEntry`, `EntrySpacing` and `ScalarLines` replace them, in
`yaml_emit/style.rs`.
In the default reporter, `BlockPlacement { Scrolling, Pinned }` replaces
`Frame::emit`'s `fixed: bool`.
In the filesystem crate, `LockMode { Exclusive, Shared }` replaces the
`exclusive: bool` of `lock_file` and `try_lock_file`, and every
implementation maps it with a match: the standard library's methods,
Android's `flock` operation, and the WebContainer host lease's flag.
In the hoisted linker, `DirectLink { Unchanged, Created,
RootShadowRemoved }` replaces the `(bool, bool)` that linking one direct
dependency returned, of which only three combinations occurred.
In pnpr:
- `ResponseBody { Sent, Omitted }` replaces the `head: bool` of three OCI
response builders and derives itself from the request method in one
place.
- `MetadataFreshness { Cached, Refetched }` replaces the `refresh: bool`
of the OIDC provider-metadata read, and separates the freshness
question from the retry rate limit that shared the flag. A preparatory
commit moves that cache into `oidc/provider_metadata.rs`, which also
brings `oidc.rs` back under the file-length limit.
- `OrgGrant { Read, Publish }` replaces the `publish: bool` that main's
organization consolidation left on `CallerOrgs::authorize` and on
`authorized_owner_key`, so the pipeline-run, compiler-cache and
shared-artifact paths ask for a grant with one named type.
- `GroupMatching { shadowed, allow_unresolved }` replaces the
`(bool, &HashSet<&str>)` tuple parameter of the directory-dependency
freshness check, whose call site read `(false, &peers.shadowed)`.
- `reject_removed_blocks` takes the parsed `packages:` and `groups:`
blocks, whose distinct types make the transposition its two same-typed
flags allowed a compile error.
- `ExistingDestination { Replace, Keep }` replaces the `replace: bool`
behind `write_atomic` and `write_atomic_new`.
- `StaleEntry { Delete, Keep }` replaces the `delete_stale: bool` of the
cargo index cache read.
- The resolution cache's eviction helpers lose a `private_first: bool`
that their callers always passed as `true`, and the fallback branch
that only ran when it was `false`;
`CandidateScope { PrivateOnly, Any }` names the choice that remains.
- `CrateDocument::max_version` reads its two passes through named
closures instead of a `yanked_allowed` flag.
Every branch on one of these enums is an exhaustive `match` that names
every variant, with no wildcard arm, no `matches!` and no `==` against a
variant, so adding a variant is a compile error at each site that has to
decide.
The criteria behind these choices are written down in
pnpm/CODE_STYLE_GUIDE.md under "Boolean parameters and fields": which
questions a `bool` may answer, how its use sites must read, what a
neighbouring flag does to it, and the exemptions that keep most of the
workspace's `bool`s as they are. This branch adds the section, including
the rule that a flag named for one of several alternatives wants an enum
of those alternatives rather than a better name, and the wildcard arm
that gives an unreachable combination away.
No behavior changes. The lockfile, resolver, package-manager, reporter,
fs, store-dir, deps-restorer, CLI and pnpr suites pass, and the first two
compare emitted lockfiles byte for byte.
|
||
|
Zoltan Kochanandgithub-actions[bot]
|
91774e72a0 |
chore(release): 11.28.7, pacquet 12.15.0, pnpr 0.1.0-alpha.16 (#17187)
* chore(release): 11.28.7, pacquet 12.15.0, pnpr 0.1.0-alpha.16 * docs(release): curate 11.28.7, pacquet 12.15.0, and pnpr 0.1.0-alpha.16 release notes Add lead paragraphs and group the patch entries by what the reader was doing, with security fixes first on each page. - v11: lead with the registry metadata name check, pinned pnpm trustPolicy check, maxTarballUnpackedSize, and the removal of registry revisions; merge overlapping entries for minimumReleaseAge metadata revalidation, frozen-lockfile directory dependency checks, syncInjectedDepsAfterScripts, verifyDepsBeforeRun, pnpm sbom, and unreadable config settings. - v12: order minor highlights from config includes and lockfile.integrityTarget down to interactive update styling; drop the pnpr-only distribution note from the CLI changelog; merge overlapping entries for workspace and pnpr install speedups, minimumReleaseAge and If-Modified-Since revalidation, frozen-lockfile and injected package checks, verifyDepsBeforeRun, pnpm prune, pnpm pack, pnpm licenses list, pnpm self-update, pinned pnpm flag forwarding, and WebContainer fixes. - pnpr: lead the minor section with the breaking ecosystem URL prefix, hosted storage layout, artifacts.orgs, and strict config schema changes; fold dist.pnpr tarball metadata into one entry and drop the redundant /metrics note; lead the patch section with reserved-username and auth security fixes. - Use one issue-link label style across the release pages. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Zoltan Kochan
|
e2a11f5611 |
perf(install): precalculate package bins and shebang runtimes (#17186)
Precalculate package binary entrypoints and shebang runtimes during tar streaming in pnpr (dist.pnpr.bins) and during cold tarball extraction in pnpm so .bin command shims can be linked without reading package.json, walking directories.bin, or opening bin scripts on disk. - Add PackageBin, ScriptRuntime, parse_shebang_from_bytes, resolve_package_bins, and verify_package_bins in pnpm-package-manifest with strict path normalization and shell-safe runtime validation. - Extend inspect_tar in pnpm-tarball to compute bins and shebang runtimes in the same streaming pass as requiresBuild when requiresBuild is false. - Emit dist.pnpr.bins in pnpr packument responses and cache it in pnpr-storage TarMetadataCache. - Propagate cold-extracted manifests and precalculated bins through CreateVirtualStoreOutput into LinkVirtualStoreBins and direct-dep bin linking, and verify warm-cache dist.pnpr.bins against the store index manifest and CAS file list before skipping shebang reads. Closes pnpm/tasks#178 |
||
|
Alessio AttilioandZoltan Kochan
|
cd814501e6 |
perf: optimize pnpm install metadata through pnpr (#17169)
- **New Features** - Verified package metadata now includes accurate tarball file counts and unpacked sizes. - Registry resolution better supports abbreviated metadata for time-based resolution and trust checks. - Package metadata preserves publisher verification markers, bundled dependencies, and precise release timestamps. - **Bug Fixes** - Trust-policy checks more reliably detect downgrades and fetch complete metadata when required. - Offline and cached metadata verification is more resilient across supported registry types. - Tarball inspection consistently records build requirements, file counts, unpacked sizes, and integrity data. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
92b74670bc |
fix(pnpr): strip dist.pnpr overlay on packument rewrite (#17178)
GET /:pkg overlays dist.pnpr from the server's tar-integrity table while the stored packument omits it. When a client round-trips that packument back to PUT /:pkg/-rev/:rev (such as during partial pnpm unpublish), enforce_published_version_immutability previously treated dist.pnpr as an unexpected extra dist field and rejected the request with 400 Bad Request. Strip versions[*].dist.pnpr in enforce_published_version_immutability before validating and persisting the rewritten packument, matching how pnpr_storage::publish strips dist.pnpr on publish. |
||
|
Zoltan Kochan
|
1e2a9a0424 |
feat: surface tarball-verified requiresBuild from pnpr and use it during install (#17164)
Compute and persist per-tarball requiresBuild in pnpr when publishing or decoding upstream tarballs, project it onto served packuments and manifests under versions[v].dist.pnpr.requiresBuild and POST /-/pnpr/v0/resolve package frames, and consume it in pnpm v12 to: - overlap remote side-effects artifact resolution with cold tarball downloads when cold snapshots already have a verified requiresBuild, - skip unnecessary node --version probes and BuildModules passes when no package requires a build, and - skip redundant background extraction scheduling and CAS package.json reads during early materialization. Closes https://github.com/pnpm/tasks/issues/177. Written by an agent (Antigravity, Gemini 4 Argon). |
||
|
Zoltan Kochan
|
e01508f253 |
feat(pnpr): harden secrets, derive subkeys, and validate ecosystems (#17126)
Complete Section B deliverables (B4, B5, B6, B7) for pnpr production readiness.
- B5: Fail fast at startup when secret-bearing configuration fields (secret,
secrets.cache, secrets.oci, s3.secretAccessKey, auth.scim.token,
clientSecret, authToken, or upstream tokens) contain unresolved \${VAR}
references rather than falling back to empty strings or per-process keys.
Support independent configuration of secrets.cache and secrets.oci, and
derive domain-separated HMAC-SHA256 subkeys from root secret.
- B4: Enforce per-ecosystem validation rules rejecting \`teamsManagedBy: api\`
on non-npm registries and \`unpublish\` rules on Cargo and PyPI registries.
- B6: Set \`publish = false\` on \`pnpr/crates/pnpr/Cargo.toml\` to guard against
accidental crates.io publication.
- B7: Document the v1 extensibility model: compile-time trait implementations
(\`UserBackend\`, \`TokenBackend\`), no dynamic plugin runtime.
Relates to pnpm/tasks#131.
|
||
|
Zoltan Kochan
|
85cd31b9e5 |
feat(pnpr): settle Section A production readiness requirements (#17112)
Complete production-readiness requirements for pnpr Section A. - Persisted-document format versions: introduce formatVersion to packuments, OCI images, crate documents, and state journals, rejecting documents written by newer format versions. - CLI surface grammar: support `pnpr serve` subcommand, and add parse_cli_interval for duration units (s/m/h/d) on --packument-ttl, --min-age, and --max-age. - Strict config validation: reject unknown fields, disallow removed configuration blocks, and normalize field naming across config and CLI flags. - Client protocol versioning: add capability handshake at /-/pnpr, v0 resolve and publish endpoints, policy echo in done frames, and structured error codes. - Distribution improvements: add Node.js fallback wrapper, fix engines range, and package tini in pnpr Docker container. Relates to pnpm/tasks#131. |
||
|
Zoltan Kochan
|
ca18902a25 |
refactor(deps-resolver): one owner for peer provider decisions (#17114)
Part of pnpm/tasks#143. - Move the peer-shadowing rule and the provider lookup with lockfile-pin precedence into one owner: provider_for/ProviderPick in peer_provider.rs and providerFor in v11 hoistPeers.ts. - Read and write peer suffixes only through PeerBindings (pnpm-lockfile) and parsePeerBindings (@pnpm/deps.path), with a round-trip test over the repo's lockfiles. - v11 picks hoisted optional peers through v12's lockfile tiers. - Report a cached node's own missing peers in every importer, so a project no longer borrows an out-of-range peer from another project's subtree (v12, resolvePeersFromWorkspaceRoot: false). - Classify optional-peer edges by the bindings the key records, so a prod install and deploy keep a package's own dependency that is also its optional peer (v11 and v12). - Add proptest graph tests on StubResolver. Reusing a pin only when the owner returns it (step 2) is left out: deciding reuse after the peer walk cost 40-60% on lockfile-only installs. |
||
|
Zoltan Kochan
|
85f7fa1e98 |
feat(pnpr)!: give the hosted store a layout with one place per thing (#17100)
The hosted store shared one key space between packages, orgs and blobs: a registry without an org stored packages at the store root, an org's registry stored them in `<org>/`, a package's blobs sat beside its document, and a nested image repository sat inside its parent's directory. So an org collided with an unscoped package of the same name (an unpublish of that package removed the org's whole namespace), the implicit `<ecosystem>~<name>` org of a grouped registry was a valid npm name, and OCI GC and verify needed ad-hoc exclusion lists. On S3, object_store percent-encodes `~` in keys while the upload sweep compared raw prefixes, so grouped OCI registries were never swept. This is part of section A4 of pnpm/tasks#131. The new layout (pnpr_storage::layout), the same on disk and in a bucket: layout.json {"layout": 1} default/packages/<name>/document.json default/packages/<name>/blobs/<file> orgs/<org>/packages/... Every name, org and file name is one segment: `[a-z0-9._-]` as is, every other byte as `!` and two hex digits. That keeps nested names siblings, separates names that differ only in case on case-insensitive filesystems, and leaves nothing for object_store to escape, so a key is byte for byte the local path. Per-namespace records stay inside their namespace; records every namespace shares stay at the root. Listing is one directory read, so the `.package-index` workaround is gone, and the blob inventory reports package and blob names instead of paths. Startup refuses a store marked with another layout or an unmarked store holding data, and marks an empty one. The proxy cache uses the same package layout and is emptied once when its marker is missing. `pnpr upgrade-storage` converts an earlier store on disk or in a bucket, telling org directories from packages by the configured orgs; it moves blobs before documents so an interrupted run can resume, and refuses a directory that is both an org and a package. export, verify and migrate-storage check the layout, and migrate-storage marks its target. The registry fixtures the pnpm test suites serve are built in the new layout, and the cached build is keyed by the layout version. |
||
|
Zoltan Kochan
|
ab9e0b8d21 |
fix: bound publish-time tar hashing and fail closed on untied audit pins (#17096)
- pnpr: hash a published tarball's tar under the tar-decode permit, so concurrent publishes cannot take the whole blocking pool. - pnpr: cache upstream revision blobs under .revisions/tar-sha512. The old directory holds blobs a .tgz digest named, which the digest route must not serve as the tar that digest now names. - audit: fail a tar-pinned entry that has no tie instead of checking its lockfile integrity as the signed one. The publish permit is owned and moves into the blocking task, so a cancelled publish cannot release it while hashing continues, and only tarball attachments take one. Follow-up to pnpm/pnpm#17092. |
||
|
Zoltan Kochan
|
7b0d6873a6 |
feat: pin tar hashes with lockfile.integrityTarget and address revisions by tar digest (#17092)
lockfile.integrityTarget: tar makes the integrity of every package tarball entry the hash of its uncompressed tar, recorded as settings.integrityTarget. The lockfile then survives a registry recompressing its tarballs, switching to tar resources, or being replaced by a mirror that does either. A lockfile recorded under another target is stale: a frozen install fails with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, and any other install re-resolves with update-checksums semantics so no hash of the old kind carries forward. The setting is opt-in and the lockfile version is unchanged. The tarball crate now reports both hashes of every extraction: the resource after content decoding, and the tar a .tgz decodes to. A pin names one of them (IntegrityKind::Resource / Tar), or either at resolution, where a locked or published integrity may be of either kind (ResourceOrTar, sha512 only). Under a tar pin every gzip member is decoded and nothing may follow the tar. The resource pin keeps today's behaviour, extracting the first member and tolerating trailing data, but then records no tar. Store rows extracted from a .tgz record tarIntegrity and get an alias under the tar key, in a table pnpm v11 never reads. Store lookups follow aliases, so a tar-pinned frozen install is served offline by a store that .tgz installs populated, and resolution takes a version's tar hash from its row before downloading anything. Registry revisions now always pin their tar, revision 0 included, and are fetched with a tar pin, so the digest route can use any content coding. Revision metadata must list revision 0 and the selected revision, and a digest-route URL without a revision is rejected. pnpr keys its digest references by tar hash on every registry format, serves the route as a negotiated tar resource, and projects revision 0 explicitly. pnpm v11 cannot extract tar resources, so it now rejects revisions with ERR_PNPM_REGISTRY_REVISIONS_UNSUPPORTED. Implements pnpm/rfcs text/0000-tar-integrity-for-every-source.md and pnpm/rfcs#42. |
||
|
Zoltan Kochan
|
430d892741 |
feat(pnpr)!: serve every ecosystem under its own prefix (#17087)
pnpr served npm at the server root when npm was the only configured ecosystem and moved it under /npm/ as soon as a second ecosystem was added. Adding an ecosystem therefore broke every .npmrc registry URL, every tarball URL already written into a lockfile, named-registry URLs, and OIDC workload publish paths, and /npm meant the package `npm` on one server and the ecosystem prefix on another. This settles section A1 of pnpm/tasks#131: every ecosystem is always prefixed. - Ecosystem::base_path is now independent of the configuration. The per-configuration Registries::base_path and is_only_ecosystem are gone, and the routes, the resolver's tarball URLs, OIDC workload paths and pnpr's own-origin route classification all use it. - The default image registry still answers at /v2/ on the host root; an image client cannot be pointed at a prefix. The registry directory keeps its `prefixed` and `namedPrefixed` flags for the web UI. - The read-only token allowlist matched the npm audit endpoints at the root only, so read-only tokens were already refused for audits on multi-ecosystem servers. It now matches them under the npm prefix. - /metrics no longer switches between Prometheus metrics and the npm package `metrics` on the Accept header; the package is /npm/metrics. - The pnpm v11 and v12 test harnesses, the registry-mock crate and the integrated benchmark address the fixture registry at /npm/ and keep the server origin for the pnpr protocol endpoints. The benchmark asks each revision mock's registry directory whether npm is prefixed, so it can still compare against a pnpr revision from before this change. |
||
|
|
573c4d1e8e |
fix(pnpr): gate server-only error variants behind server feature (#16972)
pnpr-error depended on axum, bcrypt, object_store, reqwest, rusqlite, serde_json, tokio and tracing on every non-wasm target. pnpm-config depends on pnpr-registry for Ecosystem and PackagePattern, and pnpr-registry and pnpr-package-name both depend on pnpr-error, so every consumer of pnpm-config compiled and linked the registry server's HTTP, storage and auth crates. - pnpr-error: those dependencies are optional and enabled by a new `server` feature. The response module (IntoResponse, public_message, log_kind) and the Upstream, Bcrypt, Sqlite, JoinError, ObjectStore and Json variants are gated on it, replacing the cfg(not(target_family = "wasm")) gate. - The pnpr crates that use those variants or the response rendering (auth, osv, pipeline-runs, pnpr, search, shared-artifacts, storage, upstream) enable `server`. pnpr-config only builds InvalidConfig and does not. - pnpr-registry drops its unused pnpr-error dependency. - pnpr-package-name tests compare Display output instead of public_message(), so they pass without the server feature. - Three workflow_run workflows carry their zizmor ignore directive on the trigger line. zizmor did not apply the standalone comment, which failed the zizmor job on fork PRs. Closes pnpm/tasks#134. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
e3c80c6ed0 |
fix(pnpr): reserve SCIM-provisioned usernames (#17080)
A user a SCIM client provisioned has no password account until they sign in, so anyone could take the name first through npm adduser and keep it. Registration now treats every username in the SCIM directory as reserved, like admin and OIDC-bound names. Provisioned users sign in through OIDC or get their account from an admin. A password account that exists when its user is provisioned keeps working. A login that creates an account checks the name again before issuing a token, so a provisioning that lands mid-login takes the account back, as does a directory that can no longer be read. Raised in review of pnpm/pnpm#17046. |
||
|
Zoltan KochanandClaude Opus 5.5
|
0106373cb4 |
docs(pnpr): drop a stale doc comment from compile_rule_access_list (#17082)
The block described build_rules and a duplicate-key check that build_rules does not perform. RuleTable already documents the specificity and duplicate-key contract. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
dfb72de39d |
perf(pnpr): serve name@version tarballs without the packument (#17074)
A `.tgz` registry now serves a hosted version's blob, or an upstream version's cached `.tgz`, under the canonical `<basename>-<version>.tgz` the way the filename route does, so a warm hit loads no packument. A cold upstream request still reads the requested version's dist, because the filename route binds a filename to whichever version's `dist.tarball` names it. Without a fresh cached packument, the route reads that dist from the upstream's `<name>/<version>` manifest, a few KB instead of the whole packument. A manifest of another version, as an upstream returns for a dist-tag, or one without a valid `dist.integrity` falls back to the packument, as does an upstream that serves no manifest. An unreachable upstream gets the stale cached packument at once. A version that is not a safe path segment is not found, since the canonical filename is built from it. A tar registry's response needs the version's `.tgz` integrity. A dist read from an upstream is kept in a per-replica cache. One read from a cached packument is tagged with that packument's stamp (mtime and size) and answers only while the packument is fresh and unchanged, so a refetched packument retires its dists at once. One read from a version manifest answers for the packument TTL. Authorization runs before every lookup. Hosted dists and `cache: false` upstreams are not cached. A large packument took 3-8 ms to read and parse per request (react, `@types/node`, typescript). Warm requests for those versions now take 0.06-0.8 ms. Closes pnpm/tasks#151 |
||
|
Zoltan Kochan
|
2d94114ffe |
feat(pnpr): implement package ownership, maintainer model, and tombstones (#17060)
- Implement GET /-/user/org.couchdb.user:{name} returning CouchDB user document
- Record initial publisher in packument maintainers list and retain on subsequent publishes
- Support $owners token in package publish and unpublish access policies
- Support npm owner mutations via packument rev updates
- Store package and version tombstones under .tombstones/v0 to prevent squatting and version reuse
Part of https://github.com/pnpm/tasks/issues/131
|
||
|
Zoltan Kochan
|
dcfdbc547d |
fix(pnpr): close gaps left in reserved-username registration (#17073)
Close gaps left in reserved-username registration, from the review of pnpm/pnpm#17046: - A login that saw a reserved account, and then created it because an admin removed the account in between, now takes back the account and its token and answers 403. Previously the caller's password stayed stored under the admin's name. The check runs before a token is issued. The account is deleted only while it still accepts the login's password, so an admin's concurrent rewrite stays, and the name's tokens are revoked afterwards. - `pnpr add-user` revokes tokens left under the username before it creates the account, as the admin API already did. The create-or-update step and token revocation move to `AuthState` so both share them. - Document `pnpr add-user`, including that it must run while pnpr is stopped when users live in an htpasswd file, and stop telling admins to self-register the first admin account. |
||
|
Zoltan Kochan
|
6737c4a525 |
feat(pnpr): prevent self-registration of reserved usernames (#17046)
Prevent self-registration from squatting configured admin usernames, OIDC login, and workload bindings (Section B1 of https://github.com/pnpm/tasks/issues/131): - Reject self-registration attempts for reserved usernames with 403 Forbidden and a descriptive error message. - Add `is_reserved_name` check on `IdentityConfig` covering `auth.admins`, OIDC user logins, and OIDC workload bindings. - Add `pnpr add-user <username>` CLI command for out-of-band admin bootstrapping and password updates. - Preserve normal CouchDB login for existing accounts matching reserved names when provided with the correct password. Part of https://github.com/pnpm/tasks/issues/131 |
||
|
Zoltan Kochan
|
8b69106ef5 |
feat: install from tar registries with zstd content coding (#17054)
Implements pnpm/rfcs#39 in pnpm v12 and pnpr. pnpm: - `serverType: pnpr` rebuilds tarball URLs as `<registry>/<name>@<version>`. - Tarball requests go through HTTP clients that leave Content-Encoding in place. pnpm decodes zstd (8 MiB window, RFC 9659) and gzip itself, because reqwest's built-in decoder allows a 128 MiB window and cannot be configured. - The decoded body is sniffed: the gzip magic means a .tgz, anything else a tar. The hash covers the decoded resource on both paths. - A decoded-output budget bounds what a response makes pnpm decode before its hash is checked: file payload up to `maxTarballUnpackedSize` (4 GiB), tightened by `dist.unpackedSize` for tars only; 256 MiB of non-payload; at most 10 KiB of zeros after the end-of-archive marker; empty bodies fail. - A missing `dist.tarball` is rebuilt from the declared layout, and fails with ERR_PNPM_MISSING_TARBALL_URL without one. A missing `dist.integrity` is learned by the existing resolve-time archive read. - `--update-checksums` re-resolves every locked package. pnpr: - `tarballs: tar` makes a registry's metadata describe tars and its `<name>@<version>` route serve them. Every registry serves that route. - Tar hashes are derived at publish or on the first decode and stored in a reserved `.tar-integrity` namespace of the shared hosted backend. - A cached upstream .tgz is re-checked as its tar streams; a mismatch fails the response and evicts the copy. - gzip clients get the stored .tgz. zstd clients get a level-19 variant from the local cache once a bounded worker made one, and a level-3 stream until then. Decoding stops at 4.25 GiB, and copies kept for the worker are coalesced per version and drawn from a 1 GiB quota. |
||
|
Zoltan Kochan
|
e7cfcbb252 |
test: share the three-install request-log cases between pnpm v11 and v12 (#17063)
`pnpr/.fixtures/request-logs/metadata-three-installs.json` lists the three-install request-log cases: settings, arguments, and the metadata requests of each install. The v12 CLI suite and a new v11 e2e test both assert it, so a difference in what the two stacks request fails one of them. The v11 test records requests through a proxy in front of the mock registry. Tarballs bypass that proxy, so the table leaves them out and the v12 test asserts them on its own. The table found one difference: v12's `to_http_date` read only RFC 3339 timestamps and HTTP dates, so a packument whose `modified` is a bare date revalidated with `If-None-Match` alone. pnpm v11 converts the same value with `new Date(...).toUTCString()` and sends both validators. v12 now reads a bare date as midnight UTC too. `httpdate::fmt_http_date` panics outside the years 1970 to 9999, and a registry's `modified` reached it unchecked. `to_http_date` now drops such a value. Part of pnpm/tasks#146 and pnpm/tasks#141. |
||
|
Zoltan Kochan
|
0ee3439a0c |
feat(pnpr-auth): add database schema migrations, token scopes and kind, and mysql case sensitivity (#17039)
* feat(pnpr-auth): add database schema migrations, token scopes and kind, and mysql case sensitivity Add schema migration tracking and token schema extensions to the auth store (Section A5 of https://github.com/pnpm/tasks/issues/131): - Implement schema version tracking with auth_schema_migrations table across SQLite, LibSQL, PostgreSQL, and MySQL backends. - Refuse startup with an actionable error when the database schema version is newer than the running server's supported version. - Add nullable expires_at, name, scopes, and kind columns to tokens table across all SQL backends. - Extend TokenRecord and NewToken with scopes and kind fields, marking both types non_exhaustive with constructor and builder methods. - Mark Identity in pnpr-policy and UpsertOutcome in pnpr-auth as non_exhaustive. - Set explicit binary collation (utf8mb4_bin) on MySQL username columns in users and tokens tables to enforce case-sensitive matching. - Expose scopes and kind in token creation and listing endpoints. Part of https://github.com/pnpm/tasks/issues/131 * fix(pnpr): validate token kind length and make migration version writes concurrent-safe - Use BIGINT for migration version column in SCHEMA_MIGRATIONS_TABLE_SQL to match i64 decoding on PostgreSQL. - Use idempotent version inserts (ON CONFLICT DO NOTHING / INSERT IGNORE) across all backends to handle concurrent startup migrations safely. - Validate that token kind does not exceed 64 characters in create_token endpoint and return BadRequest. |
||
|
Zoltan Kochan
|
198cc7f398 |
feat(pnpr): provide minimal production default configuration and separate test fixtures (#17027)
* feat(pnpr): provide minimal production default configuration and separate test fixtures Replace the bundled test configuration in `pnpr/crates/config/config.yaml` with a minimal production-safe default configuration (Section A3 of https://github.com/pnpm/tasks/issues/131): - Configure public npmjs upstream proxy and hosted local registry at `/~local/` with default registry set to npmjs. - Generate a secure 32-byte CSPRNG random secret by default when secret is omitted, and disable registration (`max_users: -1`). - Clean up unused Verdaccio legacy keys (`plugins`, `middlewares`, `web`) and remove test fixtures from production default configuration. - Move test mock configuration and fixture scopes to `pnpr/.fixtures/config.yaml`. - Move test presets (`REGISTRY_MOCK_LOCAL_PATTERNS`, `registry_mock_rules`, `registry_mock_graph`, and `MockConfigPresets` trait with `Config::proxy` and `Config::static_serve`) from `pnpr-config` into `pnpr-fixtures`. - Update Jest global setup and `pnpm-registry-mock` to load test configuration from `pnpr/.fixtures/config.yaml`. - Update documentation in `pnpr/docs/configuration.md` and `pnpr/docs/quick-start.md`. Related to https://github.com/pnpm/tasks/issues/131 * fix: drop obsolete quick-xml advisories from deny.toml quick-xml was upgraded to 0.41.0, resolving RUSTSEC-2026-0194 and RUSTSEC-2026-0195, so cargo-deny no longer encounters them. * fix: ignore unreachable zip advisories in deny.toml * fix(pnpr-config): require authentication to access private packages on local registry |
||
|
Zoltan Kochan
|
157994e9f4 |
docs(pnpr): document reverse-proxy and TLS setup, webhooks, lifecycle, and pnprServer (#17013)
* docs(pnpr): document reverse-proxy and TLS setup, webhooks, lifecycle, and pnprServer Document production deployment and operational architecture for pnpr and pnpm (Section C13 and Section F of https://github.com/pnpm/tasks/issues/131): - Add Reverse proxy and TLS deployment guide (pnpr/docs/tls-and-reverse-proxy.md) with architecture rationale and configuration recipes for Nginx, Caddy, Traefik, and AWS ALB. - Add Webhooks and event notifications guide (pnpr/docs/webhooks.md) documenting the decoupled event architecture, structured audit logs (pnpr::audit), and object storage bucket notifications. - Document signals, readiness probe state transitions, process lifecycle, and standard exit codes in pnpr/docs/cli.md. - Document S3 commit journal lifecycle, multi-replica considerations, and disaster recovery caveats in pnpr/docs/storage.md. - Document the pnprServer workspace setting reference in pnpm/docs/settings/dependency-resolution.md and pnpm11/docs/settings/dependency-resolution.md. - Update sidebars and quick start navigation. Related to https://github.com/pnpm/tasks/issues/131 * docs(pnpr): clarify pnprServer offloads resolution and streams package metadata Clarify that pnprServer offloads dependency resolution rather than file fetching: the server streams back resolved package metadata frames as it walks the graph so the client can prefetch tarballs from registries in parallel. * docs(pnpr): address review feedback on Vector remap and pnprServer tarball fetch boundary - Add remap transform in Vector example in pnpr/docs/webhooks.md to parse raw JSON from Docker container logs before filtering audit events. - Clarify direct public tarball fetching and server-side private upstream credential boundaries in pnpm/docs/settings/dependency-resolution.md and pnpm11/docs/settings/dependency-resolution.md. * docs(pnpr): correct probe endpoints, shutdown lifecycle, and log shippers * docs(pnpr): correct S3 commit journal staging paths and write-ahead sequence * docs(pnpr): clarify proxy hop coverage and orphan pruning scope * docs(pnpr): distinguish X-Real-IP from hop traversal and document ALB target port restriction |
||
|
Zoltan Kochan
|
bc35481b80 |
feat(pnpr): add maintenance tooling and verdaccio htpasswd support (#17002)
Add maintenance subcommands and migration tooling for pnpr (Section C12 of https://github.com/pnpm/tasks/issues/131): - Implement `pnpr config check` (`pnpr config-check`) to validate configuration and report diagnostic details on routes, upstreams, auth, storage, and surfaces. - Implement `pnpr verify` to audit hosted package documents against storage blobs, checking for missing files, corrupted checksums, and unreferenced orphans. - Implement `pnpr export --out <dir>` to dump packages, documents, and blobs, with optional `--include-auth` support. - Implement `pnpr migrate-storage` to migrate hosted packages and blobs from local filesystem storage to an S3-compatible object store. - Support Apache/Verdaccio `{SHA}` format in `pnpr-auth`, automatically upgrading authenticated users to bcrypt upon first login, and provide `pnpr import-htpasswd`. - Add backup and disaster recovery documentation in `pnpr/docs/backup-and-restore.md` and update CLI documentation. Related to https://github.com/pnpm/tasks/issues/131 |
||
|
Zoltan Kochan
|
7d4b626898 |
feat(pnpr): support login web flow for SSO (#16969)
Support npm login web flow (`/-/v1/login`) so `npm login` and `pnpm login`
authenticate via browser SSO when an OIDC login provider is configured.
- Implement `POST /-/v1/login` and scoped `POST /~{registry}/-/v1/login`:
return 404 when no OIDC login providers are configured to allow fallback
to CouchDB auth; return 202 Accepted with `loginUrl` and `doneUrl` when enabled.
- Implement `GET /-/v1/login/start`: validate session and redirect to single
provider login or render an HTML picker for multiple providers.
- Implement `GET /-/v1/login/poll/{session}`: return 202 with `Retry-After: 1`
while pending, and 200 with `{ "token": "<token>" }` once authenticated, consuming session.
- Update OIDC login flow to support `return=cli&session={session}` and fulfill
CLI web login session upon completion.
- Export `bundled_root_certs` in `pnpm-network` and configure OIDC reqwest client
with root certificates.
Related to https://github.com/pnpm/tasks/issues/131
|
||
|
Zoltan Kochan
|
a1ce6a1bc0 |
feat: split workspace settings across included files (#16967)
Implement the config fragments RFC in pnpm v12. The root pnpm-workspace.yaml can list local YAML files and config dependencies under `include`, each mapped to the settings it owns (or '*' for local files). Every setting and catalog slot (catalog, catalogs.<name>, catalogs) has exactly one owner; a listed setting is reserved even when the file omits it, and legacy aliases share their canonical setting's owner. pnpm-workspace loads the include index and merges the fragments into one document that every reader consumes: config loading, workspace package discovery, catalogs, changed-catalog detection in the projects filter, and the napi config cache. Without `include` the root text is read as before. Fragments are normalized (merge keys applied, tags dropped, scalar keys stringified) before ownership is checked, so a fragment cannot define a catalog it was not delegated through a tag or a `<<` merge. A relative path in a local fragment is relative to the directory of the fragment's include path, the way a path in the root manifest is relative to the workspace root. The loader re-expresses each fragment's relative paths from the workspace root before merging: path settings, `packages` globs, `pnpmfile`, `patchedDependencies`, a path-like `scriptShell`, and local specifiers in catalogs and `overrides` (re-anchored with the same LocalSpec logic catalogs use). Absolute, `~` and `$VAR` paths are kept. This departs from the RFC, which resolved fragment paths from the root, so a fragment can sit next to the files it names. The RFC change is pnpm/rfcs#40. workspace-manifest-writer routes every write to the owning file. Writes to settings owned by a config dependency fail, `include` itself is never written, included files are never deleted, and symlinked includes are written through. Paths it writes to a fragment, such as patch-commit entries, link: overrides and local catalog entries, are written relative to that fragment. Every file's edit is computed before any file is written, so a rejected edit leaves the workspace unchanged, and a failed write restores the files written before it. A config dependency names its settings file in package.json `pnpm.config`. The entry must stay inside the installed package and may only define delegated settings. Its values are applied at the project layer after config dependencies are installed and before hooks run, so they take effect on the first install too. Installs fail closed when an included package's entry cannot be loaded. `bootConfig` holds the policy (minimumReleaseAge*, trustPolicy*) used to resolve config dependencies and the packageManager pnpm version. Package-owned keys fall back to the values from below the project layer, env and CLI overrides still win, and self-update ignores it. |
||
|
KhảiandClaude
|
33f531eeac |
refactor(pnpr): split oidc.rs into the groups its items already form (#16970)
`oidc.rs` sat at 388 of the 400 code lines `perfectionist::overly_long_file` allows, so anything added to it has to displace something else first. Four groups of its items move to submodules of `oidc/`: - `provider_metadata`: the metadata cache, its two durations, the discovery that fills it and the helper that reads it. - `pending_login`: the payload sealed into the browser's cookie, the signing that seals and opens it, and the replay guards a callback has to pass. - `login_client`: the authorization-code client type, and its construction from the client-secret methods a provider advertises. - `transport`: the outbound HTTP the `openidconnect` crate drives through this state, and the `AsyncHttpClient` impl that reaches it. What stays is the module's shared vocabulary and the login flow it exists for: `OidcState`, `Provider`, `SessionUser`, `LoginStart`, `new`, `start`, `finish`, `verified_expiration`, and the items six siblings already import through `super::` (`MAX_ENTRIES`, `SESSION_PREFIX`, `random_secret`, `invalid_config`, `rejected`, `unavailable`). The file goes from 431 lines to 238. The items move verbatim. The only edits are visibility, raised to the `pub(super)` each one held implicitly while it sat in the parent module, and two import adjustments: the signing traits leave with `seal_login` and `open_login`, and `sha256_hex` is reached as `super::super::` from one level deeper, the spelling `handoffs.rs` and `sessions.rs` already use. Checked rather than assumed: normalizing both sides and comparing line multisets reports no logic line added, removed or altered. Every difference is an import line, an `impl OidcState` opener, or rustfmt reflowing the one signature that gained `pub(super)`. Claude-Session: https://claude.ai/code/session_01QYTBQo8HY7xD2iz6hjo1CW Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
27004ec0d2 |
feat(pnpr): add packument caching headers and conditional request support (#16962)
Add ETag generation, Vary: Accept, and Cache-Control headers to packument
responses, and return 304 Not Modified when an incoming If-None-Match or
If-Modified-Since header matches the current packument state.
- Compute strong ETag over rendered packument response body
- Return 304 Not Modified with empty body preserving ETag, Vary, and Cache-Control
- Support If-None-Match (with weak comparison per RFC 9110) taking precedence over If-Modified-Since
- Emit Vary: Accept on packument responses, preserving Accept and appending Authorization when private_no_cache applies
- Set Cache-Control: public, max-age={ttl} on public packuments from upstream maxage or config packument_ttl
- Maintain Cache-Control: private, no-store on private or caller-scoped responses
- Extract OSV filtering helpers into osv_filtering.rs and server lifecycle into lifecycle.rs to keep source files within line limits
Related to https://github.com/pnpm/tasks/issues/131
|
||
|
Zoltan Kochan
|
1d909993c4 |
feat(pnpr): admin audit log for user, rule, team, scim, and unpublish operations (#16960)
Emit structured audit log events under target `pnpr::audit` at info level for administrative and destructive actions: - User management (create, password update, delete, token revoke, register) - Rule override updates and deletions - Team creation, destruction, member add/remove, and package access grants - SCIM user provisioning, replacement, patching, and deprovisioning - Package and tarball unpublishing, dist-tag set/remove, and OCI deletions All audit events inherit active request span context (request_id, client_ip, method, uri) through tracing spans. Ref: https://github.com/pnpm/tasks/issues/131 |
||
|
Ben MaurerandZoltan Kochan
|
b260a317ec |
feat: accept zstd-compressed responses and serve them from pnpr (#16585)
Enable reqwest's `zstd` feature and set `.zstd(true)` next to `.gzip(true)` in the shared install client. pnpm now sends `Accept-Encoding: zstd,gzip,deflate` (previously `gzip,deflate`) and decodes zstd bodies transparently. Full packuments gain the most. A long field repeated across versions, such as vite's `licenseText` (up to 259 KB per version, 34 MB of the document), spans far more than gzip's 32 KB window, so gzip cannot reuse the earlier copies. vite's full packument on registry.npmjs.org is 38.9 MB: gzip -6 gives 4.65 MB, zstd -3 603 KB, zstd -19 455 KB. pnpm fetches full packuments for `trustPolicy: no-downgrade`, for `resolutionMode: time-based` on registries without a `time` field, and for `minimumReleaseAge` when the abbreviated document has no `time`. Abbreviated packuments gain less (vite: 403 KB gzip, 382 KB zstd -3). Brotli is left out on purpose. On these payloads it saves about as much as zstd and would add a second decoder to the binary. Tarballs are unaffected: registries serve them without `Content-Encoding`, so integrity verification still hashes the raw `.tgz`. Because `reqwest` is a workspace dependency, pnpr's upstream client gains zstd too. The WebContainer transport gets a no-op `zstd()` builder method to match its `gzip()`. pnpr also serves zstd: tower-http's `compression-zstd` lets its CompressionLayer pick zstd when the client prefers it. Resolving 1,235 packuments against pnpr with full metadata moved 103.6 MB instead of 115.4 MB (-10%), and took 34.5 s instead of 38.4 s on a 25 Mbit/s link. `zstd-sys` compiles the bundled libzstd with `cc`, which the build already needs on every release target. The release binary grows by 152 KiB (+0.3%). --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
24563c049e |
fix(pnpr): reject duplicate publish attachments and optimize orphan check (#16958)
In pnpr's publish attachment processing (`prepare_attachments`), two improvements address post-merge review feedback on pnpm/pnpm#16956: 1. Reject publish requests that contain multiple attachments resolving to the same canonical filename (e.g. submitting both `.sigstore` and `.sigstore.json` aliases for the same version). Previously, duplicate attachments resulted in conflicting blob staging in the journal commit, preventing proper recovery on server restart. These now return 400 Bad Request. 2. Eliminate the quadratic scan during Sigstore orphan attachment validation. Tarball versions are collected into a HashSet on the first pass, enabling O(1) existence checks for each Sigstore attachment. Related to pnpm/tasks#131 and pnpm/pnpm#16956. |
||
|
Zoltan Kochan
|
42b4aa1b79 |
feat(pnpr): support npm publish --provenance and attestation endpoints (#16956)
Support publishing provenance attestations (.sigstore/.sigstore.json) alongside package tarballs and serving them via /-/npm/v1/attestations. Rewrite dist.attestations.url in packuments to route through the pnpr registry instance, and implement /-/npm/v1/keys proxying. Clean up sigstore blobs on tarball unpublish. Ref: https://github.com/pnpm/tasks/issues/131 |
||
|
Zoltan Kochan
|
233af19afa |
feat(pnpr): support token creation, expiry, and read-only audit access (#16954)
Support creating tokens via POST /-/npm/v1/tokens and under registry prefixes with read-only restrictions, CIDR whitelisting, expiry, and token names. Enforce token expiry across all authenticated requests, and update the last_used_at timestamp on token authentication. Allow read-only tokens to query npm audit endpoints while preventing token creation and mutating writes. Ref: https://github.com/pnpm/tasks/issues/131 |
||
|
KhảiandClaude Opus 5
|
ecd3d289f4 |
refactor: remove even more map_or and map_or_else (#16702)
* refactor(napi): match on the throttle option instead of a branching default `map_or` evaluates its default argument whether or not the option holds a value, so both `Duration` constructors ran on every renderer build even when `throttleProgress` was set. A `match` builds only the arm it takes, the nested `if`/`else` flattens into a guard arm, and the chosen value now reads ahead of its fallback. Left alone in this file: the `map_or_else` in `renderer_width`. It belongs to the default-first `map_or_else` sweep, not to this eager-default shape, so it is converted in that commit instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor(cli): match on the on-fail config instead of an eager default `unwrap_or` evaluates its argument unconditionally, so the inner match over `packageManager.onFail` ran even when `pmOnFail` was configured and decided the result. Matching on both values at once keeps the configured setting first and reaches the manifest only when there is no setting. The inner match also had two arms with the same body, `Some("download") | None` and `Some(_)`, which collapse into the single catch-all the enum's default already needs. `PmOnFail` is `Copy`, so the configured value is returned by value rather than cloned. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor(cli): match instead of a branch inside an or-else closure Two `unwrap_or_else` calls wrapped a whole branch in their closure, so the fallback came before the value it falls back to and the branch sat one level down inside an argument. As sibling match arms the value leads and the branch becomes a guard or a second arm. In the audit renderer this also drops the clone of the `Option` wrapper: only the `String` that is actually returned is cloned now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor(pnpr): flatten the nested tarball-filename fallbacks `tarball_filename` nested one `map_or_else` inside another's closure, so both levels put their failure arm ahead of their success arm and the reader met two fallbacks before either value. An early return for the basename, then a `match` on the parsed name, reads top to bottom. The early return keeps the laziness the nesting provided: the package name is still parsed only when the URL carries no basename. Folding both into one match over a tuple would parse it every time, so it is deliberately not done that way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor: match on the option instead of a default-first map_or_else Across the Rust crates, 41 `map_or_else` calls were written with the default closure on its own line, so the fallback appeared before the value it stands in for and both arms were closures rather than arms. A `match` puts the value first, drops the closures, and lets the fallback read as the sibling it is. A few sites needed more than the mechanical swap: - Where the scrutinee was itself a chain, the chain is bound to a local first so the match reads against a name. - `renderer_width`, `lockfile_path`, and the reporter-flag scan carried an `if`/`else` inside the default closure; those branches are now guard arms. - `project_anchor_importers` and `update_actions_root` nested one option test inside another, so they match over a tuple of both. - `selected_prune_importer_ids` took its outer option with `?` instead of wrapping the body in a `map` closure. - The `json!` argument in `read_archive_by_custom_fetcher` is bound to a local, because a match block inside the macro's value position would be read as an object. Laziness is unchanged everywhere: no arm's work is done unless that arm is taken. Not converted: single-line `map_or` and `map_or_else` calls that already read cleanly, the `unwrap_or` and `unwrap_or_default` calls whose default is a literal or `Default::default()`, and `is_some_and`, which is the idiomatic spelling of that test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor: drop the combinator sweep from the files it should not have reached The default-first `map_or_else` sites were selected by a regex matching on where rustfmt put a line break. Those calls nest nothing, evaluate nothing early, and invert no logic, so the selector measured a formatting habit rather than finding a defect. Its output was a survey, not a work list. Four conditions disqualify such a site, each of which the sweep had already failed somewhere: - The rewrite has to invent a name whose only job is to be matched on one line later. That moves a level out of the expression and into the enclosing block instead of removing it. - The first closure diverges, so there is no default value to put second. `unwrap_or_else` and `expect` already read as "this must be Some". - The call sits in a macro's value position, where a match block does not parse as a value, so the field ends up lines away from what fills it. - It is test code, read only when it fails, where churn costs review and blame and returns nothing. Twelve files go back to their base state on those grounds, including both `tests/suite` helpers, which were also the two diverging sites, and four of the seven invented bindings. No file under `tests/` is modified by this branch any more. The conversions that remain were each reviewed on their own merits and kept at the reviewer's request for reading better than what they replaced, independently of the sweep that found them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor(cli): add to the index inside the reporter-flag match arms Review preference: the match reads as a statement whose arms each advance the cursor, rather than as an expression whose value is added to it. The arms evaluate to `()`, so the match is a statement and takes no trailing semicolon; `clippy::unnecessary_semicolon` rejects one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * chore: allow clippy::map_unwrap_or The lint rewrites `.map(f).unwrap_or_else(g)` into `map_or_else(g, f)`, which puts the fallback ahead of the value it falls back from. Reading such a call means holding the default in mind while looking past it for what the expression actually produces; the chain states the same thing in the order it evaluates. This sits with the existing `option_if_let_else` opt-out, which the workspace already allows because that lint's suggested `map_or_else` closures read worse than the `if let` they replace. Both lints push toward the same fallback-first spelling, so allowing both leaves the choice to the author rather than to whichever fires first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor: chain map and unwrap_or_else over a fallback-first map_or_else Seven sites where the value and its fallback were both arguments to `map_or_else`, the fallback first. Each becomes a `map` carrying the value and an `unwrap_or_else` carrying the default, so the call reads in the order it evaluates. Laziness is unchanged: `unwrap_or_else` takes a closure, as `map_or_else` did. Two of them read better as several maps than as one closure doing the whole conversion: - `cli_args/cache.rs` walks a path component to an owned string point-free, through `Component::as_os_str`, `OsStr::to_string_lossy` and `Cow::into_owned`. - `cli_args/pipeline/cache.rs` takes the captured stdout and then decodes it, rather than borrowing out of a closure's own argument. The three diverging fallbacks keep their `panic!` in `unwrap_or_else`, where it reads as the end of the chain rather than as the first thing the call does. `cargo_manifest.rs` keeps the comment offset as the mapped value and the trimmed line length as the fallback. The reverse pairing does not typecheck: the `Some` value is a `usize` offset, which has no `trim_end_matches`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor(cli): match on the comment offset in the dependency value range Review preference. The offset and the trimmed line length are two cases of where the value ends, so they read as arms of one match rather than as a value mapped out of the option with its fallback chained behind. `option_if_let_else` is allowed in this workspace, so nothing pushes this back toward the combinator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor: chain or match the map_or and map_or_else that main added The 128 commits merged in brought 38 new fallback-first calls, mostly from the new profile, token, SCIM and rust-toolchain work. Each becomes a `match` or a `map`/`unwrap_or` chain, as the style guide now asks. Every such call in the 33 files involved converts, including the handful that predate those commits, so no file is left half in one style. Three resisted the mechanical swap: - `ignore_walk/rule.rs` fell back to `""` through an identity closure. `.map(|segment| segment)` would trip `clippy::map_identity`, so the option is unwrapped with `.copied()` instead. - `store_index/files.rs` turned an optional error into a result through `map_or(Ok(()), Err)`. The chain for that reads as noise, so it is a match over the two cases. - `artifact_store/turborepo.rs` defaults to an empty `String`, which `clippy::unwrap_or_default` requires be spelled `unwrap_or_default`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * docs(style): record the preference against map_or and map_or_else The style guide exists for what clippy cannot enforce, and this convention has nothing enforcing it: `clippy::map_unwrap_or` rewrites the chain back into `map_or_else` and `clippy::option_if_let_else` rewrites the match into it, so both are allowed in the workspace. Without the rule written down the next contributor reintroduces the combinator and nothing explains the two opt-outs. Also names the shorter spellings that beat both forms, so the rule does not read as a blanket preference for the longer one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n * refactor: chain or match nine more map_or and map_or_else calls The 52 commits merged in added these, mostly in the new pnpr audit, health probe and access logging work and in the Python resolver. Each becomes a `match` or a `map`/`unwrap_or` chain, as the style guide asks. Two are not plain chains: - `cli/src/lib.rs` turned an optional write error into a result. A chain for that reads as noise, so it is a match over the two cases, as `store_index/files.rs` already is. - `python-resolver/candidates.rs` defaults to an empty `BTreeMap`, which `clippy::unwrap_or_default` requires be spelled `unwrap_or_default`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNxMkLRG8CRDPwe5vcd28n --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
835363841c |
feat(pnpr): support npm audit endpoints and osv bulk advisories (#16950)
Add support for POST /-/npm/v1/security/advisories/bulk, POST /-/npm/v1/security/audits, and POST /-/npm/v1/security/audits/quick. When an OSV database is configured, evaluate bulk advisory queries locally. Otherwise, proxy requests to upstream registries when configured. Ref: https://github.com/pnpm/tasks/issues/131 |
||
|
Zoltan Kochan
|
8e057376d6 |
feat(pnpr): cache eviction, lru size cap, and orphan namespace pruning (#16946)
* feat(pnpr): implement cache eviction, lru size cap, and orphan namespace pruning Add configurable cache budget and TTL retention policies for the upstream proxy cache, automated orphan namespace pruning on startup, periodic runtime cache eviction, and a new `pnpr cache-gc` CLI command. Closes pnpm/tasks#131 (Section C5). * test(registry-mock): poll for the live registry child during pending startup Poll for the child process until it appears with its command line or the timeout elapses, avoiding a race when sysinfo samples the process table before the spawned child has completed execve. * test(registry-mock): record spawned child PID via oneshot channel * test(pnpr): set accessed time alongside modified time in cache eviction tests * fix(pnpr): address PR review findings on cache eviction - Reject unknown fields on cache configuration table - Count only successfully removed files in eviction reports and size accounting - Log and continue orphan namespace pruning if individual entry removal fails - Prevent symlink traversal during orphan namespace pruning - Skip pruning ~upstreams/ when secret is not explicitly configured to protect sibling caches * fix(pnpr): preserve in-flight tmp files and check mtime before LRU eviction - Ignore active temporary files (.tmp*) unless older than 1 hour threshold - Prevent in-flight tmp files from inclusion in LRU candidates or totals - Check file effective time before LRU deletion to avoid removing replaced files - Add test verifying in-flight tmp files are preserved during eviction |
||
|
Zoltan Kochan
|
698893b8f1 |
feat(pnpr): metrics, request IDs, and enriched access logging (#16944)
* feat(pnpr): metrics, request IDs, and enriched access logging Expose Prometheus metrics endpoint at `GET /-/metrics` and `GET /metrics` emitting standard text exposition with in-flight gauges, request counters, duration histograms, and cache statistics. Forward and validate incoming `x-request-id` headers or generate fresh UUID v4s, recording them on response headers, tracing spans, and enriched access logs under `pnpr::access` alongside user identity, client IP, user agent, latency, and bytes. Resolves pnpm/tasks#131 (Section C4). * fix(pnpr): normalize metrics methods and clarify scraper docs Normalize request methods to standard HTTP methods or "OTHER" and key by static strings to eliminate allocations and bound cardinality. Clarify scraper access policy and byte accounting in documentation. * fix(pnpr): clarify metrics endpoint doc for npm accept headers Clarify that requests to GET /metrics carrying npm packument Accept headers receive the metrics package packument, as noted by CodeRabbit. |