mirror of
https://github.com/pnpm/pnpm.git
synced 2026-10-11 10:08:53 -04:00
pnpm@12.5.1
244
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zoltan Kochan
|
71611cf86e | chore(release): pacquet 12.5.0, pnpr 0.1.0-alpha.12 (#15069) | ||
|
Zoltan Kochan
|
b3da608b5b |
feat(config): name the ecosystem a registry serves (#15054)
pnpm/pnpm#15043 gave the programs pnpm downloads one setting. This does the same for the registries it downloads packages from, which were three settings sharing neither a name nor a shape: `registry` / `registries`, `python.indexUrl` with `python.extraIndexUrls`, and `cargo.indexUrl`. A `registries` entry now names the ecosystem it serves: registries: https://internal.example/simple/: ecosystem: pypi https://pypi.org/simple/: ecosystem: pypi https://index.crates.io: ecosystem: cargo `ecosystem` is absent on an npm registry, so every entry written before there was anything else to serve means what it did. An ecosystem with several indexes searches them in the order declared, and the first index that has a package supplies it. `registries` is an `IndexMap` for that reason: `python.extraIndexUrls` was a list, so a URL-keyed `BTreeMap` would have handed the search order to URL spelling. uv marks one index `default = true` and gives it lowest priority wherever it sits; pnpm reads position instead, because one rule is easier to hold than two. A URL serves whatever the last layer to declare it says, and loses every role an earlier layer gave it. Each layer's `registries` is validated on its own, so without that rule one URL could be an npm registry to the machine and a PyPI index to the repository at once. Credentials are the point. `registry`, `cargo.indexUrl` and `tools.node.mirror` all resolved credentials by origin from the machine's auth sources; `python.indexUrl` read them only from an inline `user:pass@` in the URL. A private PyPI index was therefore the one package source in pnpm whose password had to be written into the committed `pnpm-workspace.yaml`. `registries` refuses a credential in a key or a field, so moving Python indexes there settles it the way it is settled everywhere else, and deletes the machinery Python had grown for the inline form: the `IndexAuth` route hook, `validate_index_credentials`, and `ERR_PNPM_CONFLICTING_PYTHON_INDEX_CREDENTIALS`. One behavior change follows from that. The old code picked a credential by longest-matching declared index, so a credentialless index below another withheld its parent's credentials. npm matches on auth entries instead, so a credential configured for `//host/simple/` reaches `//host/simple/public/`, and the machine narrows it by configuring the narrower path. The test named for the old rule is replaced by one asserting what this model guarantees: a credential does not travel to an index on another origin. The word and its values are the repository's own: pnpr's registry config already spells `ecosystem: cargo` and `ecosystem: pypi`. The enum is not, because pnpr's carries `oci` too and pnpm installs from no OCI registry, so accepting it would be a value pnpm parses and then refuses further in. The two agree on the wire. `python.indexUrl`, `python.extraIndexUrls` and `cargo.indexUrl` have not been released, so they are gone rather than deprecated. Related to pnpm/pnpm#14945 |
||
|
Zoltan Kochan
|
854138e9fd |
feat(python): install a release from its source distribution (#15009)
Item 5 of pnpm/pnpm#14945. Two of the eight repositories surveyed there depend on a distribution that publishes no wheel at all: sentry needs `python-u2flib-server`, which has 15 source distributions and no wheels, and marimo needs `lzstring`, whose wheels stop at 1.0.3. Both failed with an empty version set that said nothing about why. A release whose wheels the target accepts none of is now offered as the source distribution the index serves beside it, ranked after every wheel, so a wheel always wins and the archive is downloaded only where nothing else installs. `.tar.gz` and `.zip` archives go through the shared artifact pipeline as `ArchiveStoreProjection::RawArchive`, which gives them the digest check, retries, store dedup and offline replay the wheel path already has. Their files are then copied out of the CAS into a temporary tree, so a backend that writes beside its sources is not writing into content every project on the machine shares. What the release requires is read from the wheel the build produces, because a source distribution declares it nowhere else. The build is therefore part of resolution rather than of installation: `Step::NeedMetadata` dispatches on the candidate kind, and `--lockfile-only` builds too. Because that wheel is the answer, `finish_build` no longer holds every build to its directory's manifest. `Contract::Interpreter` says the built wheel is the contract, which is what lets a `setup.py`-only archive with dependencies build at all; workspace projects and git checkouts keep `Contract::Manifest`. Every archive URL goes through `validate_url` before it is fetched, at the lockfile boundary and again before a build starts: the store reads a `file:` URL from the filesystem, and a `pylock.toml` is untrusted input, so without that check a lockfile could name an archive on this machine and have its build backend run. The wheel path already refuses such a URL outright. A build runs the release's own code, so it is approved the way a git dependency is, under `allowBuilds`. pnpm builds only with the interpreter running the install, so a project that locks for several environments and reaches a release none of them has a wheel for is refused rather than locked from a wheel built for the wrong platform. Where one environment does have a wheel and another does not, both land on one PEP 751 package and each environment takes what it can use, which is why `Solved::source_key` deliberately does not tell a wheel and a source distribution of one release apart. pnpr has no interpreter, so it offers source candidates but hands the project back to the client as soon as a solution needs one built. The client's pnpr fallback now recognizes any "must be resolved by the client" answer rather than only the direct-URL one. Separately, a failed resolution says which of its causes the empty version set was: no index publishes the distribution, its releases publish nothing this interpreter can install, or the project's own requirements select none of the versions it offers. Reading an index page therefore records why each release it left out was left out. A source distribution is built again on every install rather than kept as the wheel it produced; `plans/PYTHON_SPIKE.md` records that limit. Related to pnpm/pnpm#14945. |
||
|
Zoltan Kochan
|
22c6c4f2e8 |
feat(python): support git and url requirements (#14983)
Resolve Python git and direct wheel requirements in the client rather than silently replacing uv source declarations with index packages. Preserve manifest version constraints, extras, markers, and workspace inheritance. Pin git sources with PEP 751 vcs records and record wheel SHA-256 hashes. Reuse the isolated PEP 517 builder, verified wheel CAS ingestion, git transport policy, and atomic cache publication. Require approval before running code from a git dependency or its build requirements. Preserve submodules in cached checkouts and prevent recursive build dependencies from deadlocking. Keep source identity in wheel reuse and in lockfiles covering multiple target environments. Record PEP 610 origin metadata for git and URL installs. Let pnpr return a specific client-resolution response for transitive URLs. Add CLI coverage for direct and uv sources, git revisions and subdirectories, legacy projects, workspace inheritance, integrity and identity failures, build approval, recursive builds, submodules, platform-specific sources, backtracking between direct and index dependencies, requirements-file URL sources, and frozen offline replay. Document the supported forms and add a pacquet minor changeset. Related to pnpm/pnpm#14945, item 6. |
||
|
Zoltan Kochan
|
6d73648565 |
chore: run the tests a change affects, with smoke tests for its dependents (#14985)
`pnpm/CONTRIBUTING.md` required `just ready` before every commit, explicitly including documentation and comment edits: a full `cargo nextest run` over ~11,500 tests for changes that cannot break them. The root guide's "never run all tests" rule was scoped to the TypeScript sections, so it read as not applying to Rust. The checks now match what a change can break. Formatter, typos, and workspace-wide check and lint before every commit, since those are cheap and catch the cross-crate breakage a scoped selection hides, plus the tests for what the diff affects. The full local run is reserved for changes whose blast radius cannot be named, because CI already runs the suite on three platforms. `just test-affected` resolves changed files to packages through `cargo metadata` and runs them through `run-rust-tests.mjs`, expanding any `pnpr-*` selection so feature unification does not silently skip backend tests, and refusing to guess when a change reaches files every crate compiles against. Selection is crate-level rather than `rdeps()`-based: `pnpm-cli` holds a third of the workspace's tests and sits downstream of nearly everything, so `rdeps()` selects 84% or more of the suite for any core crate. Restructuring that target is tracked in pnpm/pnpm#14984. Crate-level selection leaves the dependents unrun, so the new `smoke` profile runs in their place: one end-to-end test per area of CLI behavior, triggered by the dependent set rather than added to every run. Membership is by behavior area rather than code coverage, since nearly every end-to-end test walks the same install path. An exact-name filterset fails open, so a test checks every entry against the suite sources. The `testing-changes` skill carries the selection cookbook and the gotchas that make a scoped run lie. Related to pnpm/pnpm#14984. |
||
|
Zoltan Kochan
|
d6b0ef86d1 |
feat(python): install a workspace project from its own source (#14953)
A requirement naming a Python project in the repository was resolved from
the index. Where the index served nothing under that name the resolution
failed with an empty version set, and where it served something the
install silently got that package instead of the project being edited.
Such a requirement is now satisfied by the project it names, declared
under `[tool.uv.sources]` as `{ workspace = true }` or a path, the table
Python workspaces already write. A member inherits the table its
workspace root declares, and `[tool.uv.workspace]` says which projects a
workspace contains; without one, every project pnpm discovered is one it
may link. A requirement naming a project in the workspace that no source
declares is refused, because resolving it from the index would install
different code under the same name.
Projects are built by running the PEP 517 backend each one declares, in
an environment holding only what that backend asked for, including the
requirements a backend can name only once it can see the project. This
replaces reading the source layout out of the manifest: uv runs the
backend, and a manifest cannot say where a poetry-core project keeps its
modules, what a build hook generates, or what maturin compiles. The
project's own package is built the same way, so a dynamic version is now
the version its backend computes rather than a reason to skip it.
A backend is code from the index that a build executes, so pnpm runs one
only where `allowBuilds` names it, as it does for a dependency's build
scripts. The key is the Package URL `pkg:pypi/hatchling`, because that
setting is shared with npm and both indexes publish names such as
`esbuild`, `ruff` and `black`. An install that has not approved a backend
does not build the projects needing it, which `strictDepBuilds` makes an
error.
A built project is installed editable unless the source says otherwise,
and recorded in `pylock.toml` as a PEP 751 directory package whose path
is relative to the lockfile. PEP 610's `direct_url.json` records where it
came from, which a wheel built from a directory cannot carry itself.
A build backend may write to stdout, so the host helper answers pnpm on a
copy of the descriptor and gives the operation stderr. Reading the built
wheel belongs to the interpreter it is installed for, not to the
environment the backend needed.
Related to pnpm/pnpm#14945.
|
||
|
|
80b6225274 |
fix(sbom): normalize the repository URL published in SBOMs (#14795)
* fix(sbom): normalize the repository URL published in SBOMs `pnpm sbom` published the manifest's `repository` value verbatim in the CycloneDX `externalReferences[].url` and the SPDX `homepage`. The npm `owner/repo` shorthand is not an iri-reference, so SBOM consumers such as Dependency-Track reject the document. pnpm v11 and pnpm v12 now apply the same rule to every published repository value: - expand the `owner/repo` shorthand to the `git+https` GitHub URL that npm's hosted-git-info derives for it - parse other values with the WHATWG URL parser and emit them in their normalized form, with any embedded `user:password` removed - drop values that do not parse or are not the shorthand (scp-style remotes, emails, relative paths) instead of publishing them Closes pnpm/pnpm#14773 * fix(sbom): drop a username-only authority and check shorthand segments Review follow-ups to the repository normalization, applied to pnpm v11 and pnpm v12 alike. A repository URL now loses its userinfo even when it carries no password. GitHub and GitLab both accept a token in place of the whole `user:password`, so the username alone can be the secret. An ssh remote keeps its `git@`, which names the login the host is reached with rather than a credential. A shorthand's owner and repository now have to consist of the characters the hosts allow: ASCII letters, digits, `-`, `_` and `.`. `owner/repo?%` used to expand to `git+https://github.com/owner/repo?%.git`, where the `.git` lands in the query and `%` is not a valid escape. The `github:`, `gitlab:` and `bitbucket:` prefixed shorthands expand to the URL npm's hosted-git-info derives for them, instead of being dropped from the SBOM. `bugs` and `repository` read their field and clear their credentials through one pair of helpers, so the two can no longer drift. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjkYZtr2fTKD4wu3w1GCz4 * refactor(sbom): normalize repository values with pnpm's hosted-git parser Both stacks already ship an npm-compatible hosted-git parser: the Rust `HostedGit` of `pnpm-resolving-git-resolver`, which `pnpm licenses` already applies to this same manifest field, and the `hosted-git-info` fork the TypeScript git resolver uses. The SBOM command now expands shorthands through them instead of through a parser of its own, so its output is the URL npm derives. What that changes, in both pnpm v11 and pnpm v12: - `gitlab:group/subgroup/project` keeps every namespace segment, a committish survives as the URL's fragment, and an scp-style remote such as `git@github.com:foo/bar.git` becomes the https URL of the same repository instead of being dropped. - A value counts as a URL when it parses and has a host, so `https:/github.com/foo/bar.git` is completed rather than dropped, while `github:owner/repo` still reaches the shorthand parser and `mailto:` and `file:` values, which name no repository a consumer can reach, are dropped. - A shorthand that names no owner (`github:repo`) is dropped: the URL derived from it has an empty owner segment. The ssh exception to credential stripping now names the `ssh` and `git+ssh` schemes instead of accepting any scheme whose name ends in `ssh`, so the token in `not-ssh://token@host/repo` is removed. A probe of forty repository values confirms the two stacks agree on every one, malformed percent escapes included. Both suites assert that table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjkYZtr2fTKD4wu3w1GCz4 * test(sbom): pin how both versions treat a gist repository The ownerless-shorthand guard also keeps out `gist:<id>`, which pnpm v11's parser expands and pnpm v12's does not know at all. Dropping it on both sides is what keeps the two versions publishing the same SBOM, and a gist named by its URL is published like any other URL. Both suites now assert that pair, so a later change cannot quietly expand the shorthand in one version alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjkYZtr2fTKD4wu3w1GCz4 * style(sbom): cut the prose to what the code cannot say The comments had grown into a second description of the implementation: doc comments enumerating branches the function body and its tests already spell out, rationale copied from the code into the tests that exercise it, and e2e test docs restating their own names. What survives is the part a reader cannot derive: why `CycloneDX` makes a raw manifest value unpublishable, why an ownerless shorthand and the gist form are dropped, why a host decides whether a value is a URL, what parsing buys beyond validation, and why an ssh login is not a credential. Each sits once per stack, on the function that decides it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjkYZtr2fTKD4wu3w1GCz4 * fix(sbom): drop a URL whose percent escapes are incomplete The WHATWG parser keeps a `%` that begins no `%XX` escape exactly as the manifest wrote it, so `https://example.com/%zz` survived normalization and reached the CycloneDX and SPDX documents. An iri-reference admits no such thing, which leaves the SBOM open to the rejection this change exists to prevent. Both versions now check the serialized URL before publishing it, on the path `repository` and `bugs` share. The changeset also becomes one idea per sentence, as the guide asks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjkYZtr2fTKD4wu3w1GCz4 * test(sbom): pin the committish a hosted parser decodes Both parsers decode a shorthand's committish, so `owner/repo#release%251` derives a URL ending in a stray `%1`. Dropping it is the point of the escape check, not a casualty of it: the alternative is publishing an invalid iri-reference. The two versions were measured to agree, and the case now sits beside the other incomplete escapes in both suites. The helper's doc loses the sentence that restated its name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjkYZtr2fTKD4wu3w1GCz4 * refactor(sbom): read the percent escape off the front of the segment Splitting on `%` and inspecting the two bytes that follow says the same thing as indexing from each match, without the index arithmetic that invites a question about the final byte. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjkYZtr2fTKD4wu3w1GCz4 --------- Co-authored-by: Zoltan Kochan <z@kochan.io> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ayush Singh
|
3a99eaea72 |
ci(rust): lint the workspace on Windows (#14808)
Rust CI / Clippy ran on Ubuntu only, so no job linted the #[cfg(windows)] half of the workspace. The test job builds that code, which type-checks it but runs no lints. Add a windows row to the Clippy matrix and clear what it found: a raw-pointer borrow of the console-mode out-parameter, an unused type parameter on the Windows stub of link_symlinked_executable, a redundant clone in the cmd-shim tests, and imports and test helpers that only unix code reaches. Two findings come from Windows's wider PathBuf. PackageManifest measures 200 bytes on Linux and 208 on Windows, which crosses large_enum_variant's threshold, so GateManifest::Found boxes it. result_large_err is allowed workspace-wide instead: with large-error-threshold lowered on Linux, error enums across ten crates sit just under the 128-byte limit, and the same 8-byte creep pushes them over. Boxing every one of them is its own refactor. zizmor needs the benchmark workflow's checks: write scoped to the job that posts a check, since a fork PR has advanced-security off and fails on any finding. Fixes pnpm/pnpm#14801. |
||
|
Juan PicadoandZoltan Kochan
|
0b8a1cd07d |
fix(pnpr): truncate upstream search at the fetch budget instead of failing with 400 (#14874)
An upstream with search enabled made almost every npm search fail with 400 "refine the query": the guard refused whenever the upstream reported a total above 2000, and npmjs's loose full-text search reports five-digit totals for nearly any term (jquery 23k, even narrow hyphenated names 30k+ because the tokenizer matches each word). The page and result budgets now bound what pnpr actually downloads. A source small enough to exhaust keeps the deduplicated total exact, as before; a larger one is truncated after the budgeted pages and its unscanned remainder folds into total as an over-approximation. A huge from offset walks at most the same budget and returns an empty window instead of a 400. Related to pnpm/pnpm#11973. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
55434d2463 |
feat(python): replay a lockfile on any target that installs it (#14847)
A pylock.toml was reused only when the whole marker environment and the ordered wheel tag list were equal to the ones that produced it, so a kernel update invalidated a lockfile whose interpreter, wheel tags, requirements and packages were unchanged, and a frozen install on CI failed for it. Closes pnpm/pnpm#14843. A lockfile is now replayed when it still applies to the target: the requirements, index and requires-python are the ones it was resolved for, and every pinned wheel carries tags the interpreter accepts. The recorded environment is not compared. Whether the locked graph is still the one the markers select is settled by re-solving it against them, which the locked install already did; that check is exact where an equality check on the environment refused every kernel update. A wheel that installs here is the same wheel wherever it was chosen. An install that may resolve again does so when the locked graph no longer matches the markers, with a warning naming the lockfile and the resolver's explanation, instead of failing. A frozen install keeps failing on it, and keeps failing on a lockfile pinning a wheel the target cannot install, with the reason attached. The environments marker written to the lockfile names python_version and the marker variables the solved graph reads, instead of every variable of the resolving environment. PEP 751 requires an installer to refuse a lockfile whose environments none of its markers satisfy, so a marker over the whole environment made other installers refuse it after a kernel update too. The lockfile format is unchanged; tool.pnpm still records the resolving target, which a pnpr server's answer is checked against. The environments marker names python_full_version when a locked package's Requires-Python tells patch releases of the running minor apart, so a PEP 751 installer does not accept the lockfile on a patch release a package excludes. A fetch failure on a lockfile resolved for another target replaces the lockfile when the install may resolve again: such a lockfile may pin wheels the markers no longer reach, and was replaced without fetching anything before replay existed. A lockfile resolved for this exact target pins only wheels the target needs, so its fetch failures fail every install. |
||
|
Zoltan Kochan
|
2ea04e6e52 |
style(rust): adopt complexity-aware rustfmt (#14875)
Pin pnpm's rustfmt fork and use a cached wrapper for local formatting, CI, pre-push checks, and editor integration. The formatter has its own dated nightly runtime while the workspace keeps its existing compiler. Choose chain layout by call count and argument complexity, with a 40-column allowance for short expressions. Attach the first method when the complete first line would otherwise end within the continuation indentation. Preserve simple receivers, cap intermediate leading accesses at column 80 and final accesses at 100, and separate fields and await after wrapped methods. Apply the same rules in conditions. Keep Max heuristics and compact struct literals, with an independent 35-column destructuring limit. Reformat pnpm and pnpr, extract only the helpers and test modules needed for existing size limits, and fix macro commas without relaxing lint rules. Related to pnpm/pnpm#14562 and pnpm/pnpm#14862. |
||
|
Zoltan Kochan
|
9cd124f665 |
refactor: limit Rust structs to eight fields (#14872)
Pin perfectionist to the merge containing too_many_struct_fields and configure max_fields = 8 with tests included. Refactor oversized internal state, options, and request structs across pnpm and pnpr. Group fields by responsibility, reuse existing context and policy types, and pass groups directly to the helpers consuming them. Use shorter field names inside their namespace and fetching for fetch settings. Update all callers, fixtures, and documentation links. Reuse install/project/peer groups through their consumers instead of reconstructing them. Share run/exec execution arguments and their mapping, existing archive/config-dependency store contexts, and GitSource cache inputs. Preserve owned versus borrowed representations where tasks need different lifetimes. Preserve fixed external configuration, serialized document, and binding interfaces with narrowly scoped expectations that identify the format. There are no legacy exemptions for ordinary internal structs. Cache workspace debug/release artifacts explicitly and rotate their cache namespace so old Perfectionist binaries cannot leak into CI after a pin change. Reproduced the cache inclusion with Actions glob settings and tar, and verified that the new paths exclude the lint library. Related to pnpm/pnpm#14562. |
||
|
Zoltan Kochanandgithub-actions[bot]
|
b62f5b004f |
chore(release): 11.27.0 (#14856)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Zoltan Kochan
|
7737634d5d |
fix: reject update --no-save under strict minimumReleaseAge only when a pick is immature (#14842)
`pnpm update --no-save` under `minimumReleaseAgeStrict` failed before it resolved anything, so it could never succeed even when every version it would pick is past the cutoff. Tools that refresh a lockfile without touching manifests, Renovate among them, update this way, and pnpm 11 accepts the same run. The refusal now happens where the immature picks are known. `Install`'s `persist_policy_excludes` boolean becomes a three-state `PolicyExcludes`: `Persist` writes approvals to `minimumReleaseAgeExclude`, `Skip` leaves the workspace manifest alone (`dedupe --check`, `remove`, `--dry-run`, embedder installs), and `Forbidden` marks the run the user asked not to save, where an approval that cannot be recorded is an error. Closes pnpm/pnpm#14835 |
||
|
Zoltan Kochan
|
e29016f20b |
style(rust): cap nesting depth at three (#14825)
Pin perfectionist to 495d33163b1e4ab6b0cbb4742a2130c655849f7d and enforce excessive_nesting with max_depth = 3 and exempt_tests = false. Refactor the Rust workspace's violations with guards, pattern matching, and small helpers. Retain short-circuiting, allocation behavior, synchronization boundaries, and SQL registration transaction semantics. Include optional SQL auth backends and Windows directory handling. Keep single-expression iterator closures where they clarify transformations; prefer loops or named helpers for multi-statement bodies. Related to pnpm/pnpm#14562. |
||
|
Zoltan Kochan
|
cfd4a73149 |
style: stop aligning TOML entries and comments (#14818)
taplo padded TOML with spaces so that things line up in a column, which makes unrelated lines show up in diffs. Two separate options did this. `align_entries` padded the key of every entry so the `=` signs share a column. In `[workspace.dependencies]` that meant padding every entry out to the width of the longest crate name, so adding or renaming one crate reflowed the whole block. `align_comments` did the same for trailing comments, so the clippy lint tables in the root manifest repadded every comment whenever a lint name grew. Set both to false and reformat the 107 files taplo covers. The diff is whitespace only: `git diff -w` reports `.taplo.toml` as the only file that differs. Verified with both the locally installed taplo 0.10.0 and the 0.8.1 that CI pins, so `pnpm ci:toml-fmt` passes on both. |
||
|
Zoltan Kochan
|
c74ff6e0cf |
refactor: limit Rust source files to 400 lines and tests to 800 (#14804)
Split Rust production modules and test suites across the repository to keep production files within 400 code lines and test files within 800. Group modules by responsibility and test scenarios while retaining public APIs and existing test binaries. Move Node release public keys into individual data assets and update their generator. Preserve key bytes, fingerprints, and ordering. Update module imports, documentation links, and snapshot locations after extraction. Document the limits and verify them with the proposed upstream rule's lexer without depending on an unmerged lint. Related to pnpm/pnpm#14562. |
||
|
Zoltan KochanandClaude Opus 5
|
19eb394486 |
chore(release): pacquet 12.4.1, pnpr 0.1.0-alpha.11 (#14802)
* fix(changeset): pnpr's ecosystem route prefixes break nothing published The intent declared a major bump, which the release engine applied as plain semver: 0.1.0-alpha.10 escalated its stable target to 1.0.0 and restarted the prerelease counter, so pnpr was heading for 1.0.0-alpha.0. The `/npm/`, `/cargo/` and `/pypi/` prefixes only address a registry that serves more than one ecosystem, and the ability to serve more than one arrives in this same release. No deployed pnpr can be broken by them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGJYxrAARz5kpZnESxm3xd * docs(skills): add a release-notes curation skill The wording rules for one changeset live in CLAUDE.md. Nothing describes the set: that the composed section under .changeset/changelogs is the release page and the only surviving copy once the bump deletes the intents, that a defect introduced and fixed inside one release window has no reader, that entries naming the same change from two pull requests should be merged, or that a major intent on a 0.x package jumps it to 1.0.0. Calibrate the per-entry advice against release pages worth reading. esbuild titles every entry and pairs it with before and after output. uv keeps one idea per bullet and names a flag, file, or platform in each. Playwright leads with a few highlights before its flat list, and uv's category headings give a skimmer somewhere to stop. pnpm's composer emits no headings beyond Major, Minor, and Patch, so the skill asks for the same structure to be built out of the entry order. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGJYxrAARz5kpZnESxm3xd * chore(release): pacquet 12.4.1, pnpr 0.1.0-alpha.11 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGJYxrAARz5kpZnESxm3xd * docs(changelog): curate the 12.4.1 and pnpr 0.1.0-alpha.11 release pages The composed sections were a flat dump of one intent per pull request: forty three pacquet entries and thirty four pnpr ones, several describing the same change from different angles, several describing a regression that never reached a published version, and all of them in whatever order the intent filenames happened to sort in. pacquet opened on a hard-link-limit edge case with the `ignoredOptionalDependencies` and `linkWorkspacePackages` bugs buried mid-list; pnpr stated the new ecosystem route prefixes twice and spread one container registry across seven entries. Merge the entries that describe one user-visible change, drop the notes for defects introduced and fixed inside this release window, trim the prose that only a contributor would care about, normalize the issue links, and order each page from the failures that break an install down to the wording fixes. pacquet goes from forty three entries to thirty six, pnpr from thirty four to nineteen. Give `@pnpm/napi` a line as well. It bumps through its fixed group with pacquet and had no intent of its own, so it was shipping a bare `## 12.4.1` heading. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGJYxrAARz5kpZnESxm3xd * docs(changelog): open each release page with a summary line Deno opens a release post with one sentence naming the top few changes, so a reader decides from that line whether to read on. The composed section has room for the same: `tail -n +2` in the release workflow keeps a paragraph placed under the version heading, and it becomes the first line of the GitHub release body. Widen the sources behind the skill while here. Yarn's pages are the conventional-commit log with the PR appended, which is the shape pnpm's uncurated output already has, so name it as the thing being fixed. Gitea publishes the ranking the ordering section was reaching for and puts SECURITY second, which the ranking had left out entirely; note that the Major, Minor and Patch headings cut across it, so a security fix on a patch bump lands below every feature and the lead paragraph has to carry it. SQLite and Gitea calibrate the one-sentence default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGJYxrAARz5kpZnESxm3xd * docs: address the review threads on the release pages The `updateConfig` entry said "A setting nothing set is left out", a reduced relative clause that a reader has to parse twice in a published release note. The version check told the reader to read the line for each released package, which skips the one most easily missed: `@pnpm/napi` carries no intent of its own and rides pacquet's fixed group, and release.yml fails the build when the two have drifted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGJYxrAARz5kpZnESxm3xd * docs(changelog): group the release pages by subject Thirty six patch entries under one heading is a wall. A reader looking for whether their install bug is fixed has to read every line about sbom timestamps and help text to find out. Name the groups instead. pnpm 12.4.1 splits into installing, resolving and linking, performance, scripts and tasks, commands, configuration, Windows, and messages; pnpr's minor changes into registries, container images, publishing, builds, and discovery. The order inside and between groups is unchanged. Both consumers tolerate the extra heading level. release.yml writes the Rust release body with `tail -n +2`, and getChangelogEntry slices between headings of the same depth as the version heading, so a `####` neither ends the slice nor is dropped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGJYxrAARz5kpZnESxm3xd --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Zoltan Kochan
|
495bfe7554 |
refactor: enforce 40-line production Rust functions (#14792)
* refactor: enforce 40-line production Rust functions Adopt perfectionist::overly_long_function across the Rust workspace with tests exempt. Refactor all existing violations without legacy suppressions and place helper contracts at their declarations. Related to pnpm/pnpm#14562. * chore: omit release note for internal Rust refactor |
||
|
Trevor BurnhamandZoltan Kochan
|
2af57c138c |
fix(store-dir): rebuild when the side-effects cache has nothing to restore (#14744)
A side-effects row is recorded whenever a build script ran, including a build that changed nothing inside the package directory. `calculate_diff` returns `added: None, deleted: None` for that case and the row is stored anyway, so a package whose whole build effect lands elsewhere -- a git-hook installer, a script seeding a shared download cache -- gets an empty row. `build_side_effects_maps` turned every row into an overlay, so an empty one became an overlay equal to the base files and the `is_built` gate took it as a hit. The scripts were skipped and nothing was materialized in their place: the package directory after such an install is byte-identical to the pristine tarball, and the build's effect never happens at all. The cache key does not include the consuming project, so this is not limited to reinstalls -- a project's first install can hit a row an unrelated project seeded on the same machine. pnpm 11 drops the row instead. Its `checkPkgFilesIntegrity` adds to `sideEffectsMaps` only under `if (added) ... else if (deleted)`, so an empty row matches neither branch, `sideEffectsCacheKey` is never set, `isBuilt` stays false, and the build runs. Both stacks write the same empty row and only their read paths differed, so this drops it at the same point pnpm 11 does. An entry's presence now means there is build output to materialize, which is what the `is_built` gate already assumed. A row carrying only `deleted` still counts as a hit. That build did happen and its effect is reproducible from the row. `prefetch.rs` already filters an empty `side_effects_maps` out of the prefetch result, so a package whose only row is dropped contributes no entry at all and the frozen-install fast path in `build_phase` still sees an empty table. Closes pnpm/pnpm#14717 --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
92168e9381 |
refactor: bring every production function under the local-bindings cap and enforce the rule (#14760)
Bring every production function under the 12-binding cap and enable the `too_many_local_bindings` dylint rule (`max_bindings = 12`, `exempt_tests = true`). Each oversized function reads its inputs' fields in place, takes owned fields before the borrows it needs, and hands its remaining locals to a named step or a typed view: the virtual-store creation, the workspace and peer resolvers, the tarball ingestion, the npm resolver's mirror and result assembly, the NAPI bindings, the config load, the pnpr resolver and OCI/publish handlers, and the CLI commands. Behavior, error precedence and allocations are unchanged. pnpr's streamed resolve gained `#![recursion_limit = "256"]`: with the engine's futures split into steps, rustc overflowed proving the spawned future `Send`. The config-to-executor `scriptsPrependNodePath` mapping now lives once in deps-restorer. Related to pnpm/pnpm#14562. |
||
|
Zoltan Kochan
|
2fa282950f |
style: adopt perfectionist's new size and shape rules (#14730)
Pin the perfectionist dylint library to a revision instead of a tag and enable the four rules that revision adds: arbitrary_source_item_ordering, core_instead_of_std, excessive_cognitive_complexity with a limit of 10, and redundant_derive_more_forward_template. The tree is made to pass them by refactoring rather than by exemptions: no #[expect], no #[allow], and no raised limit. Oversized functions are split into named steps, conditions that carried a meaning are given names, three-way outcomes become enums, repeated parameter lists become types that own the shared values, and the nearly sixty CLI config overrides that repeated the same three shapes are driven from macro tables. cargo-dylint refuses a library built against a newer dylint_linting, so CI moves to cargo-dylint/dylint-link 6.0.4 to match the pinned revision. The 6.0.1 bootstrap failure the previous 6.0.0 pin worked around is fixed there. Related to https://github.com/pnpm/pnpm/issues/14562. That issue plans seven size-and-shape rules; only excessive_cognitive_complexity has landed upstream so far. It also proposed a per-site #[expect] for every existing violation, which this PR deliberately does not do. |
||
|
Zoltan Kochan
|
c6cf52abe5 |
feat(pnpr): scope registry identity by ecosystem and expose directory (#14700)
Scope registry addressing by ecosystem and local name. Accept ecosystem configuration groups, qualify their serving-table keys and router sources, and support ecosystem-specific defaults while preserving flat configurations. Give grouped hosted registries isolated default storage namespaces; retain explicit org overrides for existing storage. Use ecosystem-aware addressing for reads, writes, discovery, and defaults. Keep upstream credential classification tied to the qualified identity and reject encoded path separators in named HTTP endpoints. Expose a caller-filtered registry directory with one entry per ecosystem and name, ecosystem defaults, mount information, namespace patterns, and source order. Withhold inaccessible registries and restricted routing metadata, exclude secrets and upstream addresses, and disable response caching. Test configuration ambiguity, source/default isolation, same-named registry visibility, grouped Cargo publication and downloads, absence of cross-protocol package reads, and resolver credential isolation. Document configuration, migration, and directory semantics and include a pnpr minor changeset. |
||
|
Zoltan Kochan
|
fb8625eda3 |
feat(pnpr): browse hosted packages and fix concurrent registration (#14696)
Add browse=true to the npm and Cargo search APIs. Reuse hosted discovery, routing, authorization, result projection, and pagination. Skip upstream search in browse mode. Preserve ordinary empty-search behavior. Cover pagination and totals, named registry mounts, registry and package access restrictions, routed-away packages, and zero upstream requests. Document the endpoints and add a pnpr minor release note. Serialize capped libsql registration transactions on the shared connection. Use immediate transactions and bounded retries for SQLite lock conflicts across independent backends. Retain the atomic counter as the user-limit authority. Cache the bcrypt hash within each registration request. Test independent databases opening the same file, lock release, extended SQLite error codes, and retry behavior through the real libsql HTTP client. Close temporary-file handles before atomic replacement and retry transient Windows file locks. This fixes concurrent task-cache publication failures. Add registration and Windows cache-write patch release notes. |
||
|
Zoltan Kochan
|
7abf8f6c90 |
test(pnpr): take the npm publish fixtures from one place (#14669)
Seven suites carried their own copy of the publish document a client sends and the two digests it puts in `dist`. `tests/common/npm.rs` holds them since the multi-replica suite was added; the rest now include it instead of keeping a copy each. The copies had already drifted: the S3 suite built its attachment filename from the whole package name, so it would have addressed a scoped package's tarball under the scope. Nothing depended on that, since its packages are unscoped. |
||
|
Zoltan Kochan
|
06db4f1ba4 |
feat(pnpr): keep pipeline run records with the hosted packages (#14668)
A run record is the account `pnpm pipeline` gave of one run: testimony about something that happened once, not derived data a replica can rebuild. It lived on the replica that received the submission, so behind a load balancer the pipeline UI showed a different history depending on which replica answered, and a scale-to-zero container took its records with it. Run records now live in the hosted store, beside the packages every replica shares. They keep their append-only rule across replicas: the create-only write the object store and the local backend both offer decides the winner, so a run id recorded on one replica is refused on every other. The staged-publish namespace already needed read, create-only write, compare-and-set replace, remove and list on a reserved namespace of the hosted store. That is now the backend's record API, addressed by namespace and key, with staged publishes and run records as its two callers rather than a second copy of the plumbing. `Storage` names the namespaces, as it already did for `.staged`. On a local storage root the records move from `pipeline-runs/v0` to the reserved `.pipeline-runs/v0`, which also takes them out of reach of a package named `pipeline-runs`. Related to pnpm/pnpm#12199. |
||
|
Zoltan Kochan
|
8760cfa9f3 |
fix(pnpr): approve a staged publish once across replicas (#14665)
Staged publish records live in the hosted store, which every replica of an S3-backed deployment shares, but an approval read the record and replayed the held publish without ever writing the record back. Two approvals of one stage reaching two replicas therefore both ran the publish, and a rejection that landed between another replica's read and its publish did not stop it. The approval now claims the record first: it rewrites it with the time the approval started, conditional on the bytes it read. The object-store backend puts that write under `If-Match` on the record's `ETag`; the local backend, which one process owns, compares the bytes under a staged-write lock. A second approval finds a record that is no longer the one it read and answers 409, or 404 once the winner has consumed the stage. The claim is released on every failure path, and expires after ten minutes so a replica that died mid-approval does not strand the stage. Staged records are also written create-only now, so a record can never be written over one another request owns. Adds a multi-replica test suite that drives two routers over one object store the way a load balancer spreads requests over two containers: concurrent publishes of one package keep both versions, a stage created on one replica is approved on another, two replicas approving one stage publish it once, and a dist-tag written on one replica is served by the other. Related to pnpm/pnpm#12199. |
||
|
Zoltan Kochan
|
4ed7a436ef |
feat(pnpr): add oidc and keyless publishing (#14666)
Add operator-configured OIDC providers, explicit subject/claim bindings, and separate browser and workload authentication paths to pnpr. Use the approved openidconnect dependency for discovery, authorization code exchange, and signature verification, with pnpr checks for authorized party and token times. Keep OIDC sessions short-lived and out of the persistent token stores. Bind browser login to a signed HttpOnly Secure cookie, PKCE, nonce, and one-use state. Anonymous login starts allocate no server-side state. Bound callback attempts and concurrent exchanges. Bound successful sessions, discovery response sizes, deadlines, and JWKS refreshes. Restrict discovery destinations to public IP addresses through the shared system DNS resolver. Keep valid cached keys available while a refresh performs network I/O. Redact callback query parameters from request logs. Accept prefixed workload ID tokens as registry credentials without token-store lookups, restricting them to exact packages in an explicitly named hosted npm registry and enforcing the normal publish ACL. These credentials cannot mint durable pnpr tokens or use account, unpublish, batch, or other protocol endpoints. Document provider setup, limitations, and a GitHub Actions workflow that needs no long-lived registry secret. Cover signed-token validation, claim constraints, login replay and expiry, discovery caching, and request restrictions with tests. |
||
|
Zoltan Kochan
|
5adbbbe5e9 |
fix(config): route @jsr packages through the built-in JSR registry (#14653)
`Config::resolved_registries` is the map every scope-routed lookup picks a registry from: the resolver, the lockfile's supply-chain policy verifier, `pnpm why`, `pnpm view`, `pnpm audit`. It carried only the user's configured scope routes plus the default registry, so an `@jsr/*` package fell through to the default registry, which serves no such packument. Installing a lockfile holding a JSR dependency then aborted with ERR_PNPM_META_FETCH_FAIL on a 404. The built-in `@jsr` route was reachable in two narrower places only: the resolver's `jsr:` specifier path, and `resolved_registry_declarations`, which is what makes `pnpm config list` print a route the install disagreed with. The TypeScript CLI puts the route in the scope map itself, which is why the same lockfile installs there. Add it to the resolved map, where every consumer sees it, with a configured `@jsr:registry` still winning. The route stays out of `registry_declarations`, the shape a pnpr server is told about, so npm.jsr.io is not put in front of its allowlist on requests that resolve no JSR package. It stays out of the package-manager bootstrap too: that map resolves the package manager alone. pnpr consumes the same map, and its route allowlist — the resolver's SSRF boundary — carried one built-in public route, npm, so a graph holding a JSR dependency could not resolve through a default server. npm.jsr.io joins it: pnpm reaches that registry without configuration, and it hosts no private content. Both built-in routes now allowlist `https` alone, since that allowlist also gates every redirect hop. `pnpm changelog` picked its registry with a hand-rolled scope lookup that had the same gap; it now calls `pick_registry_for_package` like every other command. Closes pnpm/pnpm#14649 |
||
|
Zoltan Kochan
|
6e87cbf149 |
feat(pnpr): finish oci registry follow-ups (#14652)
Complete the OCI registry's remaining hosted and upstream workflows. Reuse the verified streaming cache and publish journal. OCI batch entries carry a base64 manifest and reference layers uploaded through /v2/ first. Upstream tokens are cached by repository with bounded expiry and capacity; credentials are restricted to the configured origin and trusted token service. Bind short-lived OCI credentials to the original token's hash and the addressed registry path. Check the original token's restrictions and current registry policy on subsequent requests. Scoped tokens cannot access other pnpr APIs, and mounts require pull scope for the source repository. Maintain a separate filesystem package index, migrating legacy stores once at startup. Catalog requests can find nested repositories without walking through the parent's layer population. Prune index entries after removal or failed creation. Uncached manifest probes use upstream HEAD, falling back to a verified GET when legacy registries omit the digest header. Integrity failures abort shared blob streams, and OCI batch validation preserves the original registry errors. Explicit blob deletion reserves a marker in the repository document before removing bytes. Increment its generation so a staged or recovered publish cannot restore a manifest referencing the deleted content. An interrupted deletion blocks new publishes until offline oci-gc finishes it. All replicas must be upgraded before using online deletion. Filesystem stores retain their existing exclusive-owner requirement. Add regression coverage for scope and expiry checks, credential revocation, cache poisoning, upstream authentication boundaries, cumulative upload limits, nested catalogs, batch rollback, interrupted deletion, and a manifest commit racing blob deletion on another replica. Closes pnpm/pnpm#14630. |
||
|
Zoltan Kochan
|
85fb54b776 |
feat(pnpr): complete oci protocol surface (#14647)
Add the remaining hosted OCI protocol endpoints tracked in pnpm/pnpm#14630. Ranged reads seek on the filesystem and use conditional ranged object-store requests on S3. Keep full reads for HEAD, mismatched If-Range validators, and unsupported or malformed ranges. Mounts resolve the source through the normal hosted read gate and the target through the publish gate. Reuse the streamed upload and digest verification path, including bounded S3 transfers. This avoids sending the layer again from the client, while still using server scratch space and copying its bytes. Reject image referrers without an artifact type or config media type. Persist only subjects and artifact-type hashes in the journaled repository document. Read annotations from matching manifests on bounded referrers pages, with at most 32 manifest reads and 8 MiB of manifest content per request. Backfill older entries incrementally as pages are queried. Coordinate migration with a separate lock table when a page encounters unindexed metadata. Hold the package writer lock only for the conditional metadata merge, so blob reads do not stall publishers. Fully indexed pages do not acquire migration locks. Preserve concurrent document changes and serialize deletion with metadata updates on the filesystem backend. Referrer discovery follows retained manifests, so deleting tags keeps referrers and deleting manifests removes them. Exclude catalog names before the cursor before authorizing candidates, then apply page sizes to the authorized results. Preserve the addressed API base in continuation links. Zero-sized catalog pages skip enumeration. General catalog pages still use the existing full storage listing because filesystem traversal is unordered; this change adds protocol pagination, not a repository index. Related to pnpm/pnpm#14630. This affects pnpr only; the pnpm CLIs are unchanged. |
||
|
Zoltan Kochan
|
06ecd063a9 |
feat(pnpr): add oci registry maintenance (#14644)
Add offline OCI blob collection and shared upload sessions for S3-backed registries. These changes address the correctness and operations section of pnpm/pnpm#14630. Use immutable objects for request chunks and compare-and-set writes for the session record. Only a successfully recorded chunk advances the accepted offset. Completion assembles the chunks on local scratch for digest verification, then uses the existing multipart blob path. Losing a replica's scratch no longer loses an accepted upload. Conditionally freeze the accepted chunks and digest before promotion, keeping failed promotion retryable with the same digest. After promotion, session cleanup is best effort; cleanup failures do not turn a committed blob into a failed response. Closing or expiring a session invalidates stale writers before chunk deletion. Keep garbage collection offline so no publisher can introduce a new reference between the mark and delete phases. Walk retained manifests and index children rather than tags alone, and validate their integrity before deleting any candidate. Inventory blob files separately from the request-path package listing to include unpublished, document-only, and nested image repositories without making catalog requests enumerate blob populations. Stream the inventory into temporary SQLite storage so a large registry does not require its complete file listing and candidate set in memory. Startup expiry and large-blob multipart transfer were already present in the initial OCI implementation. Extend expiry to shared sessions and reuse and test multipart cleanup for both promotion and session chunks. Related to pnpm/pnpm#14630. This affects pnpr only; there is no pnpm CLI implementation to mirror. |
||
|
Zoltan Kochan
|
3610b85e30 |
feat(pnpr): serve container images (#14629)
Adds OCI as a fourth ecosystem beside npm, Cargo, and Python: a hosted image registry that docker, podman, and skopeo push to and pull from. The distribution API cannot be moved under a path prefix. A client derives the API root from the image reference's host, so `pnpr.example.com/acme/app:1.0` always requests `/v2/acme/app/manifests/1.0`. `/v2/` therefore mounts at the host root whatever else is served, and the repository name alone selects the registry through the declared-provenance rules the other surfaces use. Artifactory has to burn the first path segment as a repository key for want of that invariant; pnpr does not, so the image name stays the image name. `/oci/v2/` and `/oci/~<name>/v2/` are served too, for podman and containerd, whose registry configuration does accept a path. Repository names are `/`-joined, which the pattern language had no shape for. `PackagePattern::parse` now takes the ecosystem and offers each one only the wildcard its names can carry: npm keeps `@scope/*` and `@*/*`, images get `<namespace>/*` over a single leading component, and Cargo and PyPI get neither, since a flat name could never match one. A `packages:` key is normalized through that language rather than through the name rules, so a wildcard key parses as itself. Blob uploads stream to a local file across requests instead of buffering in memory, and are verified against the promised digest before anything is stored. The manifest write is the commit point and rides the existing publish journal; blobs land outside it, content-addressed and invisible until a manifest names them, which is what leaves unreferenced blobs for a collector rather than half-publishing a release. A tag carries the time it last moved, so a transaction recovered after a crash cannot drag one back to an older manifest. `Basic` credentials now carry a token as the password under any username, which is how `docker login` sends them, and `GET /v2/` challenges an anonymous caller even where reads are open: a client settles its authentication scheme on that one response, so a 200 would leave it no way to authenticate a push. Verified end to end against docker 29.7.2 (login, push, pull, run), podman 5.8.4, and skopeo 1.22.2, plus an anonymous pull and a push refused after logout. The two client stacks take different upload paths and between them cover both: moby sends a blob monolithically, while containers/image chunks it. Not yet served: proxying an upstream image registry, blob collection, the referrers API, cross-repository blob mounts, and ranged blob downloads. |
||
|
Zoltan Kochan
|
f00e7680aa | chore(release): 11.26.0 (#14638) | ||
|
Zoltan Kochan
|
39dcf560af |
feat: add pipelines, run records, and Cargo build caching (#14233)
Compose frozen installation, affected-project selection, workspace task scheduling, output caching, and run reporting in pnpm pipeline. Add a polling watch agent and an authenticated pnpr run-record service. Keep completed task results separate from Cargo incremental state. Opted-in Cargo tasks share immutable snapshots across linked worktrees, materialize private writable targets, and always execute. Keep snapshot storage separate from installed-package storage so eviction cannot break existing builds or installations. Key Cargo snapshots by repository inputs and compilation environment. Invalidate local-unit and build-script freshness after relocation, and invalidate freshness after content changes even when mtimes are retained. Use staged, integrity-checked restoration and coordinate pipeline writers through locks outside the disposable cache. Document the prototype's input, storage, and remote-service limitations. Add regression coverage for worktree reuse, relocation, deletion, concurrency, copy fallback, and invalid snapshot handling. Related to pnpm/rfcs#22, pnpm/rfcs#23, and pnpm/rfcs#25. |
||
|
Zoltan Kochan
|
b71013fa64 |
feat(pnpr): answer cargo search over hosted crates (#14624)
The Cargo surface served the sparse index, downloads, publish, yank and unyank, but not `GET api/v1/crates`, so `cargo search --registry <pnpr>` failed. It now answers over the crates the registry hosts, matching on crate name the way npm search does. A crate document gains the description of its most recent publish. The sparse index carries none, so it was the one field of the crates API that pnpr had nowhere to read from. The hosted half of a search — the discovery-source walk, the per-source access gate, and the per-name hosted gate — is now shared with the npm surface, which differs only in how it renders a name into a result. Upstream sources contribute nothing yet, as an npm upstream does until its `search` is turned on. Searching one needs the `api` base from its `config.json` rather than the index base pnpr proxies. Related to pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
4db65b22fc |
feat(pnpm): configure the Cargo sparse registry (#14619)
Add a workspace setting for the Cargo sparse-index URL and thread it through local and pnpr-accelerated resolution. Preserve that source in Cargo.lock, read the registry download template, and configure Cargo to use pnpm's vendored directory for the selected registry. Take the registry source as an argument to the resolver rather than defaulting to crates.io, so local and accelerated resolution record the same source. Accept a lockfile's default-registry source only when the configured index is crates.io, and reject a dependency that names a third-party registry rather than every registry but crates.io. Keep crates.io as the default and preserve its existing cache and authentication behavior. Bound the registry configuration document, and skip the registry entirely for a workspace with no registry crates. Share Cargo download-template expansion and sparse-index prefixes between the client and pnpr. Related to pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
6ba99fb386 |
feat(pnpr): share cargo compilation caches (#14620)
Expose named compiler caches through the WebDAV subset used by sccache. Apply pnpr account access and publication policies before buffering uploads, including existing token restrictions. Entries are immutable and verified against a digest bound to their cache scope, key, and bytes before serving. Reuse the shared artifact store's filesystem/S3 selection, quota accounting, publication lifecycle, and orphan reclamation. Bound compiler uploads to two concurrent bodies per server and reject excess requests before buffering. Store digest and payload as separate buffers in one conditional object write. Use metadata-only HEAD and duplicate checks; GET still verifies content before serving it. Stock sccache trusts the server and allowed publishers; it does not verify pnpm signed envelopes. Document trusted CI publication and HTTPS requirements. Also document sccache 0.17's absolute Rust build-path requirement and its read-only multilevel behavior: remote hits backfill disk, but new compilation misses are not cached locally when a tier is read-only. Add HTTP, policy, integrity, quota, and real Cargo integration coverage. Install sccache for Rust CI and local test setup. Extract the existing miette diagnostic normalization into a shared test helper to fix a shim assertion exposed by the full suite's longer temporary paths. |
||
|
Zoltan Kochan
|
3ae388bf73 |
refactor(pnpr): canonicalize ecosystem package names (#14617)
Centralize npm, Cargo, and Python package-name validation and normalization in `pnpr-package-name`. Make storage and cache keys canonical by construction, and keep journal recovery keyed by the recorded ecosystem. This removes protocol-crate dependencies from `pnpr-config` and eliminates duplicate normalization in server and resolver call sites. Related to pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
b2b28e1d43 |
feat(pnpr)!: prefix multi-ecosystem registry routes (#14616)
Use ecosystem URL prefixes when a pnpr instance serves multiple package ecosystems. This prevents npm package names such as `npm`, `cargo`, and `pypi` from colliding with protocol routes. Keep package routes at the root when the registry serves only one ecosystem. This preserves existing npm-only registry URLs and keeps test and benchmark clients compatible. Generate metadata and tarball URLs from the same conditional layout, and classify named private registry URLs under either form. Related to pnpm/pnpm#14610. |
||
|
Zoltan Kochan
|
97554102d0 |
docs(pnpr): record the registry-prefix tilde in the release note (#14615)
Explicit routes spell the registry prefix `/~{registry}`, and matchit
matches the raw path, so a percent-encoded `~` no longer selects those
routes on any ecosystem surface. `RawPathParams` decoded the segment
before `tilde_registry` read it, so `/%7Enpmjs/pkg` used to reach the
named registry. `~` is unreserved, so a conforming client sends it
literally, but the change is visible and belongs in the note.
Also state the method contract on the `405` test directly rather than
by contrast with the dispatcher that no longer exists.
Follows pnpm/pnpm#14610.
|
||
|
Zoltan Kochan
|
f99d19da55 |
feat(pnpr): resolve Python projects through the install accelerator (#14613)
Resolution lived inside the CLI, where it could only run beside an interpreter: the pubgrub provider read a registry that downloads wheels and asks `host.py` what they require. A registry server resolving on a client's behalf can do neither, so the rules move to `pnpm-python-resolver` and the fetching stays with whoever can do it. `step` runs one pubgrub pass and either solves the project or names the one distribution or wheel it still needs, which is the shape the CLI's loop already had. `POST /-/pnpr/v0/resolve` reads `"ecosystem": "pypi"` beside npm and Cargo. The body carries the project's requirements and the interpreter they are for; the answer is the `pylock.toml` the client writes. pnpr reads the metadata file an index publishes beside each wheel (PEP 658, either spelling), falls back to the wheel itself only for an index that publishes none, and keeps what it read for every client that follows — so the client stops downloading whole wheels for versions a resolution then rejects. The client verifies the answer rather than trusting it: the lockfile has to record the inputs this install asked about, its wheels are downloaded and checked against the index's digests, and the project is re-solved against the metadata of what actually arrived. Server-side, a read that does not match the digest the index published is refused before it reaches the solver. Reads are bounded and scoped as the Cargo path's are: an allowlisted index origin, per-project route classification, single-flighted cold reads, and caps on the distributions, metadata reads, each response, and the bytes one request holds. Asking a server which ecosystems it resolves is one memo for both ecosystems now, keyed by server and ecosystem. Related to pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
3d1036bf55 |
refactor(pnpr): give the npm surface explicit routes (#14610)
The npm surface dispatched by segment count: seven catch-all routes
(`/{a}/{b}/{c}`, `/{a}/{b}/{c}/{d}`, ...) whose handlers re-derived the
resource from `-`, `-rev`, `@` and `~` markers, once per arity and once
more per method. matchit 0.8 matches static text inside a segment, so a
registry prefix can be spelled `/~{registry}` and each npm URL can be its
own route. The path-less base needs no deprecation: what blocked explicit
routes was the assumption that a parameter has to own a whole segment.
Each address is now registered once, for the default target and for a
named registry, and handlers are named for the resource they serve. The
`-` and `-rev` markers anchor the table, so a scoped name spelled as two
literal segments and the same name percent-encoded land on the same
handler. The account, Cargo and Python surfaces move to `/~{registry}`
too, so a first segment that is not a registry prefix no longer reaches
an endpoint that only meant to serve one.
`serve_packument`, `serve_version_manifest` and `serve_tarball` take the
addressed registry, folding away the path-less/`~<name>` split that each
call site used to make, and `private_if_caller_gated` folds into
`caller_scoped`, which already did the same job for the other ecosystems.
A path no route claims now answers 404 rather than reaching an arity
catch-all, and `/{pkg}/-/whoami` reads as the tarball address it is
instead of being shadowed by the account endpoint.
Closes one item of pnpm/pnpm#14599
|
||
|
Zoltan Kochan
|
77026ff61d |
refactor(pnpr): name storage after documents and blobs (#14609)
The storage layer serves three ecosystems but was named for one. Rename its vocabulary so a Cargo index file and a Simple API page are documents and a `.crate` and a wheel are blobs. `pnpr-storage` now exposes `read_hosted_document`, `open_hosted_blob`, `reserve_hosted_blob`, `finalize_blob_slot` and their peers on both the filesystem and the S3 backend, and a publish write conflict is `RegistryError::DocumentWriteConflict`, logged as `document_write_conflict`. `PackageName` keeps its name: it holds a package's name and never a version, and a name is what a crate and a Python project have too. The npm surfaces keep their own packument and tarball vocabulary, which is accurate there. Separately, a name now rejects `?`, `#`, `%`, whitespace, and control characters. A request path is percent-decoded before a handler parses a name from it, and a name is interpolated into the upstream URL, so `GET /foo%23bar` was authorized and cached as `foo#bar` while fetching `foo` — a package rule that named `foo` never ran. The journal manifest's `packument_file` and `tarballs` aliases had no coverage, so a sealed entry that spells its keys that way now has a recovery test. Related to pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
09b546affe |
feat(pnpr): publish across ecosystems in one transaction (#14608)
`PUT /-/pnpr/v0/publish` takes a batch whose entries each name their ecosystem and carry what that surface's own publish endpoint takes, with the binary parts base64-encoded. An entry that names none is an npm publish document, so a body the npm batch endpoint accepts is already a valid one. The address is in pnpr's own namespace rather than beside the npm batch endpoint, which is part of the npm surface and answers under `/npm/` with it. `GET /-/pnpr` advertises the protocol as `publish: [0]`, and now answers for a registry-only tier, which has a pnpr protocol of its own for the first time. Every entry is parsed, authorized and verified before any of them is staged, every blob is staged before any of them is committed, and the commit is the single journal transaction the publish flow already had: a release that spans ecosystems becomes visible all at once. The per-ecosystem publish endpoints keep their own behavior; they and the batch now share one staging path, `StagedPublish`, which carries the ecosystem of the package it staged. Closes one checkbox of pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
614936e1ba |
feat(pnpr): resolve Cargo dependencies through the install accelerator (#14607)
`POST /-/pnpr/v0/resolve` served npm alone. It now serves every ecosystem from one address, with the body naming which one it speaks: an absent `ecosystem` field is npm, as it was for every client written before this, and `"ecosystem": "cargo"` carries a `cargo metadata` document plus the sparse index to resolve it against. The server walks that index in waves — asking pacquet's Cargo resolver which crate names are still missing, fetching those, repeating — and answers with the rendered `Cargo.lock`. Index files are cached under the server's cache directory with the `packument_ttl` npm metadata gets, in a namespace keyed by registry origin and by the caller's route scope, so a private index cached for one caller's credential never satisfies another's fetch. The registry a request names is checked against the route allowlist before any fetch, and every credential comes from the server's route policy rather than the request, as on the npm surface. `pnpm install` and `pnpm add crate:<name>` use it whenever `pnprServer` is set, so a cold Cargo install no longer fetches one index file per crate in the graph. The handshake advertises the ecosystems the server reads; a server that does not name `cargo` leaves the client resolving locally, which keeps a new pnpm working against an older pnpr. The request carries the dependency graph alone: `cargo metadata` reports absolute paths in `manifest_path`, `workspace_root`, every target's `src_path`, and the package ids, and none of it takes part in resolution. There are no per-package frames for Cargo. Resolution is a single pubgrub solve rather than an incrementally-yielding tree walk, so nothing is known before everything is, and the response is the terminal `done` frame alone. Related to pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
1cb9a559f9 |
feat(pnpr): journal the Cargo and Python publish transactions (#14606)
The Cargo and Python surfaces reserved a blob, finalized it, then updated the project document, all outside the commit journal the npm surface uses. A crash between the finalize and the document write left a blob nothing mentions: harmless to readers, but inconsistent, and a second copy of a problem npm had already solved. The journal now carries a document of any ecosystem. Each entry records the ecosystem beside the computed document bytes, and the caller supplies a `HostedDocuments` merge that both the commit and startup recovery run for that format, so the merge rule stays with the surface that owns it. Committing is one operation for every surface — seal, apply, remove the entry — and the apply is exactly what recovery runs, so the npm publish flow no longer keeps an inline second copy of it. `store_hosted_artifact` now stages the blob and the document it computed and commits them together. A blob whose immutable slot another writer already owns is left out of the document, and a transaction that ends up recording nothing writes no document at all; the surface reports that as the duplicate publish it is. A post-seal apply that fails partway re-runs once from the journal, as the npm flow did inline before. Related to pnpm/pnpm#14599. |
||
|
Zoltan Kochan
|
1e83fc5f8b |
feat(pnpr): serve Cargo and Python registries beside npm (#14598)
Add Cargo and Python registry protocols to pnpr. Concrete registries declare an ecosystem, and routing filters mixed router sources by the requested protocol. Ecosystem prefixes provide default and named registry endpoints while preserving the original npm aliases. Reuse hosted document and blob storage, package access rules, upstream metadata caching, and verified artifact streaming across the new surfaces. Keep upstream artifact cache identities bound to metadata checksums. Check metadata before cache lookup so changed or removed entries take effect. Honor package-specific Cargo privacy rules when advertising auth-required. Require authentication before reading Python upload bodies. Bound multipart boundaries and use the existing regex crate for delimiter searches, checking boundary suffixes so binary lookalikes remain part of the uploaded file. Reject Cargo archive traversal, links, oversized decompressed streams, and manifests whose package identity differs from the publication metadata. Reject package configuration keys that collide after normalization. Apply route allowlists to Cargo and Python metadata and artifact fetches, including redirects. Require secure same-origin destinations for configured headers. Rebuild headers on every approved redirect so cross-origin metadata and artifact downloads succeed without forwarding upstream credentials. Share this redirect loop with the existing network metadata fetcher and retain its request-budget guard while response bodies are read. Transfer permits to streamed artifact responses so body consumption or cancellation releases them. Keep one pnpr timeout budget across redirects and the final response body. Stream through a bounded producer whose deadline runs independently of downstream polling. Reject hosted Python blobs absent from publication metadata. Abort publication on an immutable object-store conflict and remove the conflicting staged file. The filesystem backend remains single-writer; replicated deployments use the object-store backend. Journaled publication across all registry surfaces remains tracked in pnpm/pnpm#14599. Add protocol unit tests and HTTP regressions for publication, routing, authentication, content negotiation, caching, and integrity validation. |
||
|
|
ea9b8f22e2 |
feat(pnpr): allow configuration through environment variables (#14434)
In somes cases it's easier to configure the pnpr server through environment variables (e.g. containers). This commit binds all CLI arguments to an environment variable (e.g. `--config` -> `PNPR_CONFIG`, `--public-url` -> `PNPR_PUBLIC_URL`) --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
879347fa1d |
feat(pnpr): support browser registry discovery (#14544)
Add an exact-origin CORS allowlist for separately hosted browser registry UIs. Complete hosted search pagination and maintainer discovery, record authenticated publishers, and serve npm-compatible organization package maps. Apply registry routing and access rules before counting or returning local packages. Add opt-in upstream search and organization discovery. Use only configured upstream credentials. Treat upstream pages as a visible, deduplicated sequence so filtered entries neither underfill pages nor leak through provider-reported totals. Derive publisher and organization permission metadata from pnpr authorization rather than client or upstream service-account values. Closes pnpm/pnpm#14543 |