Discover Python projects from shallow paths to deep ones.
Each uv workspace declaration controls which descendants pnpm reads.
Outside declared workspaces, skip conventional sample and documentation directories.
Template and test-fixture directories are skipped too.
An explicit workspace member can still include any of these directories.
Closespnpm/pnpm#15058.
Keep the latest checksum manifest as the common path. When an exact patch is
absent, binary-search python-build-standalone release tags and cache the
immutable checksum manifest used to find it.
Closespnpm/pnpm#15060
* fix(cli/add): name the ecosystem an add refuses, not the prefix
`pnpm add pkg:cargo/serde --global` was refused with "crate: dependencies
cannot be installed globally", a prefix the command line did not carry.
A Package URL reaches every one of these checks, so the messages name the
ecosystem they are about instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(resolving-npm-resolver): reject an alias that shadows a prefix in any case
A named-registry alias was compared to the reserved prefixes exactly,
while a specifier's prefix is read case-insensitively wherever the
notation it comes from is. A URL scheme is, so `PKG:npm/lodash` is a
Package URL, and a registry named `PKG` was accepted at configuration
time and then unreachable: every selector that named it was read as a
malformed purl instead.
An alias is now rejected whatever case it is written in.
`shadows_reserved_version_prefix` is separate from
`is_reserved_version_prefix` because reading a dep path back must keep
matching the prefix pnpm wrote, exactly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(cli/global): cover the package a global purl installs
The mark a Package URL carries reaches the global path through the group
each request splits into, and nothing failed if it stopped arriving: the
purl tests all ran project installs, so dropping the mark there would
have left them green while `pnpm add -g pkg:npm/node@22.0.0` went back to
installing the Node.js runtime.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(deps-path): cover the case policy each reserved-prefix check takes
The two checks differ only in case sensitivity, which is the whole point
of having both, so a test pins each side: an alias shadows a prefix in any
case, and a dep path's version slot still reads `PKG:1.0.0` as a named
registry because that is the prefix pnpm wrote.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(deps-path): reserve a mixed-case alias only where the prefix is read that way
Folding the case of every reserved prefix rejected aliases that worked: a
`npm:` specifier is read exactly, so `Npm:lodash` names a registry called
`Npm` and that entry resolves. Only a prefix a selector may spell in any
case has to be reserved that way, which today is the Package URL scheme.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(cli/add): cover every refusal that names an ecosystem
Four of the six refusals had no assertion on their text, and none had a
Package URL case, which is the spelling that motivated the wording. Each
is now asserted in both spellings.
The changesets say what pnpm does and stop there. One of them appended the
old behavior and the other explained why the entry was unusable, both of
which belong in the commit body.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(cli/add): drop a comment the refusal table already says
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(python): install wheels with bad RECORD hashes
Treat wheel RECORD as a structural manifest. The locked archive hash remains the integrity boundary.
Recompute hashes for the installed RECORD from imported contents.
Closespnpm/pnpm#15061
* test(python): accept mismatched RECORD hashes
* fix(python): reject malformed wheel records
Keep validating RECORD hash and size syntax without comparing the declared values to file contents.
* perf(python): stream installed RECORD hashes
Hash imported wheel files in bounded chunks and stream Copy-mode imports to their destinations.
* test(python): verify streamed RECORD digest
* perf(napi): take the repeat-install fast path with in-memory manifests
The Node-API bindings disabled both repeat-install short-circuits, because
the workspace-state check judged manifest freshness by the package.json
mtime, which says nothing about a manifest the caller supplied in memory
and may not exist on disk at all (a Bit component directory carries no
package.json). Every repeat install therefore ran the full frozen path,
re-copied every file: directory snapshot, and reported them as added, so
the embedder saw dependencies change on every run.
Add ManifestFreshness to the check. Mtime keeps the CLI behavior. Content
skips the stat and compares every importer with the wanted lockfile by
content, which is the authoritative check the mtime shortcut only avoids.
The bindings select Content and re-enable both short-circuits for
installs; peer-issue queries stay excluded because they exist to resolve.
The wanted-vs-current assertion in the content check now uses the
reachable-shape comparison the frozen-tree gate already used: the current
lockfile records only what the importers reach and none of the top-level
keys pnpm does not define, so a wanted lockfile with an unreachable
snapshot or an embedder's extension block was never equal to it and kept
forcing a full install. The skipped reason is now logged at debug level.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* chore: address the first review round
Keep the changeset user-facing, drop a test doc comment that narrated the
test's own setup and assertions, and require the content-mode check to
advance the recorded validation timestamp rather than merely hold it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: state the plain lockfile policy's contract instead of its fields
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
`pnpm add` reads a `pkg:` selector as the Package URL specification
defines it and routes it to the ecosystem its type names: npm to
package.json, cargo to Cargo.toml, and pypi to pyproject.toml. Each
purl is rewritten into the selector that ecosystem's own protocol
already takes, so it inherits that path's flags, defaults, and errors
instead of growing a second set.
Every decoded component is validated before the pieces are joined back
together. A name followed by a spec is how pnpm spells an npm alias, so
a name or a version carrying a decoded at-sign or colon would otherwise
redirect the install to another package. Qualifiers and subpaths are
rejected rather than dropped: a repository_url qualifier or a subpath
that pnpm ignores installs something other than what the selector names.
AddArgs::package_names now holds the selectors the npm add path receives
rather than the ones argv carried, which is what lets a purl reach the
--global and --config forms as well. AddPipeline keeps only the selectors routed
to another ecosystem, so the split has a single home, and
dispatch_install::add delegates that split to a named step to stay
within the local-binding limit perfectionist enforces.
A purl always names a package to install. pnpm add node@22.0.0 records a
runtime and pnpm add npm@11.0.0 records the project's package manager,
both from a bare name a person typed, so the package-manager and runtime
recognition skips a selector a purl was rewritten into.
That mark belongs to the request rather than to its text. package_names
holds AddRequest values, each carrying its own selector and whether a purl
was rewritten into it, so two requests that normalize to one selector keep
their own meanings and no second list has to be matched against or kept in
sync.
A rejected selector is quoted back by what it decodes to, because an
encoded separator would otherwise hide a user:pass@ authority from a check
made on the raw text. What decoding leaves unreadable to that check is
stopped at instead: a colon followed by an at-sign is how a URL spells
userinfo, and nothing pnpm installs pairs the two past its protocol, so no
list of the characters that can sit between a scheme and its slashes has to
be complete.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Zoltan Kochan <z@kochan.io>
A `namedRegistries` alias addressing a URL that `registries` serves to
another ecosystem is dropped, and a later layer's alias meets an index
role an earlier layer declared: only `registries` can retract that role,
so the alias loses there too. Covered by its own test now.
`add_python_index` is renamed to `add_python_index_searched_first`, which
carries what its doc comment was there to say, and the comment on
`write_index_credentials` keeps only the part the code cannot say: the
setting refuses a credential, so the `.npmrc` is where one lives.
Follow-up to review comments on https://github.com/pnpm/pnpm/pull/15054
pnpm/pnpm#15043 gave the programs pnpm downloads one setting. This does the
same for the registries it downloads packages from, which were three
settings sharing neither a name nor a shape: `registry` / `registries`,
`python.indexUrl` with `python.extraIndexUrls`, and `cargo.indexUrl`.
A `registries` entry now names the ecosystem it serves:
registries:
https://internal.example/simple/:
ecosystem: pypi
https://pypi.org/simple/:
ecosystem: pypi
https://index.crates.io:
ecosystem: cargo
`ecosystem` is absent on an npm registry, so every entry written before
there was anything else to serve means what it did.
An ecosystem with several indexes searches them in the order declared, and
the first index that has a package supplies it. `registries` is an
`IndexMap` for that reason: `python.extraIndexUrls` was a list, so a
URL-keyed `BTreeMap` would have handed the search order to URL spelling.
uv marks one index `default = true` and gives it lowest priority wherever
it sits; pnpm reads position instead, because one rule is easier to hold
than two.
A URL serves whatever the last layer to declare it says, and loses every
role an earlier layer gave it. Each layer's `registries` is validated on
its own, so without that rule one URL could be an npm registry to the
machine and a PyPI index to the repository at once.
Credentials are the point. `registry`, `cargo.indexUrl` and
`tools.node.mirror` all resolved credentials by origin from the machine's
auth sources; `python.indexUrl` read them only from an inline `user:pass@`
in the URL. A private PyPI index was therefore the one package source in
pnpm whose password had to be written into the committed
`pnpm-workspace.yaml`. `registries` refuses a credential in a key or a
field, so moving Python indexes there settles it the way it is settled
everywhere else, and deletes the machinery Python had grown for the inline
form: the `IndexAuth` route hook, `validate_index_credentials`, and
`ERR_PNPM_CONFLICTING_PYTHON_INDEX_CREDENTIALS`.
One behavior change follows from that. The old code picked a credential by
longest-matching declared index, so a credentialless index below another
withheld its parent's credentials. npm matches on auth entries instead, so
a credential configured for `//host/simple/` reaches
`//host/simple/public/`, and the machine narrows it by configuring the
narrower path. The test named for the old rule is replaced by one asserting
what this model guarantees: a credential does not travel to an index on
another origin.
The word and its values are the repository's own: pnpr's registry config
already spells `ecosystem: cargo` and `ecosystem: pypi`. The enum is not,
because pnpr's carries `oci` too and pnpm installs from no OCI registry, so
accepting it would be a value pnpm parses and then refuses further in. The
two agree on the wire.
`python.indexUrl`, `python.extraIndexUrls` and `cargo.indexUrl` have not
been released, so they are gone rather than deprecated.
Related to pnpm/pnpm#14945
perfectionist's `cloning_getter` (KSXGitHub/perfectionist#410) and
`overly_complex_condition` (KSXGitHub/perfectionist#409) both landed
upstream, so the pinned revision moves to the one that carries them and
the workspace comes under both rules at once.
`TestRegistry::url` and `PnprBenchmarkRegistryOverride::resolve_registry`
return `&str`, and the call sites that keep the value copy it there,
where the reader can see the copy. Most do not: they interpolate the URL
into a request or a fixture path. `EmulatedCancellation::receiver`
becomes `to_receiver` instead, since its one caller stores the receiver
in the run it builds, so the copy is the method's business rather than
the call site's. The test helpers that only forwarded a registry URL into
`config.registry` now take `&str` and copy once inside.
Conditions are capped at five `&&`/`||` operators, the point past which a
guard's clause list stops reading as one predicate. Five conditions were
over it. Each one either names the concept its clauses decide
(`is_valid_repository_path`, `snapshot_is_reusable`) or folds the clauses
that share a shape into one pattern. `frozen_tree_up_to_date`'s
eleven-operator `let` chain becomes a sequence of guard clauses, each
carrying the comment that explains the gate it holds, with the build
checks in a helper of their own.
Both caps are documented in the Rust code style guide.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keep init pinning metadata in the fixture cache even when a test replaces
its workspace YAML. Reuse the mtime helper after staging an env-document
conflict so the repeat-install check sees the edit.
Closespnpm/pnpm#15052.
python.pythonVersions repeated the table it sits under, so it read as
"python python versions". The table already says which ecosystem the
versions belong to, and the setting is now python.versions.
supportedVersions was the other candidate, for symmetry with
supportedArchitectures. It is not used: pyproject's requires-python is
what says which versions a project supports, pnpm reads it, and it
bounds what this setting may name. Naming the bounded thing after the
bound invites the two to be read as one. The two settings are not
adjacent in the file either, so the symmetry would rarely be seen.
The pylock.toml tool.pnpm key stays python-versions, and the internal
names that feed it with it. Inside a lockfile the key sits beside
platforms, where versions on its own says nothing about what is
versioned.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
pnpm downloads a Node.js runtime, a Bun runtime, a Deno runtime, a Yarn
release and, since this week, a Python interpreter. Two of those could be
pointed at a mirror, through two settings that shared neither a name nor a
shape: `node-mirror:<channel>` and `python.downloadUrl`. Bun and Deno had
none at all.
`tools` names them, keyed by the tool, with `mirror` saying where its
builds come from:
tools:
node:
mirror: https://mirror.example.com/node/download
bun:
mirror: https://mirror.example.com/bun
python:
mirror: https://mirror.example.com/python-build-standalone/releases
The key is the tool itself, with no category above it. Every taxonomy
this could have used breaks on its next entry: Node.js is a runtime, Yarn
is not, a Python interpreter arguably is, a Rust toolchain would not be,
and a cargo subcommand certainly is not. `tools` inherits no such split
and needs no rename when the next one arrives. It is also the word both
tools that manage many runtimes use, in proto's `[tools.*]` and mise's
tool-namespaced settings.
A mirror carries the project's own layout below it, so only the host
above it differs. That is what lets one string serve tools whose trees
are otherwise nothing alike, and why `tools.node.mirror` is the base its
release channels hang off, exactly as nodejs.org lays them out.
This is the tool an ecosystem runs on, not where its packages come from.
`registry`, `python.indexUrl` and `cargo.indexUrl` keep answering that,
which is why `tools.python` and the `python` section can mean different
things without competing.
`python.downloadUrl` has not been released, so it is gone rather than
deprecated, and its pending changeset now names the setting that replaces
it. `node-mirror:<channel>` has shipped and stays: it names one release
channel where the base names them all, so it decides the channel it names
and leaves the rest to the base.
Deno and Yarn are left out. Both read the GitHub API for release metadata
and take asset URLs out of the response, which a base URL cannot stand in
for. pnpm/pnpm#15042 covers the URL rewriting that would reach them.
`pnpm pack-app` also downloads the Node.js it embeds through `tools.node`
now. It passed no mirror at all, under a comment saying pacquet had no
such setting, which stopped being true when `nodeDownloadMirrors`
landed. It reads only `tools.node`: that runtime runs as the builder and
is kept in the shared pack-app cache, so a repository naming
`node-mirror:<channel>` would be choosing the program that packs
everyone's app.
Related to pnpm/pnpm#14945
The pre-commit formatter rewrites the file it is given, and accepted a
staged path whenever its last component was a regular file. That is not
enough to know the bytes it rewrites belong to the commit.
rustfmt truncates rather than replaces, so the inode survives and every
other name for it is rewritten too. Git stages a hardlink like any other
file and reports the tree clean, so an in-checkout `.rs` link to a file
anywhere on the machine reached the formatter, and formatting the commit's
own file rewrote the other one, with nothing in `git status` or the diff to
say so.
The paths that resolve out of the checkout were not reachable: git refuses
to index one beyond a symbolic link, and reports one whose directory became
a link as deleted from the working tree, which the unstaged check already
holds back. The rule is the script's now rather than a consequence of those
behaviors and the order of two checks.
A staged path is formatted only when it names one regular file inside the
checkout, and one that leads nowhere is named rather than fatal: `ENOTDIR`
and `ELOOP` used to escape as a stat error and abort the commit. The
formatter receives the resolved path, so it does not walk the links again;
the window between the check and the open stays open, as Node offers no way
to open a path one component at a time.
Follow-up to https://github.com/pnpm/pnpm/pull/15045, which merged before
the review comments this answers.
Three papercuts that together make a lint cost a full pre-push sweep.
`test-affected` took its diff against the local `main`, which is only as
current as the last time someone checked it out — and `main` usually lives
in another worktree here, so it lags. Every commit it lagged by read as a
change of the current branch: crates someone else touched got tested, and
a file every crate compiles against, landing upstream, made the whole run
refuse to scope and send you to `just ready`. Six commits of lag was
enough to attribute an unrelated `Cargo.lock` to a branch that had not
touched it. A `--base` that names a branch now diffs against that
branch's remote-tracking ref. A tag, a revision expression, a qualified
ref and `HEAD` are taken as given, since each already names one commit.
The pre-commit hook formats the Rust files being committed, with the
pinned rustfmt. Formatting is the one Rust check that needs no compile, so
it is the one that can run per commit; clippy and dylint stay in pre-push
where their cost is paid once. A file with unstaged changes as well is
left alone and named, since formatting the working tree and staging the
result would commit the part of it the author held back. The pathnames are
handled in Node, one argument per file, so a path holding a space, a glob
character, or a leading `-` reaches rustfmt intact, and only a regular
file is formatted: rustfmt writes through a symlink.
Both lint failures in pre-push named the command that reports the
findings rather than the one that applies them. `just fix` and the new
`just dylint-fix` fix most of what either finds, which is the difference
between one more sweep and several.
The doc named the collision the tags were added to prevent, which
describes the encoding they replaced rather than the one in the code.
Raised in review of pnpm/pnpm#15013 by Qodo.
`supportedArchitectures` stood for the cross product of its `os`, `cpu`
and `libc` axes, which cannot express the platform set a workspace
usually ships: Linux x64, macOS arm64 and Windows x64 is three platforms,
but as axes it is six, including the three nobody asked for. The setting
now also reads as a list of the platforms themselves.
An entry parses either as pnpm's own `<os>-<cpu>[-<libc>]` spelling, the
one `pnpm pack-app --target` already takes, or as the Rust target triple
of the same machine, the one uv and the rest of the Python tooling use.
Both canonicalize to one value, so two spellings of a platform are one
platform rather than two. On Linux the C library field also carries a
Python wheel ABI baseline, as in `linux-x64-manylinux_2_28`, which is the
Python-specific dimension the axes have nowhere to put. A Linux platform
that names no C library is the glibc platform, as `linux-x64` means
wherever else it is written.
This lets `python.platforms` go. It named the same thing in a spelling of
its own, in a per-ecosystem setting, while the setting that says which
platforms an install prepares for was already there. `pylock.toml` is now
resolved for the platforms `supportedArchitectures` names, so a workspace
declares its platforms once for its npm and its Python dependencies both.
The axes keep their meaning for a project that writes them, Python
included, where they stand for every platform they cross into.
The optional-dependency check judges a listed platform as a whole rather
than an axis at a time: `[linux-x64, darwin-arm64]` refuses a package
built for darwin x64, where the axes take it because each axis matches on
its own. The dlx cache key and the runtime-archive selector read the new
form too, the latter preferring the listed platform this machine is over
mixing one axis with another.
`python.platforms` and `python.pythonVersions` have not been released, so
this reuses their pending changeset rather than adding a migration.
Related to pnpm/pnpm#14945
Nothing outside the crate builds a cache key for a projection other than
the ordinary package one, so `ArchiveStoreProjection::mem_cache_key` goes
back to being internal and `package_mem_cache_key` carries the contract.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
python.downloads answered the same question runtimeOnFail already answers
for the Node.js, Deno and Bun runtimes: what an install does when the
runtime a project needs is not on this machine. Neither the setting nor
the Python support around it has shipped, so the two merge before the
first release rather than after it.
requires-python is to an interpreter what engines.runtime is to a Node.js
runtime, so every runtimeOnFail mode now has a meaning for Python.
download, which is also the unset default, installs an interpreter. error
reports the project. warn and ignore install with an interpreter the
machine has that requires-python rejects, which python.downloads had no
spelling for.
A declared environment matrix outside requires-python stays an error
under every mode: no interpreter choice can satisfy it, so it is a
configuration error rather than a runtime mismatch.
Related to pnpm/pnpm#14945.
`MemCache` slots were named by tarball URL and archive projection alone,
and `CacheValue::Available` was handed to the caller without comparing
the request's expected hash against the one the slot's bytes were
verified with. Two resolutions naming one URL but pinning different
integrities shared a slot, and whichever arrived second was served the
other's bytes.
`ArchiveStoreProjection::mem_cache_key` now carries the expected
integrity, and every writer and reader of the cache agrees on that
identity: the resolve-time fetch publishes under the hash it settles,
which is the hash its resolution records, and the install pass looks up
the hash the lockfile carries. The three gates that deduplicated by URL
for the same reason move with it, in `PrefetchingResolver`, in the pnpr
client's `TarballPrefetcher`, and in `ObservingResolver`.
The key is tab-separated (projection, network policy, integrity, URL).
`ssri` splits an integrity on whitespace, so no hash it holds can carry
a tab, and a URL cannot either; two distinct identities therefore cannot
spell the same key. An unpinned archive still keys on the URL alone,
because the hash is what such a fetch is there to learn.
That also removes the constraint noted in pnpm/pnpm#15013: a read of an
already-pinned archive can now publish its extraction, so a
manifest-less pinned resolution downloads its tarball once instead of
once during resolution and again during installation.
`FetchTarballForResolution` carries the revision mode along with it, so
a revision-addressed archive read during resolution spends that
protocol's single GET and publishes under the identity the install pass
looks it up by, rather than fetching directly and publishing where
nothing reads.
The bug is v12-only: the mem cache is a pacquet construct with no pnpm
v11 counterpart.
Closespnpm/pnpm#15021
Reuse the scheduler's iterative strongly connected components utility to
collapse cycles. Count pending targets and live outgoing dependencies per
component, then propagate removals backwards only when a component loses
its final live source. Reachability preprocessing, updates, and storage
are linear in graph size.
Group pending targets by package name and version, keeping peer-context
keys distinct for dependency classifications. Saturating a finding only
checks that finding's keys.
Extract the existing SCC utility into a shared module and expose its
component indices, avoiding a second implementation or a new dependency.
This handles cycles and deep graphs without recursion.
Stop the existing depth-first path walker before traversing branches with
no unsaturated findings. Keep peer-context keys distinct until their dev
and optional classifications have been merged, and keep traversing when
a nested finding or another package version still needs paths.
Add layered-diamond, post-saturation classification, many-target chain,
many-peer-context chain, sequential version saturation, and cyclic
saturation regressions.
The bug is specific to pnpm v12; pnpm v11 already prunes these branches.
Closespnpm/pnpm#15005.
Follow-up to pnpm/pnpm#15019 for the review items that had no inline
thread when it merged.
pnpm run and pnpm exec compare the configured workspace path and the
command's directory with links resolved, so a workspace reached through
a link still finds the environment its members share.
An add without --filter reads the workspace around the edited project
only when a [tool.uv.workspace] is declared at or above it, which is
where a project can take siblings from the repository or share their
environment. A project outside any declared workspace is read alone, as
it was before, so the common add does not walk the whole workspace.
Shared membership reuses the workspace each project belongs to from the
scope reading, rather than walking ancestors again per member. The
disagreement search compares distinct ranges rather than every member's
requirement, so many members asking for one range add no pairs.
Related to pnpm/pnpm#15015.
Item 14 of pnpm/pnpm#14945, the placement half. Every Python project pnpm
installed grew a `.pnpm/python-envs/` directory of generations beside its
`.venv` link, so airflow's 141 projects got 141 generation directories
where the repository maintains three environments.
Generations now live under `<store>/python-envs/<project>/`, one directory
per project named by the short hash of its canonical path, and `.venv` is
the only thing left in the project: an absolute link to the generation the
project currently runs. Publication is unchanged, an atomic symlink rename
in the project directory, and so is rollback, which republishes the previous
link. The link is a symlink or junction, so the store and the project need
not share a filesystem. Where they do, the clone and hardlink methods of
`packageImportMethod` now always have the environment on the store's
filesystem.
A `.venv` is managed when it links to a generation directory of any
project directory in the store, not only this project's, so a project that
moves keeps its environment rather than being refused as unmanaged. A link
into the project's own `.pnpm/python-envs`, where 12.4 published
environments, is managed too, so an upgrade relinks it on the next install
and leaves the old generations for the user to delete. A link whose target
is gone is replaced: the store can now be removed without the project, and
a link to nothing protects nothing. Under frozenStore, which promises that
nothing under the store is written, generations stay beside the project in
the 12.4 layout, with its real-directory checks. A link to any other directory, or a
`.venv` that is a real directory, is refused as before. The checks that
the project's `.pnpm` is a real directory go away with the directory: an
install no longer writes into the project's `.pnpm` at all.
On Windows the link holds the absolute target through the new
`pnpm_fs::force_absolute_symlink_dir`. The existing helper writes true
symlinks relative to the link, which node_modules relies on and which a
link into a store elsewhere on the disk cannot survive a move with.
Generation collection stays open, as `pnpm/plans/PYTHON_SPIKE.md` records;
a stale generation is now in the store rather than in the project.
Closespnpm/pnpm#15014.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Python override entries now accept `pkg:pypi/<distribution>[@<version>]` identities. pnpm normalizes them at the configuration boundary into the existing PEP 508 requirement representation.
PURL qualifiers and namespaces are rejected. Existing PEP 508 overrides and constraints retain their current behavior.
Let the members of a uv workspace opt into one Python environment.
A workspace root that declares [tool.uv.workspace] can set
`shared-environment = true` under [tool.pnpm.python]. pnpm then resolves
every member as one graph into one pylock.toml and one .venv at the
workspace root, the way uv resolves a workspace, while each project keeps
an environment of its own by default. Sharing is opt-in because uv's
model, which always shares, forces a repository whose projects genuinely
conflict to split into separate workspaces; per-project resolution needs
no such split.
The installer now prepares units rather than projects: a unit is a
project on its own or the members of a shared workspace, with one
interpreter, one lockfile and one environment. Workspace::memberships
groups the selected projects by the directory they install into, and
selecting any member of a shared environment prepares the whole
membership. The lockfile records the members under tool.pnpm.members and
the interpreter range they accept together as requires-python, so a
member joining or leaving invalidates it. A single-project unit records
neither, so existing lockfiles stay valid.
Two members that require versions of one distribution no offered
release satisfies at once are refused with an error naming both, wrapped
around pubgrub's report, which names the project as one root and cannot
say which members disagree.
An add without --filter in a workspace now discovers the workspace
around the edited project instead of that manifest alone: whether the
project shares an environment, and which projects it may take from the
repository, are declared around it.
pnpm run and pnpm exec resolve the environment the way the install
assigned it: the nearest directory with a manifest is the project, the
nearest workspace declared at or above it is the one it belongs to, and
only a member of a workspace that shares its environment uses the root's
.venv. The dynamic metadata of a shared workspace's members is prepared
with the interpreter the members share, and an add rereads every member
of the units it edits once the workspace lock is held.
The workspace state file is renamed into place through
pnpm_fs::rename_with_retry, so two pnpm processes installing one
workspace at once no longer fail on Windows when the second rename finds
the first still holding the file.
Follows up on item 14 of pnpm/pnpm#14945. Closespnpm/pnpm#15015.
`manifest` is optional in the `ResolveResult` a pnpmfile `resolvers`
hook returns. pnpm 11 awaits the fetch and reads the bundled manifest
when the resolver supplies none; pacquet had no equivalent, so
`walk::fallback_manifest` synthesized an identity-only manifest and
`extract_children` found no children. The package installed alone, with
no dependencies, no engines, exit 0 and no warning.
`PrefetchingResolver` already fetches an archive during resolution to
learn an integrity the resolution does not carry. Widen that to the
manifest: one read settles both, shared across every edge that resolves
to the same content.
The read now verifies the bytes against a pinned hash. Discovering an
integrity has nothing to check against, but a resolution that already
pins one is read here only for its manifest, and unchecked bytes would
put a tampered archive in charge of the dependency walk. The native
cache key carries the pinned hash too, so a read that verifies is never
served the bytes of one that could not.
Closespnpm/pnpm#15000
Record the maintainer's project philosophy as a shared reference for
implementation, triage, and review. Explain how feature value, abstractions,
compatibility, and user-selectable tradeoffs fit pnpm's core requirements.
Add a repository implementation workflow that checks existing capabilities,
prioritizes deduplication, assesses architectural impact, and validates through
the existing testing and review skills. Update review guidance to require
major releases for stable breaking changes.
Use the existing packageImportMethod setting and importer for unchanged Python
wheel files in project and isolated build environments. Remove python.linkMode
and PythonLinkMode rather than maintaining a second method vocabulary and
fallback implementation.
Return the actual import method from the shared low-level importer so Python
can restore wheel executable permissions without chmodding hardlinked store
files. Scope Python fallback tiers per source filesystem and environment, and
use private imports for isolated build requirements. Respect unpacked wheel permissions even when a data filename ends in
the CAS -exec suffix. Preserve private generated metadata and rewritten scripts.
Update the pending Python file-sharing release note and internal documentation.
Cover workspace settings, CLI overrides, auto selection, strict clone failures,
write isolation, and unpacked executable permissions.
Related to pnpm/pnpm#14945 and pnpm/pnpm#15006.
Extract the existing PR review-and-fix workflow into a repository skill and
invoke it from the git-wt hook. Commit and push verified fixes automatically,
then reuse the pull-requests skill to follow CI and review rounds to completion.
Preserve the existing PR status and leave merging to the maintainer.
Move the shared conflict-resolution script into the pull-requests skill.
Add task concurrency groups to pnpm v12. A task in pnpm-workspace.yaml
names a `concurrencyGroup`, and `concurrencyGroups` gives each group a
limit: at most that many tasks of the group run at once on the machine,
counted across every pnpm process. A task past the limit waits for a
running one to finish and prints who holds the slots.
A group's pool is N slot files under `<stateDir>/run-slots/<group>/`. A
task takes an exclusive advisory lock on one in `run_stages` and holds it
for every stage of its script, so the single-project run, the recursive
run, and the tasks a `pnpm pipeline` runs in-process are all counted,
each project's task separately. The operating system releases the lock
when the holder ends, however it ends, so a killed holder never leaves a
stale slot. `pnpm exec` names no task and is not gated.
A holder lists its group in `PNPM_HELD_CONCURRENCY_GROUPS` for the
scripts it spawns. A nested `pnpm run` of a task in the same group runs
under the slot its parent holds, which is what keeps a script that calls
`pnpm run` from waiting on itself under a limit of one.
`concurrencyGroups` merges entry by entry across the configuration
layers, so a machine can lower one group's limit through
`PNPM_CONFIG_CONCURRENCY_GROUPS` without restating the rest.
Every process that is meant to be counted has to start as
`pnpm <script>`, so the root manifest gains scripts that wrap the just
recipes and node scripts (`test:rust`, `test:rust-affected`,
`test:rust-smoke`, `check:rust`, `lint:rust`, `ready:rust`,
`pre-push:rust`), the pre-push hook runs its Rust checks through one,
and the agent guide, the testing-changes skill, and the contributing
guide name the pnpm scripts first.
This repository's own task declarations wait for a pnpm release that
knows the field: the pinned CLI rejects an unknown task setting, so
adding `concurrencyGroup` to pnpm-workspace.yaml now would break every
command for it.
Item 5 of pnpm/pnpm#14945. Two of the eight repositories surveyed there
depend on a distribution that publishes no wheel at all: sentry needs
`python-u2flib-server`, which has 15 source distributions and no wheels,
and marimo needs `lzstring`, whose wheels stop at 1.0.3. Both failed
with an empty version set that said nothing about why.
A release whose wheels the target accepts none of is now offered as the
source distribution the index serves beside it, ranked after every
wheel, so a wheel always wins and the archive is downloaded only where
nothing else installs. `.tar.gz` and `.zip` archives go through the
shared artifact pipeline as `ArchiveStoreProjection::RawArchive`, which
gives them the digest check, retries, store dedup and offline replay the
wheel path already has. Their files are then copied out of the CAS into
a temporary tree, so a backend that writes beside its sources is not
writing into content every project on the machine shares.
What the release requires is read from the wheel the build produces,
because a source distribution declares it nowhere else. The build is
therefore part of resolution rather than of installation:
`Step::NeedMetadata` dispatches on the candidate kind, and
`--lockfile-only` builds too. Because that wheel is the answer,
`finish_build` no longer holds every build to its directory's manifest.
`Contract::Interpreter` says the built wheel is the contract, which is
what lets a `setup.py`-only archive with dependencies build at all;
workspace projects and git checkouts keep `Contract::Manifest`.
Every archive URL goes through `validate_url` before it is fetched, at
the lockfile boundary and again before a build starts: the store reads a
`file:` URL from the filesystem, and a `pylock.toml` is untrusted input,
so without that check a lockfile could name an archive on this machine
and have its build backend run. The wheel path already refuses such a
URL outright.
A build runs the release's own code, so it is approved the way a git
dependency is, under `allowBuilds`. pnpm builds only with the
interpreter running the install, so a project that locks for several
environments and reaches a release none of them has a wheel for is
refused rather than locked from a wheel built for the wrong platform.
Where one environment does have a wheel and another does not, both land
on one PEP 751 package and each environment takes what it can use, which
is why `Solved::source_key` deliberately does not tell a wheel and a
source distribution of one release apart.
pnpr has no interpreter, so it offers source candidates but hands the
project back to the client as soon as a solution needs one built. The
client's pnpr fallback now recognizes any "must be resolved by the
client" answer rather than only the direct-URL one.
Separately, a failed resolution says which of its causes the empty
version set was: no index publishes the distribution, its releases
publish nothing this interpreter can install, or the project's own
requirements select none of the versions it offers. Reading an index
page therefore records why each release it left out was left out.
A source distribution is built again on every install rather than kept
as the wheel it produced; `plans/PYTHON_SPIKE.md` records that limit.
Related to pnpm/pnpm#14945.
Import unchanged Python wheel files using the existing reflink library and add
python.linkMode with copy, hardlink, and reflink modes. Default to copy-on-write
cloning with a copy fallback to preserve isolation from runtime writes.
Keep generated metadata and scripts requiring content or permission changes
independent. Reserve generated RECORD destinations before deferred imports so
wheel data cannot collide with installed metadata. Apply the setting to project
and isolated build environments.
Reuse verified wheel RECORD hashes for unchanged files, preserve executable
permissions for unpacked wheels, and cache copy fallback per source filesystem
within each environment. Fall back to private copies when restricted containers
deny cloning. Reuse the shared filesystem reflink capability in both importers.
Website documentation is in pnpm/pnpm.io#936.
Addresses file sharing from item 14 of pnpm/pnpm#14945. The issue remains open
for its other requests, including environment directory placement.
Separate lockfile dependency traversal from build activation for weak Cargo
dependency-feature references. Include the referenced optional dependency
and propagate its requested features recursively in the lockfile graph.
Keep unselected optional dependencies inactive.
Cover nested weak references across targets, compare generated lockfiles
with Cargo, and verify locked offline builds compile only active dependencies.
Add default and no-default uuid workspaces to the live equivalence corpus.
Isolate Cargo homes outside install workspaces from local user configuration.
The shared resolver also serves pnpr. Cargo support is Rust-only, so the
TypeScript pnpm v11 CLI is unaffected.
Closespnpm/pnpm#14978.
The Python participant resolved, locked and materialized every project in
the workspace, whatever `--filter` selected, and `pnpm add pypi:` refused a
filtered selection outright.
The participant now reads every discovered manifest once, so the projects
the selection leaves out still answer what a selected project's declared
sources need, and prepares only the selected ones. A project sharing a
directory with an npm workspace project follows that project's selection,
because the npm project's name is what the workspace knows the directory
by. A project in a directory of its own runs through the same selector
engine the npm side uses, against the distribution it declares, its path,
and the `[tool.uv.sources]` edges to its siblings.
An empty `--fail-if-no-match` selection is no longer raised while
resolving the npm projects: the install decides once the Python selection
is known, so a selector that names only a Python project is a match, and a
workspace whose projects are all Python is not reported as holding none.
Dynamic metadata is now prepared only for the projects the install reads,
and `pnpm add pypi:` names the directory it found no Python project in
instead of failing on a missing file.
Closes item 12 of pnpm/pnpm#14945
`read_file` snapshots a metadata path by opening it with `openat` and then
checking that what it opened is a regular file. Opening a FIFO read-only
blocks until a writer arrives, and the type check never runs, so a
`package.json`, `pnpm-lock.yaml` or `pyproject.toml` that is a FIFO hung
the command before preparation began.
Adding `O_NONBLOCK` makes the open return for a FIFO or a device, so the
type check rejects it with the error it already has for a path that is not
a regular file or a symlink. A regular file is unaffected: the flag does
not change the semantics of reading one.
The regression test captures a FIFO on a thread and waits on a channel, so
a reintroduction fails in 30 seconds with a clear message rather than
hanging until nextest's timeout.
A project whose `requires-python` no interpreter on the machine satisfied
was reported and left uninstalled, and a `.python-version` naming a version
the machine did not have was warned about. Four of the eight repositories in
pnpm/pnpm#14945 pin an interpreter that way, so the install stopped before
resolving anything.
pnpm now installs one. The builds are python-build-standalone's, which uv,
rye, hatch and mise install too, so the interpreter a Python developer
expects is the one that arrives. A release holds an interpreter of every
version line it supports, and its `SHA256SUMS` is both the list of what pnpm
can install and the digest every download is checked against; it is cached
for a day. The newest build the project accepts is the one installed, so a
range takes the newest release inside it and a pin takes the version it
names.
An interpreter lands under `<store>/python/`, shared by every project and
repository on the machine, and is found there afterwards like any other
interpreter: a later install uses it without reading the release at all, an
offline one included. It is unpacked beside where it belongs and moved
there, so a directory under `python` is one an install can use and never a
download that stopped halfway, and two installs racing for one build end
with the interpreter either of them unpacked.
`downloads: never` keeps pnpm from installing any, and an offline install
installs none. Both report the project, saying which of the two refused.
`downloadUrl` names a mirror of the releases.
Related to pnpm/pnpm#14945.
Prepare Python projects with bounded concurrency after selecting their
interpreters and workspace sources. Collect all results before reporting an
error, preserving discovery order for publication and error selection.
Share fresh registry resolutions within one install using normalized lockfile
inputs (including overrides, constraints, and extra indexes), requires-python,
and the interpreter report as the key. A per-key mutex
allows one resolution to populate the cache while identical projects wait.
Existing lockfiles retain their replay and frozen validation paths. Local
sources and direct URLs do not share fresh resolutions. Environment installs
seed and validate the shared lockfile through the existing acceptance path.
Make backend environment sharing safe for concurrent projects. Siblings wait
for one environment to finish; recursive builds track their own active keys
and reuse completed nested environments without waiting on other backend chains.
This preserves cycle diagnostics
and avoids dependency cycles between in-flight cache entries.
Add regression coverage for identical projects, frozen lockfile validation,
project dependency-rule isolation, concurrent slot replenishment, and settling failed preparation without
publishing lockfiles. Python installation exists only in pnpm v12.
Replace benchmark artifacts on reruns so the combined report and privileged
comment workflow consume current samples. Seed distinct whole-second manifest,
lockfile, and validation times in the synchronous fast-path test; equal
whole-second mtimes conservatively trigger a content check, so the prior
fixture could fail when its initial mtime landed on a second boundary.
Related to https://github.com/pnpm/pnpm/issues/14945 (item 13).
The repository's skills live in .agents/skills. Codex discovers that
directory on its own (verified with `codex debug prompt-input` on codex-cli
0.154.0, which lists a skill placed there). Claude Code does not: it scans
only .claude/skills, so the repo's skills were invisible to it unless each
contributor symlinked them into ~/.claude/skills by hand.
Add .claude/skills as a symlink to ../.agents/skills. Claude Code follows it
and lists the skills behind it, so the skills stay in one place and every
agent reads the same copy.
.claude was ignored wholesale, which would have kept the symlink untracked.
Narrow the rule so the root .claude directory and the skills symlink are
tracked while everything else under it, such as settings.local.json, stays
ignored.
No .codex/skills symlink: Codex already reads .agents/skills, so it would
only add a second path to the same files.
Add extra Python index URLs with ordered first-index selection, per-URL
credentials, cached 404 responses, and offline reuse. Do not merge versions
across indexes or fall back after authorization and other non-404 failures.
Apply PEP 508 overrides and constraints throughout Python dependency
resolution and locked graph validation. Inherit uv dependency rules from
the declared workspace root and combine them with pnpm workspace settings.
Persist sanitized index URLs and normalized dependency rules in lockfile
inputs so frozen installs reject changes to resolution settings.
Keep resolution local when the pnpr protocol cannot represent these inputs.
Keep project dependency rules separate from isolated build requirements.
Reuse the network crate’s authentication encoder alongside the shared HTTP,
archive-store, hashing, and filesystem infrastructure.
Related to pnpm/pnpm#14945, item 9.
`pnpm add pypi:<package>` built `<dir>/pyproject.toml` unconditionally and
handed it to `manifest::add`, which read it with a bare `.into_diagnostic()?`.
At a workspace root without a root `pyproject.toml` the whole diagnostic was
`No such file or directory (os error 2)`: no path, no hint at what was
missing.
`plan_add` now checks for the manifest before it builds the install task, the
way the `crate:` counterpart in `cargo_deps::add::plan` already does, and
reports the path it looked for along with a help line pointing at a directory
that has a `pyproject.toml`. Checking at plan time also keeps the failure
ahead of the interpreter and the lockfile, so nothing is written.
The check is `try_exists` rather than `is_file`, which answers false for
every manifest that is not a regular file and not just an absent one. A
`pyproject.toml` that is a directory, or one whose metadata cannot be read,
keeps the error the filesystem gave.
The two reads that could still surface an unlabeled OS error now carry
`read {path}` context, matching `read_project_manifests`.
Related to pnpm/pnpm#14945 (item 11).
The WHEEL file's Tag fields are informational. Installers select a wheel on
its filename, and a wheel whose filename tags were changed after the build
routinely keeps the tags it was built with, so requiring the two to agree
rejected valid published wheels such as mysql-connector-python. pip and uv
install them.
The filename's tags are still checked against the target interpreter before a
wheel is downloaded or a built wheel is accepted, so the inspect request no
longer needs the filename at all.
Related to pnpm/pnpm#14945 (item 10).
Resolve Python git and direct wheel requirements in the client rather than
silently replacing uv source declarations with index packages. Preserve
manifest version constraints, extras, markers, and workspace inheritance.
Pin git sources with PEP 751 vcs records and record wheel SHA-256 hashes.
Reuse the isolated PEP 517 builder, verified wheel CAS ingestion, git
transport policy, and atomic cache publication. Require approval before
running code from a git dependency or its build requirements. Preserve
submodules in cached checkouts and prevent recursive build dependencies
from deadlocking.
Keep source identity in wheel reuse and in lockfiles covering multiple
target environments. Record PEP 610 origin metadata for git and URL installs.
Let pnpr return a specific client-resolution response for transitive URLs.
Add CLI coverage for direct and uv sources, git revisions and subdirectories,
legacy projects, workspace inheritance, integrity and identity failures,
build approval, recursive builds, submodules, platform-specific sources,
backtracking between direct and index dependencies, requirements-file URL sources, and frozen offline replay. Document the supported forms and add a pacquet
minor changeset.
Related to pnpm/pnpm#14945, item 6.
Treat workspace Python groups and extras as defaults that apply only when
provided by each project. Add independent per-project overrides through
[tool.pnpm.python]. Empty overrides disable the corresponding default;
explicit unknown names still fail.
Share extra selection between static manifests and prepared dynamic metadata.
Normalize extra names using the existing Python requirement types. Preserve
strict included-group validation and existing production/development
projections. Compute validated production and development requirements once
per project for locking and borrow their selected slice for the environment.
Dependency requirements still select local distribution extras
independently of the source project's own installation settings.
Move selection and group expansion into a manifest module to keep the
manifest implementation within the repository's file-size convention.
Related to pnpm/pnpm#14945 (item 8).
Agents routinely opened a pull request, pushed, and ended the turn: CI
results and review rounds arrive minutes later and nothing in the repo
said the task was still open. AGENTS.md documented how to open a PR and
how to resolve a thread, but never that a push leaves work outstanding.
Add a `pull-requests` skill covering the path a change takes. Open the
PR as a draft, because CI runs on a draft here and the reviewers do not,
so a failing check or a bug the author's own pass catches costs one push
rather than a review round. Mark it ready once the checks are green and
that pass is clean.
From there, run a loop after every push: wait for the checks, read the
whole round including the findings a reviewer leaves only in its summary
comment, verify each one before acting, reply on every thread with the
commit that fixed it once that commit is on the remote branch, keep the
title and description current because they become the squash commit, and
push again. A round counts as finished only when every reviewer's
summary names the head commit, since a round that has not started looks
exactly like one that found nothing.
The skill also covers conflicts that appear while the PR waits, and why
the fix is a rebase: main requires linear history and has merge commits
disabled. It covers opening a PR for an issue, where the timeline
cross-reference says nothing about what was done, so the issue gets a
comment naming the approach and whether it closes the issue whole or in
part. And it covers the traps: rounds land on the previous round's
fixes, resolving a thread needs GraphQL, and a fork run cancelled by the
approval sweep is not a failure.
The skill names no reviewer. The roster changes; what an author does
with a finding does not.
Replace the PR bullets in AGENTS.md with a pointer to the skill and the
rules that have to hold even when it is not loaded: open as a draft, a
push is not the end of the task, and agent-authored content is signed.
Prepare dynamic metadata before constructing the Python dependency graph.
Use prepare_metadata_for_build_wheel where available and build a wheel as
the PEP 517 fallback. Retain fallback wheels for noneditable installation
with the same interpreter. Preserve hook-prepared metadata directories for noneditable builds. Validate
static dependency and interpreter declarations before resolution, and reject
final wheels whose dependencies, interpreter support, or extras differ from
the prepared metadata. Reuse the prepared manifests and workspace scopes when resolving sources.
Evaluate active platform markers and requested extras during source traversal.
Discover requirements.txt through the shared workspace inventory and treat
requirements-only directories as dependency projects without building the
directory itself or generating a pyproject.toml. Included files contribute
to the requirement inputs used for frozen-lockfile validation.
Backend requirements retain the existing allowBuilds policy. Metadata
preparation fails when approval is missing because resolution cannot
continue without running the backend. Dynamic metadata is currently
prepared for each project's selected interpreter and refreshed if dynamic
interpreter support changes that selection. External, undiscovered path sources
still require static metadata. Adding dependencies to dynamic metadata is
not part of this change.
Related to pnpm/pnpm#14945 (item 7).
`pnpm/CONTRIBUTING.md` required `just ready` before every commit, explicitly
including documentation and comment edits: a full `cargo nextest run` over
~11,500 tests for changes that cannot break them. The root guide's "never run
all tests" rule was scoped to the TypeScript sections, so it read as not
applying to Rust.
The checks now match what a change can break. Formatter, typos, and
workspace-wide check and lint before every commit, since those are cheap and
catch the cross-crate breakage a scoped selection hides, plus the tests for
what the diff affects. The full local run is reserved for changes whose blast
radius cannot be named, because CI already runs the suite on three platforms.
`just test-affected` resolves changed files to packages through `cargo
metadata` and runs them through `run-rust-tests.mjs`, expanding any `pnpr-*`
selection so feature unification does not silently skip backend tests, and
refusing to guess when a change reaches files every crate compiles against.
Selection is crate-level rather than `rdeps()`-based: `pnpm-cli` holds a third
of the workspace's tests and sits downstream of nearly everything, so `rdeps()`
selects 84% or more of the suite for any core crate. Restructuring that target
is tracked in pnpm/pnpm#14984.
Crate-level selection leaves the dependents unrun, so the new `smoke` profile
runs in their place: one end-to-end test per area of CLI behavior, triggered by
the dependent set rather than added to every run. Membership is by behavior
area rather than code coverage, since nearly every end-to-end test walks the
same install path. An exact-name filterset fails open, so a test checks every
entry against the suite sources.
The `testing-changes` skill carries the selection cookbook and the gotchas that
make a scoped run lie.
Related to pnpm/pnpm#14984.
Limit survivor ownership reads to command names that cleanup could actually remove. Snapshot replaced groups before activation and retain fail-closed scanning whenever an existing command is dropped, while allowing same-command global upgrades to proceed past unrelated incomplete groups.
Apply the same ownership rules to pnpm v11 and v12. Canonicalize the v11 hash-link target and parent before computing rollback links on macOS. Isolate package-manager provisioning tests from user-level Yarn path overrides.
Zoltan Kochan