Zoltan Kochan
854138e9fd
Item 5 of pnpm/pnpm#14945. Two of the eight repositories surveyed there depend on a distribution that publishes no wheel at all: sentry needs `python-u2flib-server`, which has 15 source distributions and no wheels, and marimo needs `lzstring`, whose wheels stop at 1.0.3. Both failed with an empty version set that said nothing about why. A release whose wheels the target accepts none of is now offered as the source distribution the index serves beside it, ranked after every wheel, so a wheel always wins and the archive is downloaded only where nothing else installs. `.tar.gz` and `.zip` archives go through the shared artifact pipeline as `ArchiveStoreProjection::RawArchive`, which gives them the digest check, retries, store dedup and offline replay the wheel path already has. Their files are then copied out of the CAS into a temporary tree, so a backend that writes beside its sources is not writing into content every project on the machine shares. What the release requires is read from the wheel the build produces, because a source distribution declares it nowhere else. The build is therefore part of resolution rather than of installation: `Step::NeedMetadata` dispatches on the candidate kind, and `--lockfile-only` builds too. Because that wheel is the answer, `finish_build` no longer holds every build to its directory's manifest. `Contract::Interpreter` says the built wheel is the contract, which is what lets a `setup.py`-only archive with dependencies build at all; workspace projects and git checkouts keep `Contract::Manifest`. Every archive URL goes through `validate_url` before it is fetched, at the lockfile boundary and again before a build starts: the store reads a `file:` URL from the filesystem, and a `pylock.toml` is untrusted input, so without that check a lockfile could name an archive on this machine and have its build backend run. The wheel path already refuses such a URL outright. A build runs the release's own code, so it is approved the way a git dependency is, under `allowBuilds`. pnpm builds only with the interpreter running the install, so a project that locks for several environments and reaches a release none of them has a wheel for is refused rather than locked from a wheel built for the wrong platform. Where one environment does have a wheel and another does not, both land on one PEP 751 package and each environment takes what it can use, which is why `Solved::source_key` deliberately does not tell a wheel and a source distribution of one release apart. pnpr has no interpreter, so it offers source candidates but hands the project back to the client as soon as a solution needs one built. The client's pnpr fallback now recognizes any "must be resolved by the client" answer rather than only the direct-URL one. Separately, a failed resolution says which of its causes the empty version set was: no index publishes the distribution, its releases publish nothing this interpreter can install, or the project's own requirements select none of the versions it offers. Reading an index page therefore records why each release it left out was left out. A source distribution is built again on every install rather than kept as the wheel it produced; `plans/PYTHON_SPIKE.md` records that limit. Related to pnpm/pnpm#14945.
简体中文 | 日本語 | 한국어 | Italiano | Português Brasileiro
Fast, disk space efficient package manager:
- Fast. Up to 2x faster than the alternatives (see benchmark).
- Efficient. Files inside
node_modulesare linked from a single content-addressable storage. - Great for monorepos.
- Strict. A package can access only dependencies that are specified in its
package.json. - Deterministic. Has a lockfile called
pnpm-lock.yaml. - Works as a Node.js version manager. See pnpm runtime.
- Works everywhere. Supports Windows, Linux, and macOS.
- Battle-tested. Used in production by teams of all sizes since 2016.
- Experimental Rust port. Includes pacquet, an experimental port of the CLI written in Rust.
- See the full feature comparison with npm and Yarn.
To quote the Rush team:
Microsoft uses pnpm in Rush repos with hundreds of projects and hundreds of PRs per day, and we’ve found it to be very fast and reliable.
Platinum Sponsors
|
|
|
|
Gold Sponsors
|
|
|
|
|
|
|
|
Silver Sponsors
|
|
|
|
|
|
|
|
|
| ⏱️ Time.now |
Support this project by becoming a sponsor.
Background
pnpm uses a content-addressable filesystem to store all files from all module directories on a disk. When using npm, if you have 100 projects using lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be stored in a content-addressable storage, so:
- If you depend on different versions of lodash, only the files that differ are added to the store.
If lodash has 100 files, and a new version has a change only in one of those files,
pnpm updatewill only add 1 new file to the storage. - All the files are saved in a single place on the disk. When packages are installed, their files are linked from that single place consuming no additional disk space. Linking is performed using either hard-links or reflinks (copy-on-write).
As a result, you save gigabytes of space on your disk and you have a lot faster installations!
If you'd like more details about the unique node_modules structure that pnpm creates and
why it works fine with the Node.js ecosystem, read this small article: Flat node_modules is not the only way.
💖 Like this project? Let people know with a tweet
Getting Started
Benchmark
pnpm is up to 2x faster than npm and Yarn classic. See all benchmarks here.
Benchmarks on an app with lots of dependencies:
License
MIT, except the pnpr/ directory, which is source-available under the PolyForm Shield License 1.0.0.