Zoltan Kochan
6cb50b4fe5
The pick-and-fetch path was calling synchronous `load_meta` and `load_meta_headers` directly from async tasks. Each call reads the mirror file (`fs::read_to_string`, a blocking syscall) and parses the multi-KB to multi-MB packument body (`serde_json::from_str`, CPU-bound). Neither yields, so the tokio worker stayed pinned for the duration of every disk hit. The hot path runs them on every cache-warm pick: - `fetch_full_metadata_cached` calls `load_meta_headers` to feed the conditional GET headers. - A 304 response then calls `load_meta` to re-materialise the packument. - `pick_package`'s offline / pickLowestVersion, version-spec, and publishedBy shortcuts also call `load_meta` directly. With ~600 unique packuments per install all running through this sequence concurrently behind `try_join_all`, the blocking time serializes against the worker pool size and the async scheduler can't make progress on unrelated resolves / HTTP fetches in the gaps. This change adds async siblings — `load_meta_async` and `load_meta_headers_async` — that dispatch the sync body to `tokio::task::spawn_blocking`. Callers in `pick_package` and `fetch_full_metadata_cached` await the async wrappers. The sync functions stay (tests and the writeback path still call them directly). Matches upstream pnpm's posture, which performs the equivalent work via awaited `fs.readFile` + `JSON.parse` on libuv's worker pool. Test: all 194 tests in `pacquet-resolving-npm-resolver` continue to pass (existing `load_meta` / `load_meta_headers` coverage in the mirror tests and via the fetcher integration tests is the behaviour-level proof — the wrappers delegate verbatim).
简体中文 | 日本語 | 한국어 | Italiano | Português Brasileiro
Fast, disk space efficient package manager:
- Fast. Up to 2x faster than the alternatives (see benchmark).
- Efficient. Files inside
node_modulesare linked from a single content-addressable storage. - Great for monorepos.
- Strict. A package can access only dependencies that are specified in its
package.json. - Deterministic. Has a lockfile called
pnpm-lock.yaml. - Works as a Node.js version manager. See pnpm runtime.
- Works everywhere. Supports Windows, Linux, and macOS.
- Battle-tested. Used in production by teams of all sizes since 2016.
- See the full feature comparison with npm and Yarn.
To quote the Rush team:
Microsoft uses pnpm in Rush repos with hundreds of projects and hundreds of PRs per day, and we’ve found it to be very fast and reliable.
Platinum Sponsors
|
Gold Sponsors
|
|
|
|
|
|
|
|
Silver Sponsors
|
|
|
|
|
|
|
⏱️ Time.now |
Support this project by becoming a sponsor.
Background
pnpm uses a content-addressable filesystem to store all files from all module directories on a disk. When using npm, if you have 100 projects using lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be stored in a content-addressable storage, so:
- If you depend on different versions of lodash, only the files that differ are added to the store.
If lodash has 100 files, and a new version has a change only in one of those files,
pnpm updatewill only add 1 new file to the storage. - All the files are saved in a single place on the disk. When packages are installed, their files are linked from that single place consuming no additional disk space. Linking is performed using either hard-links or reflinks (copy-on-write).
As a result, you save gigabytes of space on your disk and you have a lot faster installations!
If you'd like more details about the unique node_modules structure that pnpm creates and
why it works fine with the Node.js ecosystem, read this small article: Flat node_modules is not the only way.
💖 Like this project? Let people know with a tweet
Getting Started
Benchmark
pnpm is up to 2x faster than npm and Yarn classic. See all benchmarks here.
Benchmarks on an app with lots of dependencies:
