mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-28 09:55:39 -04:00
f1521cfc8c
* feat(publish): add --batch flag to publish all packages in a single request
When publishing recursively, the new opt-in --batch flag packs every
selected package first and sends them all to the registry in one
"PUT /-/v1/multi-publish" request per target registry, instead of one
request per package. The endpoint is not part of the standard npm
registry API; registries that lack it are reported with
ERR_PNPM_MULTI_PUBLISH_UNSUPPORTED.
pnpr implements the endpoint with all-or-nothing semantics: every
document is validated (name, publish policy, attachment integrity) and
every tarball is fully written to a tmp slot before anything becomes
visible, so a batch that fails midway leaves no new versions behind.
The single-package publish handler was refactored into shared
validate/stage/commit steps, and package-lock stripes are acquired in
sorted order so overlapping batches cannot deadlock.
* refactor(publish): namespace the multi-publish endpoint under /-/pnpm/v1/
Vendor-namespace the batch publish endpoint as /-/pnpm/v1/multi-publish,
mirroring how the npm client keeps its registry extensions under
/-/npm/v1/. The unprefixed /-/v1/ namespace is effectively owned by the
npm registry API (login, search, tokens), so a future npm endpoint at
the same path could collide with different semantics. The pnpm prefix
makes the path unambiguously a pnpm-client extension while staying
server-implementation-neutral.
* refactor(publish): rename the batch endpoint to /-/pnpm/v1/publish
The endpoint is not inherently about multiple packages — a single
package is just a batch of one — so name it after the operation rather
than the cardinality. The unsupported-registry error is renamed to
ERR_PNPM_BATCH_PUBLISH_UNSUPPORTED to match the other BATCH_PUBLISH_*
error codes.
* feat(pnpr): crash-atomic publish commits via a journal
A publish (single-package or batch) is applied in several non-atomic
steps — one rename/upload per tarball, one packument write per package
— so a crash mid-apply could leave a batch partially published. Before
anything is promoted, the full intent (merged packument bytes + staged
tmp-file locations) is now persisted under .pnpr-journal/<txn>/ and
sealed with a single atomic rename of the commit marker. Startup
recovery rolls sealed transactions forward (every apply step is
idempotent, and the packument is re-merged into the current on-disk
state so versions published between a failed apply and the restart
survive) and rolls unsealed ones back, so a publish is either fully
visible or fully absent.
* test(pnpr): pin canonicalization of scoped dist.tarball URLs on serve
libnpmpublish-based clients submit dist.tarball with the scoped
filename (.../-/@scope/name-1.0.0.tgz). The registry never serves that
string verbatim: rewrite_dist_tarball rebuilds the URL from the
basename, so consumers always see the routable 4-segment form. The
batch publish scoped test now submits the real client wire form and
asserts the served packument exposes the canonical URL.
* test: align getConfig env-var warning test with request destination blocking
Project-level .npmrc request destinations (registry=, proxy keys) no
longer go through env expansion at all — they are dropped with a
dedicated warning — so registry=${VAR} can't produce the generic
'Failed to replace env in config' warning anymore. Exercise the
env-replace warning through cafile= (still expanded) and pin the
request-destination ignore warning in its own test.
* fix(pnpr): harden crash-atomic publish commit against partial states
Two reliability fixes from PR review:
- Journal recovery treated any I/O error probing the commit marker as
"unsealed" (`try_exists(...).unwrap_or(false)`), so a transient error
could roll back an already-committed transaction and delete its staged
tarballs. Propagate the error instead, so startup fails loudly rather
than risk losing a sealed publish.
- After sealing, the commit loop applied packages with `?`; a mid-loop
failure returned an error while earlier packages were already visible,
leaving a running server partially published until the next restart's
recovery. On apply failure, complete the sealed transaction immediately
via the same idempotent roll-forward, falling back to the original
error with startup recovery as the final backstop.
* fix(pnpr): fail recovery loudly when a staged tarball can't be probed
roll_forward treated any fs::try_exists error on a staged tarball as
"missing", so a transient I/O error would skip promotion, still write the
packument, and delete the journal entry — advertising a tarball with
nothing on disk and no journal state left to retry from. Propagate the
error instead, mirroring the commit-marker probe, so recovery aborts and
the transaction survives for a later attempt.
* test: fix env-var warning assertion in the cafile getConfig test
cafile=${ENV_VAR_123} asserted the registry request-destination warning,
which is only emitted for registry/proxy URLs; cafile is still
env-expanded, so an unresolved placeholder surfaces the generic
"Failed to replace env in config" warning. Assert that instead and
retitle the test (the request-destination case has its own test).
* fix(pnpr): serialize package deletes with the same package lock
delete_package and delete_tarball mutated package storage without taking
the per-package lock that every publish and dist-tag path holds, so a
same-package DELETE could race a stage-and-commit and remove the package
or a tarball mid-write, leaving the on-disk state dependent on filesystem
timing. Acquire the lock before the removal, completing the same-package
serialization guarantee.
Also bind the batch-publish test stub URL to 127.0.0.1 to match the
listener, avoiding intermittent IPv6 connection failures.
简体中文 | 日本語 | 한국어 | Italiano | Português Brasileiro
Fast, disk space efficient package manager:
- Fast. Up to 2x faster than the alternatives (see benchmark).
- Efficient. Files inside
node_modulesare linked from a single content-addressable storage. - Great for monorepos.
- Strict. A package can access only dependencies that are specified in its
package.json. - Deterministic. Has a lockfile called
pnpm-lock.yaml. - Works as a Node.js version manager. See pnpm runtime.
- Works everywhere. Supports Windows, Linux, and macOS.
- Battle-tested. Used in production by teams of all sizes since 2016.
- See the full feature comparison with npm and Yarn.
To quote the Rush team:
Microsoft uses pnpm in Rush repos with hundreds of projects and hundreds of PRs per day, and we’ve found it to be very fast and reliable.
Platinum Sponsors
|
|
Gold Sponsors
|
|
|
|
|
|
|
|
Silver Sponsors
|
|
|
|
|
|
|
|
⏱️ Time.now |
Support this project by becoming a sponsor.
Background
pnpm uses a content-addressable filesystem to store all files from all module directories on a disk. When using npm, if you have 100 projects using lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be stored in a content-addressable storage, so:
- If you depend on different versions of lodash, only the files that differ are added to the store.
If lodash has 100 files, and a new version has a change only in one of those files,
pnpm updatewill only add 1 new file to the storage. - All the files are saved in a single place on the disk. When packages are installed, their files are linked from that single place consuming no additional disk space. Linking is performed using either hard-links or reflinks (copy-on-write).
As a result, you save gigabytes of space on your disk and you have a lot faster installations!
If you'd like more details about the unique node_modules structure that pnpm creates and
why it works fine with the Node.js ecosystem, read this small article: Flat node_modules is not the only way.
💖 Like this project? Let people know with a tweet
Getting Started
Benchmark
pnpm is up to 2x faster than npm and Yarn classic. See all benchmarks here.
Benchmarks on an app with lots of dependencies:
License
MIT, except the pnpr/ directory, which is source-available under the PolyForm Shield License 1.0.0.