mirror of
https://github.com/pnpm/pnpm.git
synced 2026-07-19 12:12:34 -04:00
159bd27ecb
* feat(pack): port the pnpm pack command to pacquet Port `pnpm pack` to Rust: create a reproducible gzipped tarball from a project, running the prepack / prepare / postpack lifecycle scripts, building the publish manifest, computing the file list, and writing the archive. - Extract npm-packlist into a shared `pacquet-fs-packlist` crate so the git and directory fetchers and the new pack command share one copy, matching pnpm's standalone `@pnpm/fs.packlist` package. - Extend `pacquet-exportable-manifest` with the full createExportableManifest: manifest obfuscation, workspace / catalog / jsr specifier rewriting, publishConfig hoisting, README embedding, and the bin / peerDependenciesMeta / repository transform. - Add the `pacquet-pack` crate (single-project api plus JSON / text output) and wire `pacquet pack` into the CLI, including the recursive (-r) workspace sweep in topological order. The dependency-converter composition is a straight function sequence rather than upstream's higher-order combineConverters, and the tarball write phase uses the capability-trait DI seam (a Host provider plus Fs* traits) so its filesystem error branches stay testable without process-global mutation. Ported from https://github.com/pnpm/pnpm/blob/cab1c11c69/releasing/commands/src/publish/pack.ts https://github.com/pnpm/pnpm/blob/ef87f3ccff/releasing/exportable-manifest/src/index.ts Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4k3hPXaosUkKBPbnChg4q * style(pack): rustfmt the pack test fixture Apply `cargo fmt` to the `fixture` helper's `package.json` write, which drifted after a manual edit. No behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4k3hPXaosUkKBPbnChg4q * style(exportable-manifest): satisfy perfectionist dylint lints - Add trailing commas to two multi-line macro invocations in the transform tests (perfectionist::macro_trailing_comma). - Replace a Unicode ellipsis with ASCII "..." in a create.rs doc comment (perfectionist::unicode_ellipsis_in_docs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4k3hPXaosUkKBPbnChg4q * style(pacquet): satisfy perfectionist single-letter and layout lints - Rename the reporter generic `R` to `Reporter` (bounded `self::Reporter`) in pack's `api` / `run_scripts_if_present` and the CLI `pack` command, matching the convention already used in executor, git-fetcher, and package-manager (perfectionist::single_letter_generic). - Rename `locale_compare_en`'s `a` / `b` parameters to `left` / `right` (perfectionist::single_letter_function_param). - Move the inline `transform` test module into an external `transform/tests.rs` (perfectionist::unit_test_file_layout). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4k3hPXaosUkKBPbnChg4q * test(pack): cover the prepack/prepare/postpack lifecycle path Port pnpm's `pack: runs prepack, prepare, and postpack` (releasing/commands/test/publish/pack.ts) so `run_scripts_if_present` and `script_body` are exercised. The existing pack tests all pass `ignore_scripts: true`, so that branch was never entered. - runs_prepack_prepare_and_postpack (Unix-gated, like the executor's other script-running tests): with scripts enabled, each of prepack / prepare / postpack runs and leaves a marker file. - pack_succeeds_when_scripts_enabled_but_absent: an empty `prepack` makes `script_body` yield `None` and `run_scripts_if_present` return early, so packing still succeeds without shelling out. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4k3hPXaosUkKBPbnChg4q * test(pack): port bundleDependencies, custom-directory, and peer/jsr tests Port the in-scope pnpm pack / createExportableManifest tests that close coverage holes in this PR's new code: - pack: bundleDependencies (array) and bundleDependencies: true cover the fs-packlist bundle recursion and the node_linker: hoisted allow-path. - pack: publishConfig.directory covers reading the manifest from the build-output dir while resolving workspace: deps against the project root's node_modules. - createExportableManifest: a peerDependencies workspace: rewrite and a jsr: specifier without a version selector. Ported from releasing/commands/test/publish/pack.ts and releasing/exportable-manifest/test/index.test.ts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4k3hPXaosUkKBPbnChg4q * docs(pack): pin pack.ts source citations to a valid commit The `cab1c11c69` SHA the crate docs referenced does not exist upstream. Repoint every `pack.ts` source link in `pack`'s module docs to `54c5c0e028`, where pnpm's releasing command now lives under `pnpm11/releasing/commands/...`, so the citations resolve. * test(pack): port the transitive bundleDependencies pack test Port pnpm's `pack: bundles transitive dependencies of bundled dependencies (hoisted)` test. `top` is the only directly-bundled package, but it depends on `nested`, which is hoisted to the root `node_modules`; both must end up in the tarball. This exercises `fs-packlist`'s transitive bundled-closure walk that landed via pnpm/pnpm#12620 (resolving pnpm/pnpm#12602) and arrived on this branch through the latest merge from `main`. Also repoint the existing pack-test citations to the valid `54c5c0e028` SHA, matching the docs fix in the surrounding crate. * docs(cli): document the pacquet pack command Add a `pacquet pack` section to the CLI README listing its flag support (`--dry-run`, `--pack-destination`, `--json`, `--out`, `--pack-gzip-level`, `--skip-manifest-obfuscation`, recursive mode), so the new command ships with the same flag-status table the other commands carry. Also pin the last `pack.ts` citation in `cli_args/pack` to the valid `54c5c0e028` SHA, matching the rest of the crate. * test(pack): assert the empty-prepack guard skips, not just succeeds Mutation-testing the pack tests surfaced that `pack_succeeds_when_scripts_enabled_but_absent` did not actually guard the behavior its name and doc claim. It asserted only that packing succeeds with an empty `prepack` script — but an empty lifecycle script runs as a successful no-op, so removing `script_body`'s `.filter(|s| !s.is_empty())` left the test green while the empty script was now executed. Record the emitted log events with the crate's `RecordingReporter` idiom and assert no `pnpm:lifecycle` event fires, pinning the "returns early without shelling out" half of the contract. With the empty-string filter removed the empty `prepack` shells out and emits a lifecycle event, so the test now fails — making it a real regression guard for `script_body`. * fix(pack): align manifest validation with pnpm and harden license injection Address review feedback on the pack port: - Validate `--pack-gzip-level` against 0-9 at the CLI boundary. - Skip recursive-pack targets whose name/version are empty strings, so they match the pacquet_pack::api precondition. - Stop coercing a non-object peerDependencies into an empty object in create_exportable_manifest; leave it in place like the other dep fields. - Gate transform_required_fields on JavaScript truthiness to match upstream's `if (!manifest.name)` check instead of requiring a non-empty string. - Gate the bundledDependencies guard on JS truthiness so an explicit `false` packs cleanly under the default non-hoisted linker. - Only inject a workspace LICENSE when the entry resolves to a regular file, mirroring fast-glob's onlyFiles default and avoiding an "Is a directory" failure in the later read/size pass. * fix(pack): skip symlinked workspace LICENSE to prevent file leakage A symlinked workspace-root LICENSE was followed via fs::metadata, so a symlink pointing outside the workspace could leak its target's bytes into a sub-package's tarball. Use DirEntry::file_type (which does not follow symlinks) so only a regular file is injected, mirroring the symlink-skipping read_readme_file in exportable-manifest. Add a unix regression test. Also pin the empty-string version case in the required-field truthiness test. * fix(pack): write the tarball atomically to avoid symlink-redirected overwrite Host::write used std::fs::write, which follows a symlink at the output path and truncates in place. A repo-controlled symlink planted at the tarball's output path could therefore redirect the write to clobber an arbitrary file, and a crash mid-write could leave a truncated .tgz. Write a sibling temp file, fsync it, then rename it over the target. The rename replaces a symlink instead of following it and is atomic, matching the write-file-atomic pattern pacquet-package-manifest already uses for package.json. The mode is preserved on overwrite (else 0o644) so the archive is not left owner-only. Add a unix regression test. * fix(pack): reject path-traversal in version and main/bin packlist fields Two manifest-controlled path-escape vectors, since manifests are attacker-controlled per REVIEW_GUIDE: - pacquet-pack interpolated the version into the default tarball name (<name>-<version>.tgz) after only a non-empty check, so a version containing a path separator could write the tarball outside the destination directory. Reject a version with a / or \. - pacquet-fs-packlist force-included main/bin after only stripping a leading ./ or /, so ../secret (or a Windows drive/UNC path) escaped the package root and was read into the tarball/CAS — the packlist also feeds the git fetcher on untrusted packages. Require the normalized field to stay inside the package (only normal components, no backslash) and to be a regular file via symlink_metadata so a symlinked main/bin can't leak an out-of-tree target. Add regression tests for both. * fix(pack): reject --out with --pack-destination in recursive mode Recursive pack resolved a shared destination before `api` ran, so the `--out` branch silently dropped `pack_destination` and the mutually- exclusive pair was accepted — unlike single-project pack, which rejects it with ERR_PNPM_INVALID_OPTION. Validate the pair up front in the recursive path so both modes behave the same. * fix(pack): stream the tarball, honor unsafe_perm, and harden output Address the remaining Qodo findings on the pack port: - Stream the gzip+tar archive into the atomic temp file instead of buffering the whole archive in a Vec, so peak memory no longer scales with the packed size. Per-file reads stay in memory, matching pnpm's readFileSync. The FsWrite seam becomes FsAtomicWrite, taking a streaming write body. - Thread Config.unsafe_perm into PackOptions instead of hardcoding unsafe_perm = true for prepack/prepare/postpack, so packing honors the same policy (and TMPDIR isolation) as pnpm. - Strip control characters from the text pack output, so a filename with raw ANSI escapes can't spoof the terminal. JSON output is left as data. - Reject an --out value that resolves to no file name (., .., "") up front instead of failing later with a confusing OS write error. - Sort the contents list with one cached lowercase key per item instead of recomputing to_lowercase on both sides per comparison, and check bin executability against a HashSet instead of a linear scan. Add regression tests for the --out and control-character cases. * fix(pack): canonicalize main/bin containment and harden tarball mode Two follow-ups from review: - The main/bin force-include verified containment only on the path string, so an intermediate symlinked directory (subdir -> /outside) could still resolve a host file into the packlist. Verify the canonical resolved path is strictly inside the canonical package root instead, which also follows symlinks that stay in-package (matching npm-packlist) while failing closed on any escape. - The atomic tarball write read the destination's mode with metadata (follows symlinks), so a symlink at the output path could donate its target's or a directory's mode to the archive. Use symlink_metadata and only inherit from an existing regular file, and set the mode before syncing so content and metadata flush together. * test(package-manager): adapt patch-commit escape test to packlist filtering Hardening the shared packlist to drop `..`-escaping main/bin fields means patch-commit's `prepare_pkg_files_for_diff` no longer receives an escaping entry, so the old integration test's `expect_err` no longer fires. Unit-test patch-commit's own `safe_package_file_path` guard directly (defense in depth), and rework the integration test to assert the escaping `main` is filtered and never materialized into the diff view. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Zoltan Kochan <z@kochan.io>
简体中文 | 日本語 | 한국어 | Italiano | Português Brasileiro
Fast, disk space efficient package manager:
- Fast. Up to 2x faster than the alternatives (see benchmark).
- Efficient. Files inside
node_modulesare linked from a single content-addressable storage. - Great for monorepos.
- Strict. A package can access only dependencies that are specified in its
package.json. - Deterministic. Has a lockfile called
pnpm-lock.yaml. - Works as a Node.js version manager. See pnpm runtime.
- Works everywhere. Supports Windows, Linux, and macOS.
- Battle-tested. Used in production by teams of all sizes since 2016.
- Experimental Rust port. Includes pacquet, an experimental port of the CLI written in Rust.
- See the full feature comparison with npm and Yarn.
To quote the Rush team:
Microsoft uses pnpm in Rush repos with hundreds of projects and hundreds of PRs per day, and we’ve found it to be very fast and reliable.
Platinum Sponsors
|
|
Gold Sponsors
|
|
|
|
|
|
|
|
Silver Sponsors
|
|
|
|
|
|
|
|
|
| ⏱️ Time.now |
Support this project by becoming a sponsor.
Background
pnpm uses a content-addressable filesystem to store all files from all module directories on a disk. When using npm, if you have 100 projects using lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be stored in a content-addressable storage, so:
- If you depend on different versions of lodash, only the files that differ are added to the store.
If lodash has 100 files, and a new version has a change only in one of those files,
pnpm updatewill only add 1 new file to the storage. - All the files are saved in a single place on the disk. When packages are installed, their files are linked from that single place consuming no additional disk space. Linking is performed using either hard-links or reflinks (copy-on-write).
As a result, you save gigabytes of space on your disk and you have a lot faster installations!
If you'd like more details about the unique node_modules structure that pnpm creates and
why it works fine with the Node.js ecosystem, read this small article: Flat node_modules is not the only way.
💖 Like this project? Let people know with a tweet
Getting Started
Benchmark
pnpm is up to 2x faster than npm and Yarn classic. See all benchmarks here.
Benchmarks on an app with lots of dependencies:
License
MIT, except the pnpr/ directory, which is source-available under the PolyForm Shield License 1.0.0.