Commit Graph

38 Commits

Author SHA1 Message Date
Zoltan Kochan
60fd20536d fix: pin integrity of git-hosted tarballs in lockfile (#11481)
For git-hosted tarballs (`codeload.github.com` / `gitlab.com` / `bitbucket.org`) the fetcher dropped the integrity it computed while downloading, so the lockfile only ever stored the URL. A compromised git host or man-in-the-middle could serve a substituted tarball on subsequent installs and pnpm would install it — the lockfile had no hash to compare against.

This pins the SHA-512 SRI of the raw tarball in the lockfile, in the same `sha512-<base64>` form npm-registry tarballs use. The only difference is the source: for npm we pass through `dist.integrity`, for git we compute it locally from the downloaded buffer. Subsequent installs validate the download against that integrity in the worker (`addTarballToStore` → `parseIntegrity` → hash compare), so a tampered tarball fails with `TarballIntegrityError`.

## Why git-hosted stays on `gitHostedStoreIndexKey`

The lockfile pins integrity for security, but the *store key* for git-hosted resolutions stays on `gitHostedStoreIndexKey(pkgId, { built })` rather than collapsing under the integrity-based key. Reason: git-hosted tarballs are post-processed (`preparePackage` / `packlist`), so the cached file set depends on whether build scripts ran during fetch. The integrity-only key would fold the built and not-built variants into a single slot, letting one overwrite the other and serving the wrong content if `ignoreScripts` was toggled between runs. Keeping git-hosted on the existing key shape preserves that dimension; the integrity is still validated on every fresh download.

## How the routing stays clean

The naive way to express "use gitHostedStoreIndexKey for git-hosted, integrity key for npm" is to call `isGitHostedPkgUrl(resolution.tarball)` everywhere a store key is computed — fragile, scattered, and easy to forget when adding new readers (Copilot caught two of those during review). Instead, a typed annotation: `TarballResolution` gets an optional `gitHosted: boolean` field. The git resolver sets it; the lockfile loader (`convertToLockfileObject`) backfills it for entries written by older pnpm versions; `toLockfileResolution` carries it through on serialize. Every consumer reads `resolution.gitHosted` directly. URL detection lives in exactly two places — the resolver and the loader — instead of seven.

## Changes

### Security fix
- `fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts` — return the `integrity` that the inner remote-tarball fetch already computed (was being silently dropped by the destructure).

### Lockfile schema (additive)
- `@pnpm/lockfile.types` and `@pnpm/resolving.resolver-base` — `TarballResolution` gains optional `gitHosted: boolean`.
- `@pnpm/resolving.git-resolver` — sets `gitHosted: true` on every git-hosted tarball it produces.
- `@pnpm/lockfile.fs` (`convertToLockfileObject`) — backfills the field on load for older lockfiles via inlined URL detection.
- `@pnpm/lockfile.utils` (`toLockfileResolution`, `pkgSnapshotToResolution`) — preserve / read the field.

### Store-key consumers (now one-line typed reads, dropped the URL-sniffing dep)
- `installing/package-requester` (`getFilesIndexFilePath`)
- `store/pkg-finder` (`readPackageFileMap`)
- `modules-mounter/daemon` (`createFuseHandlers`)
- `building/after-install` (side-effects-cache lookup + write)
- `store/commands/storeStatus`
- `installing/deps-installer` (agent-mode store-controller wrapper)

### Fetcher routing
- `fetching/pick-fetcher` — `pickFetcher` prefers `resolution.gitHosted`; URL fallback retained for ad-hoc resolutions.

### Tests
- New integrity-validation test in `tarball-fetcher` (mismatched `integrity` on the resolution must throw `TarballIntegrityError`).
- New git-hosted lookup test in `pkg-finder` asserting routing through `gitHostedStoreIndexKey` even when integrity is present.
- New `toLockfileResolution` test asserting `gitHosted: true` flows through serialization.
- `fromRepo.ts` lockfile snapshot updated for the now-pinned integrity + `gitHosted: true`.
- `git-resolver` tests updated to assert `gitHosted: true` in produced resolutions.
2026-05-06 13:22:25 +02:00
Zoltan Kochan
72c1e050e9 feat: add pnpm pack-app command for packing CJS entries into standalone executables (#11312)
* fix: give each runtime variant its own global virtual store entry

When a runtime package (e.g. node@runtime:X.Y.Z) uses a variations
resolution, createFullPkgId() in @pnpm/deps.graph-hasher was hashing
the whole VariationsResolution — the same hash on every host — so the
global virtual store path collided between variants. Whichever variant
installed first won, and a later `pnpm add --libc=musl node@runtime:<v>`
silently reused the cached glibc (or macOS/Windows) binary.

The fix threads supportedArchitectures down to createFullPkgId so the
selected variant's integrity is used as the package fingerprint. Two
related cleanups land with it:

- Extract the platform-variant selection logic to @pnpm/resolving.resolver-base
  as selectPlatformVariant/resolvePlatformSelector. The helper's libc
  match also required a fix: a variant with no libc is the "default"
  build, and a request for a non-default libc (e.g. musl) must require
  an exact match so the default variant doesn't silently win.
- @pnpm/installing.package-requester's findResolution now delegates to
  the shared helper, and the new supportedArchitectures param is plumbed
  through calcDepState / calcGraphNodeHash / iterateHashedGraphNodes /
  lockfileToDepGraph and their callers in deps-resolver, deps-restorer,
  deps-installer, graph-builder, and building.after-install.

* feat: add pnpm build-sea command for building Node.js SEA executables

Adds `pnpm build-sea` under @pnpm/releasing.commands. Takes a CommonJS
entry file and a set of target triplets (linux-x64, linux-x64-musl,
linux-arm64, linux-arm64-musl, macos-x64, macos-arm64, win-x64,
win-arm64) and produces a standalone executable per target under
dist-sea/<target>/.

Each target's Node.js runtime is fetched via `pnpm add node@runtime:<v>
--os=<os> --cpu=<arch> --libc=<libc>` into $PNPM_HOME/build-sea/<target>-<v>/
so binaries are hardlinked from the global content-addressable store and
`pnpm store prune` can reclaim them.

Requires Node.js v25.5+ to perform the --build-sea injection. If the
running Node is older, a v25 binary is downloaded and used as the builder
automatically. macOS outputs are ad-hoc signed with codesign (on macOS)
or ldid (when cross-compiling from Linux), which is required because SEA
injection invalidates the binary's existing signature.

* fix(build-sea): reject malformed --target, --output-name and use mkdtemp for config

Addresses Copilot review feedback on the build-sea command:

- parseTarget() previously destructured the target string, silently
  accepting extra `-` segments. Inputs like `linux-x64-musl-../../outside`
  would pass validation and flow into path.join. Validation is now done
  with a strict anchored regex.
- --output-name was passed into path.join() without sanitization, so a
  caller could escape the output directory with path separators or `..`.
  validateOutputName() now rejects anything that isn't a plain basename.
- The per-target SEA config file was written to a predictable path under
  os.tmpdir() (derived from the target name and Date.now()), which is
  unsafe on multi-user systems. It now lives inside a fresh mkdtemp()
  directory and is opened with the exclusive "wx" flag.
- New test cases cover extra-segment targets, uppercase/whitespace
  variants, and the full matrix of invalid --output-name inputs.

* rename: build-sea → pack-app

`build-sea` required knowing what a SEA is. `pack-app` is self-describing,
doesn't collide with pnpm's existing `bin` concept, and parallels the
existing `pack` command.

- Command name: build-sea → pack-app
- Default output dir: dist-sea → dist-app
- Error codes: PACK_APP_* (was BUILD_SEA_*)
- Export/type: packApp / PackAppOptions (was buildSea / BuildSeaOptions)
- Install cache dir: $PNPM_HOME/pack-app (was $PNPM_HOME/build-sea)

The Node.js `--build-sea` flag name itself is unchanged — that's a
Node.js feature and outside this project's naming.

* fix(pack-app): reject directory entries, pin builder to >=25.5, refuse macOS target on Windows

Addresses Copilot review feedback on the pack-app command:

- entry validation now rejects non-file paths (directories, symlinks to
  non-files) with a dedicated PACK_APP_ENTRY_NOT_FILE instead of
  surfacing a less actionable error later in the SEA build.
- DEFAULT_BUILDER_SPEC was the bare major ("25"), which would satisfy
  with 25.0.x if that version is still present — those point releases
  predate --build-sea support. Tightened to ">=25.5.0 <26.0.0" so the
  download is guaranteed to support the flag without ever crossing a
  major.
- adHocSignMacBinary() silently skipped re-signing on Windows hosts.
  Now throws PACK_APP_MACOS_SIGN_UNSUPPORTED_HOST with a hint to build
  the target on macOS/Linux or re-sign manually.
- resolvePlatformSelector() JSDoc now matches what the code actually
  does (picks the first entry when it is not "current"; later entries
  are ignored).
- New test case covers the directory-as-entry rejection.

* refactor(pack-app): switch target OS names to process.platform constants

Previously `pack-app` accepted `macos-*` / `win-*` as the OS portion of a
target triplet and translated them to `darwin` / `win32` internally. The
translation layer made the CLI surface inconsistent with the values that
`pnpm add --os=…` and `supportedArchitectures.os` already use, and added
a small footgun (e.g. users setting `supportedArchitectures: { os: [darwin] }`
but typing `macos-arm64` for pack-app).

The supported target OS set is now `linux | darwin | win32`, matching
`process.platform`. Old inputs like `macos-arm64` or `win-x64` now fail
validation with a clear error pointing to the new naming. The internal
parseTarget helper drops its TARGET_OS_MAP lookup entirely.

This is a change to an unreleased command so there is no back-compat
concern. pnpm's own artifact directory names (`pnpm/artifacts/macos-*/`,
`pnpm/artifacts/win-*/`) are an internal implementation detail and are
not affected by this change.

* feat(pack-app): read defaults from pnpm.app in package.json

Every pack-app flag (--entry, --target, --node-version, --output-dir,
--output-name) can now be preconfigured in the project's package.json
under a new "pnpm.app" object:

  {
    "name": "my-cli",
    "pnpm": {
      "app": {
        "entry": "dist/index.cjs",
        "targets": ["linux-x64", "darwin-arm64", "win32-x64"],
        "nodeVersion": "25",
        "outputDir": "release",
        "outputName": "my-cli"
      }
    }
  }

CLI flags always win. --target replaces the configured list rather than
appending, so a user can narrow the default set at the command line.

The config loader is strict: unknown keys under pnpm.app and any
type-mismatched values throw PACK_APP_INVALID_CONFIG so mistakes surface
at invocation time instead of silently being ignored.

Chose pnpm.app over pnpm.packApp because it's the shorter, cleaner
namespace for anything related to the app bundle (future sibling
commands like run-app / deploy-app could share the same object without
a naming clash). Chose package.json over pnpm-workspace.yaml because
the config is inherently per-project, whereas pnpm-workspace.yaml is
workspace-root-only.

* fix(pack-app): deterministic libc selection and stricter output-name validation

Addresses Copilot review feedback:

- ensureNodeRuntime() now always passes an explicit --libc for linux
  targets. Without a suffix, linux-x64 and linux-arm64 default to
  --libc=glibc instead of letting the user's supportedArchitectures.libc
  config or the host's detected libc decide the variant. The install
  cache directory mirrors this, so glibc and musl variants are always
  distinct (linux-x64-glibc vs linux-x64-musl).
- resolveBuilderBinary() now pins the host libc when downloading a
  builder Node on Linux. A user whose config sets supportedArchitectures.libc
  to musl no longer ends up with a musl Node that the glibc host cannot
  execute.
- validateOutputName() rejects Windows-invalid filename characters
  (<>:"|?* and NUL), Windows reserved device names (CON, NUL, COM1, etc.),
  and names ending in a dot or space — problems surface at invocation
  time rather than during writeFile(outputFile, ...) on Windows.
- lockfileToDepGraph variants tests no longer derive the "host"
  variant from process.platform/process.arch; they always pass an
  explicit supportedArchitectures selector so the expectations hold on
  any CI host (including Alpine/musl).

* chore: add "toctou" to cspell wordlist

`TOCTOU` (time-of-check-to-time-of-use) is the standard term for the
race-condition class the pack-app SEA-config comment describes. Adding
it to the wordlist unblocks the Lint CI step.

* fix: lint
2026-04-20 14:29:49 +02:00
Brandon Cheng
9b0a460c8d fix: non-deterministic resolution causing pnpm dedupe --check to unexpectedly fail (#11110)
* test: ensure prerelease weighting is correct

* fix: use higher weight for package versions already in lockfile

* test: remove fundamentally incompatible test

* fix(test): use undici MockAgent instead of nock for HTTP mocking

nock only patches Node's built-in http/https modules, but pnpm uses
undici for HTTP requests. Replace nock with @pnpm/testing.mock-agent
(which wraps undici's MockAgent) so the regression test actually
intercepts registry metadata requests.

* fix(benchmarks): show errors from store populate step

The populate step redirected both stdout and stderr to /dev/null,
hiding the actual error when pnpm install fails during benchmarks.

* fix(benchmarks): replace deprecated packages in benchmark fixture

The old fixture used deprecated babel 6, gulp, and other legacy
packages whose transitive dependencies (e.g. es-abstract) are missing
the "time" field in registry metadata, causing ERR_PNPM_MISSING_TIME
with time-based resolution mode.

Replace with modern equivalents (babel 7, webpack 5, MUI, Redux
Toolkit, etc.) that maintain a similar dependency tree size (~1300
packages) while using well-maintained packages with proper registry
metadata.

* fix(benchmarks): drop eslint plugins that pull in es-abstract

eslint-plugin-react, eslint-plugin-import, and eslint-plugin-jsx-a11y
transitively depend on es-abstract, whose registry metadata lacks the
"time" field. Replace them with eslint-plugin-prettier to avoid
ERR_PNPM_MISSING_TIME with time-based resolution.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-04-07 23:46:27 +02:00
Zoltan Kochan
1c8c4e49f5 style: add eslint-plugin-simple-import-sort (#10947)
Add eslint-plugin-simple-import-sort to enforce consistent import ordering:
- Node.js builtins first
- External packages second
- Relative imports last
- Named imports sorted alphabetically within each statement
2026-03-13 02:02:38 +01:00
Brandon Cheng
01914345d5 build: enable @typescript-eslint/no-import-type-side-effects (#10630)
* build: enable `@typescript-eslint/no-import-type-side-effects`

* build: disable `@typescript-eslint/consistent-type-imports`

* chore: apply fixes for `no-import-type-side-effects`

pnpm exec eslint "**/src/**/*.ts" "**/test/**/*.ts" --fix
2026-03-08 00:02:48 +01:00
Zoltan Kochan
50fbecae7d refactor(env): pnpm env use now delegates to pnpm add --global (#10666)
This PR overhauls `pnpm env` use to route through pnpm's own install machinery instead of maintaining a parallel code path with manual symlink/shim/hardlink logic.

```
pnpm env use -g <version>
```

now runs:

```
pnpm add --global node@runtime:<version>
```

via `@pnpm/exec.pnpm-cli-runner`. All manual symlink, hardlink, and cmd-shim code in `envUse.ts` is gone (~1000 lines removed across the package).

### Changes

**npm and npx shims on all platforms**

Added `getNodeBinsForCurrentOS(platform)` to `@pnpm/constants`, returning a `Record<string, string>` with the correct relative paths for `node`, `npm`, and `npx` inside a Node.js distribution. `BinaryResolution.bin` is widened from `string` to `string | Record<string, string>` in `@pnpm/resolver-base` and `@pnpm/lockfile.types`, so the node resolver can set all three entries and pnpm's bin-linker creates shims for each automatically.

**Windows npm/npx fix**

`addFilesFromDir` was skipping root-level `node_modules/` (to avoid storing a package's own dependencies), which stripped the bundled `npm` from Node.js Windows zip archives. Added an `includeNodeModules` option and enabled it from the binary fetcher so Windows distributions keep their full contents.

**Removed subcommands**

`pnpm env add` and `pnpm env remove` are removed. `pnpm env use` handles both installing and activating a version. `pnpm env list` now always lists remote versions (the `--remote` flag is no longer required, though it is kept for backwards compatibility).

**musl support**

On Alpine Linux and other musl-based systems, the musl variant of Node.js is automatically downloaded from [unofficial-builds.nodejs.org](https://unofficial-builds.nodejs.org).
2026-02-22 12:06:34 +01:00
3w36zj6
bb8baa7cff fix(npm-resolver): request full metadata for optional dependencies (#10455)
close #9950

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-01-26 01:13:06 +01:00
Trevor Burnham
88263a8be7 refactor: force re-fetch when resolution integrity changes (#10454)
* fix: force re-fetch when resolution integrity changes

When a resolver returns a resolution with a different integrity than
the current package's resolution, automatically force re-fetching the
package. This allows custom resolvers to trigger re-fetches by simply
returning the updated integrity, without needing to explicitly set
a forceFetch flag.

Closes #10451

* refactor: remove forceFetch

* test: fix

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-01-20 01:57:16 +01:00
Zoltan Kochan
0bcbaf9994 refactor: move out skip resolution logic from package requester (#10439) 2026-01-12 13:08:50 +01:00
btea
facdd717bf feat: add trustPolicyIgnoreAfter (#10359)
* feat: add `trustPolicyIgnoreAfter`

* Update .changeset/big-lies-pump.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor: npm-resolver

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-12-28 02:01:09 +01:00
Trevor Burnham
38b8e357b5 feat: add custom resolvers and fetchers (#10246) 2025-11-30 14:19:04 +01:00
Ryo Matsukawa
9d3f00b09a feat: add support for trustPolicyExclude (#10168)
close #10164
2025-11-11 13:00:20 +01:00
Ryo Matsukawa
10bc39152e feat: add support for npm package trust evidence check via a new trustPolicy setting (#10103)
close #8889

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-11-09 23:23:58 +01:00
Ryo Matsukawa
7c1382f7b7 feat: add support for exact versions in minimumReleaseAgeExclude (#10059)
close #9985

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-10-18 11:10:08 +02:00
Zoltan Kochan
86b33e91ea feat: support installing Bun runtime (#9815)
* feat: support installing Bun runtime

* feat: support installing Bun runtime

* fix: cache libc resolution

* refactor: shasum file fetching

* docs: add changesets

* feat: installing the right artifact

* test: supported architectures

* test: fix on Windows
2025-07-31 13:46:13 +02:00
Zoltan Kochan
d1edf732ad feat: support installing Deno runtime (#9791)
* feat: support installing Deno runtime

* refactor: use npm registry to resolve deno version

* feat: wip

* feat: installing deno runtime

* style: fix

* test: fix

* test: deno

* test: fix

* feat: deno

* feat: deno

* feat: create zip fetcher

* style: fix

* refactor: node fetch

* feat: support a new binary fetcher

* test: fix

* feat: rename zip-fetcher to binary-fetcher

* refactor: change naming

* fix: windows

* refactor: rename packages

* refactor: deno resolver

* refactor: runtime resolvers

* refactor: binary fetcher

* refactor: runtime resolvers

* refactor: runtime resolvers

* refactor: create SingleResolution

* refactor: remove not needed change

* refactor: package requester

* docs: add changesets

* refactor: use VariationsResolution and AtomicResolution

* refactor: implement CR suggestions

* docs: add changesets

* fix: address comment in CR

* feat: update formatting of pnpm-lock.yaml
2025-07-30 11:27:07 +02:00
Zoltan Kochan
f91922c938 fix: store every Node.js artifact's integrity separately in the lockfile (#9798)
* fix: store every Node.js artifact's integrity separately in the lockfile

* fix: store every Node.js artifact's integrity separately in the lockfile

* style: fix

* Potential fix for code scanning alert no. 76: Incomplete string escaping or encoding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix: windows

* refactor: node install

* fix: test

* fix: test on Windows

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-07-25 16:31:23 +02:00
Zoltan Kochan
1a07b8fc7c feat: install Node.js runtime as a dependency (#9755) 2025-07-19 15:11:40 +02:00
Khải
6acf819c69 refactor!: stricter Resolution type (#9582)
* refactor!: stricter `Resolution` type

* docs(changeset): don't leave it out

* fix: test

* fix: test

* refactor: replace `as` with `!`
2025-05-30 09:17:12 +02:00
Khải
272129193c refactor: improve resolver types (#9544) 2025-05-16 13:14:06 +02:00
Zoltan Kochan
8a9f3a4835 refactor: rename pref to bare specifier (#9445) 2025-04-20 22:58:08 +02:00
Khải
9c3dd03710 feat: add support for the jsr: protocol (#9358)
close #8941

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-04-20 11:51:51 +02:00
Zoltan Kochan
5b73df1eb1 refactor: resolvers should calculate the specifiers that are saved into package.json (#9426)
* refactor: resolvers should return specifier templates

* refactor: updating workspace protocol specs in package.json

* refactor: move workspace selector calculation logic to npm-resolver

* refactor: move workspace selector calculation logic to npm-resolver

* refactor: calculating range in npm-resolver

* refactor: rename normalizedPref and specifierTemplate to specifier

* refactor: specifiers creation

* refactor: npm-resolver

* refactor: remove which-version-is-pinned package

* refactor: which version is pinned

* docs: add changesets

* refactor: implement suggestions

* refactor: revert regex usage
2025-04-18 17:48:03 +02:00
Zoltan Kochan
81f441cd3a refactor: updating specs in the manifest (#9410) 2025-04-13 21:44:35 +02:00
Khải
72cff38486 refactor: pass whole registries to the resolver (#9375) 2025-04-04 01:20:16 +02:00
Brandon Cheng
f9c53da20b feat: return workspace for resolvedVia for workspace packages (#9293)
* feat: return `workspace` for `resolvedVia`

* chore: update .changeset/shaggy-chefs-reply.md
2025-03-17 01:40:56 +01:00
Zoltan Kochan
6483b646fe feat: a new setting for injecting workspace packages (#8836) 2024-12-05 17:37:15 +01:00
Zoltan Kochan
0406d4ad3d refactor: use a branded type for project root dir paths (#8272) 2024-07-04 13:13:34 +02:00
Zoltan Kochan
341656f9b3 refactor: make workspace packages a map 2024-07-02 22:49:38 +02:00
Zoltan Kochan
dd00eeb18c refactor: normalize dir field names across project objects (#8262) 2024-07-01 11:27:04 +02:00
Zoltan Kochan
9927c2855a refactor(core): workspace packages (#8248)
Make workspace packages optional.
2024-06-30 15:17:07 +02:00
Zoltan Kochan
45f4262f03 refactor: create a separate type for NodeId (#8092) 2024-05-17 11:23:52 +02:00
Khải
0ec0a44da7 fix(update): stop downgrading prereleases (#7466)
close #7436

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2024-01-27 14:41:58 +01:00
Rex Zeng
b13d2dc1ae feat: add sub folder support for git url (#7487)
close #4765

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2024-01-24 02:15:02 +01:00
Steven Petryk
4c2450208d fix(lockfile): remove registry from tarball resolutions (#7262)
In some cases, pnpm would add a `registry` field to tarball resolutions
in pnpm-lock.yaml. This field is no longer used, so it was removed,
which should make lockfiles smaller and their generation more
deterministic.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2023-10-31 23:57:17 +02:00
Zoltan Kochan
ee61ca4cb7 style: update eslint (#6236) 2023-03-18 14:57:29 +02:00
Zoltan Kochan
029143cfff fix: give priority to packages installed in the root, when deduping (#6054) 2023-02-08 13:19:40 +02:00
Zoltan Kochan
4ca53b0b50 refactor: group projects in different subdirectories (#5659) 2022-11-20 01:35:22 +02:00