Commit Graph

28 Commits

Author SHA1 Message Date
Zoltan Kochan
212315de16 fix: cap lockfile verification memory and add trustLockfile opt-out (#11878)
* fix: cap lockfile verification memory and add trustLockfile opt-out

Verifying a multi-thousand-entry lockfile against `minimumReleaseAge`
or `trustPolicy: no-downgrade` retained every fetched packument in a
per-install cache for the entire install. On large workspaces this
OOM'd CI runners with a 2GB heap cap. Project both caches down to just
the fields each check reads (per-version trust evidence + the `time`
map for trust; package-level `modified` + version-name set for the
abbreviated shortcut) so the bulk packument is GC'd as soon as the
fetch returns.

Also adds a `trustLockfile` setting (default `false`) that skips the
verification pass entirely for environments where the lockfile is
already part of the trusted base. Mirrored in pacquet. Closes #11860.

* perf: share resolver packument cache with the lockfile verifier

The verifier kept its own per-install dedup Maps and re-fetched every
packument the resolver had already pulled during the same install.
Plumb the resolver's per-install `PackageMetaCache` through to the
verifier (via `createNpmResolutionVerifier` / `build_resolution_verifiers`)
so a name already in the resolver's LRU short-circuits the verifier's
disk/network round-trip — fast path only, the cached document is
projected for the trust check so the verifier's memory footprint stays
bounded.

In pnpm, `installing/client` now constructs one LRU and hands it to
both `createResolver` and `createResolutionVerifiers`. In pacquet, the
`InMemoryPackageMetaCache` is lifted to `Install::dispatch` and passed
to both `build_resolution_verifiers` and `InstallWithFreshLockfile`.
2026-05-23 20:33:03 +02:00
Zoltan Kochan
1627943d2a feat(outdated): include node, deno, and bun runtimes (#11739)
`pnpm outdated` and `pnpm update --interactive` previously skipped runtime dependencies (`node`/`deno`/`bun` installed via the `runtime:` protocol). Both commands go through `outdatedDepsOfProjects` → `outdated()`, and the inner loop bailed out for anything `parseBareSpecifier` couldn't parse — which is everything `runtime:`-shaped. A runtime was only ever reported if the current install differed from the wanted lockfile entry, so the latest available version was never surfaced. The same gap silently affected `jsr:` and named-registry deps too.

Commits, smallest fix first → progressively cleaner architecture:

1. **`feat(outdated)`** — minimal fix: special-case runtime deps in `outdated.ts` so they appear in the table and the interactive update picker.
2. **`refactor(outdated)`** — per-resolver dispatch. Each protocol resolver gets its own "what's the latest?" function; `@pnpm/resolving.default-resolver` composes them.
3. **`refactor(outdated)`** — rename to `resolveLatest` (the function returns info regardless of whether the dep is outdated; "outdated" described a state, not an action).
4. **`refactor(outdated)`** — let the local-resolver own the `link:`/`file:` skip, drop the matching short-circuit in `outdated.ts`.
5. **`refactor(outdated)`** — slim `LatestQuery` / `LatestInfo` to the bare essentials; move `pickRegistryForPackage` into the npm-resolver where it belongs; derive `current`/`wanted` display from `pkgSnapshot.version` in `outdated.ts`.
6. **`chore(outdated)`** — drop stale tsconfig project reference left behind by #5.
7. **`refactor(outdated)`** — drop `wantedRef` from the query; resolvers detect protocol from `bareSpecifier` alone.

## Final architecture

`@pnpm/resolving.resolver-base` defines a single tiny protocol:

```ts
interface LatestQuery {
  wantedDependency: WantedDependency
  compatible?: boolean
}

interface LatestInfo {
  latestManifest?: PackageManifest
}

type ResolveLatestFunction = (query: LatestQuery, opts: ResolveOptions) =>
  Promise<LatestInfo | undefined>
```

- `undefined` from a resolver means "I don't claim this dep — try the next one."
- `{}` means "I claim it, but I can't tell you what's latest" (policy-blocked, network unavailable, or a protocol with no concept of latest — git/tarball).
- `{ latestManifest }` is the happy path.

Each protocol resolver (npm/jsr/named-registry, git, tarball, local, node/bun/deno runtimes) exports its own `resolveLatest*` function alongside its `resolve*`. `@pnpm/resolving.default-resolver` composes them into a single first-match dispatcher, surfaced through `@pnpm/installing.client` as `createResolver(...).resolveLatest`.

`outdated.ts` is protocol-agnostic: dispatches, then derives `current`/`wanted` display from `pkgSnapshot.version` (falling back to the raw ref for URL-shaped refs where the URL is the only diff signal between commits), uses raw `wantedRef !== currentRef` for the lockfile-shifted check, and pulls `packageName` from `dp.parse(relativeDepPath).name` so aliased deps still report under the real package name.

Per-resolver responsibilities:
- **npm-resolver** (`resolveLatestFromNpm` / `resolveLatestFromJsr` / `resolveLatestFromNamedRegistry`): match their respective spec shapes, call the matching `resolveFromX` with `'latest'` (or the original spec under `--compatible`), handle `MINIMUM_RELEASE_AGE_VIOLATION` and `ERR_PNPM_NO_MATCHING_VERSION` so policy-blocked deps don't surface as available updates. Picks the per-package registry internally via its ctx.
- **node/bun/deno runtime resolvers**: claim deps via `bareSpecifier.startsWith('runtime:')` + alias match, query their release sources for the latest version (only the version — no asset-hash fetches), return `{ latestManifest }`.
- **git / tarball resolvers**: claim deps via spec shape, return `{}` (no concept of "latest"); the caller still surfaces a ref-mismatch report if the lockfile shifted to a different commit/URL.
- **local-resolver**: returns `undefined` so `link:`/`file:`/`workspace:` deps fall through and get silently skipped.
2026-05-19 19:15:07 +02:00
Zoltan Kochan
4195766f10 feat: tighten minimumReleaseAge — auto-exclude, lockfile verification, and interactive prompt (#11705)
Three coordinated changes that close the silent-bypass gap in loose `minimumReleaseAge` mode AND the discover-by-loop UX problem in strict mode (#10488), plus a parallel hardening of the lockfile verifier:

1. **Auto-collect into `minimumReleaseAgeExclude` (loose mode)** — fresh resolutions that fall back to a version newer than the cutoff are auto-recorded into the workspace manifest's `minimumReleaseAgeExclude`. A single info message lists what was persisted. The workspace manifest writer dedupes against existing entries.

2. **Lockfile verifier runs in loose mode too** — `createNpmResolutionVerifier` no longer gates on `minimumReleaseAgeStrict`. With auto-collect keeping the exclude list explicit, every accepted-immature pin must be on the list — same contract strict mode enforces. Lockfiles produced under a weaker (or absent) policy that still hold immature entries are rejected the same way strict mode would.

3. **Strict mode prompts on the aggregate set instead of throwing on the first** — the resolver always collects every immature direct and transitive in one pass; the install command's `handleResolutionPolicyViolations` checkpoint decides what to do with the set. Interactive (TTY) prompts the user once with the full list (default = No) and asks whether to add them all to `minimumReleaseAgeExclude` and proceed. Approve → install continues, persisted at the end. Decline → resolution aborts before the lockfile, package.json, or modules dir is touched. Non-interactive (CI) keeps `ERR_PNPM_NO_MATURE_MATCHING_VERSION` as the exit code but lists every offending entry instead of just the first one the resolver happened to hit.

4. **The lockfile verifier now also covers `trustPolicy: 'no-downgrade'`.** The same post-resolution gate that re-checks `minimumReleaseAge` on lockfile entries now re-runs `failIfTrustDowngraded` for every npm-registry entry whose name isn't on `trustPolicyExclude`. The two checks share a single full-metadata fetch per package, so the extra coverage doesn't cost an extra round trip when both policies are active. Resolver-time trust checks still run as before — this just closes the gap when an entry bypasses resolution (peek path, `--frozen-lockfile`, restored CI cache).

The steady-state flows:

- **Loose mode, `pnpm add foo@immature`**: lockfile clean, verifier no-op, resolver picks via lowest-version fallback, `foo@immature` lands in `minimumReleaseAgeExclude`, install succeeds. Subsequent `pnpm install --frozen-lockfile` in CI verifies against the populated list and succeeds.
- **Strict mode (interactive), security bump to `next@15.5.9`**: resolver collects `next@15.5.9` AND every immature `@next/swc-*@15.5.9` shim. pnpm prompts once with the full list. User approves → install completes, all entries persisted in `pnpm-workspace.yaml`. CI then runs the populated config cleanly.
- **Strict mode (non-interactive / CI)**: aborts with `ERR_PNPM_NO_MATURE_MATCHING_VERSION` listing every immature entry's `name@version` and publish time — no more discover-by-loop dance.
- **Teammate commits a poisoned lockfile**: single-policy batches reject with `ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION` (or `ERR_PNPM_TRUST_DOWNGRADE`); a batch that trips both policies escalates to the generic `ERR_PNPM_LOCKFILE_RESOLUTION_VERIFICATION` and lists each entry's per-policy code in the breakdown.

### Implementation

- The npm resolver always falls back to the lowest matching version when no mature version satisfies the range, and flags the result with `ResolveResult.policyViolation` instead of throwing `NO_MATURE_MATCHING_VERSION`. `deferImmatureDecision` and `strictPublishedByCheck` are gone — every caller (install, dlx, outdated, self-update) inspects the violation and decides what to do.
- `policyViolation` flows from `ResolveResult` → `PackageResponse.body.policyViolation` → a shared accumulator in `ResolutionContext` → the `resolutionPolicyViolations` field on `resolveDependencyTree`'s return → out through `mutateModules` / `addDependenciesToPackage` to the install command.
- The violation type lives in `@pnpm/resolving.resolver-base` as `ResolutionPolicyViolation`; the npm resolver exports the two built-in codes (`MINIMUM_RELEASE_AGE_VIOLATION_CODE`, `TRUST_DOWNGRADE_VIOLATION_CODE`) as constants so consumers reference one source of truth.
- `handleResolutionPolicyViolations` runs between `resolveDependencyTree` and `resolvePeers` — the resolver-agnostic checkpoint where the install command's plan prompts (TTY) or aborts (no-TTY) with the full violation list.
- `setupPolicyHandlers` (in `installing/commands/src/policyHandlers.ts`) composes per-policy handlers behind a uniform plan interface: each handler has its own `handleResolutionPolicyViolations` (filter by code, decide what to do) and `pickManifestUpdates` (return a typed `WorkspaceManifestPolicyUpdates` patch the install command spreads into `updateWorkspaceManifest`). Today the only registered handler is `createMinimumReleaseAgeHandler` — strict + TTY prompts via `enquirer`, strict no-TTY throws `ERR_PNPM_NO_MATURE_MATCHING_VERSION` with every entry listed, loose mode auto-persists at the tail. Strict + `--no-save` is rejected up-front via `ERR_PNPM_STRICT_MIN_RELEASE_AGE_REQUIRES_SAVE`. Future policies plug in via a sibling factory + push into the handlers list, with no changes to `installDeps.ts` / `recursive.ts`.
- `installDeps` / `recursive` drain `pickManifestUpdates` after install and spread the patch into `updateWorkspaceManifest`. Plain `pnpm install` (no `--update`, no params) now still updates the workspace manifest when any handler contributes a patch. The `install` command's CLI schema gained `save: Boolean` so `--no-save` actually flows through to `opts.save = false` instead of being silently dropped by nopt.
- `makeResolutionStrict` (in `installing/client`) wraps a `ResolveFunction` and rethrows any `policyViolation` as a `PnpmError`. Used by `dlx` and `self-update` under strict `minimumReleaseAge` OR `trustPolicy: 'no-downgrade'`, since one-shot callers have nowhere to defer a violation to. Violation-code → error-code mapping lives in one place so future violation kinds get consistent UX.
- `createNpmResolutionVerifier` extends its check to `trustPolicy: 'no-downgrade'` — same per-entry fan-out, same cache key, sharing the full-metadata fetch with the maturity check. Trust-fetch errors now propagate up so the violation reason carries the underlying message (network code, 404 detail) instead of a generic "metadata is unavailable".
- `verifyLockfileResolutions`'s aggregate throw uses the per-policy code when every violation in the batch shares it, and escalates to a generic `LOCKFILE_RESOLUTION_VERIFICATION` (with per-entry codes in the breakdown) for mixed batches.
- The pnpm agent path refuses installs under `trustPolicy: 'no-downgrade'` (`ERR_PNPM_TRUST_POLICY_INCOMPATIBLE_WITH_AGENT`) — the agent has no server-side counterpart to that check yet, so silently allowing it would land a lockfile the local verifier would later reject. `minimumReleaseAge` is forwarded to the agent and enforced server-side, so that combination is fine.

### Pacquet parity

Pacquet only carries a stub reference to `minimumReleaseAgeExclude` (see `pacquet/crates/package-manager/src/version_policy.rs`); the broader `minimumReleaseAge` and `trustPolicy` policies aren't ported yet, so this feature is outside pacquet's current surface area. It'll come along when pacquet ports the policies.

### Closes

- Closes #10488 (resolves the discover-by-loop dance for security bumps without needing `withTransitives`).
2026-05-18 09:51:11 +02:00
Zoltan Kochan
fcf95c7faa perf: cache the post-resolution lockfile verification gate (#11691)
Closes #11687.

## What

Cache the result of the post-resolution lockfile verification gate (#11583) so repeat installs against an unchanged lockfile skip the per-package registry round trips entirely. Persisted as JSON Lines at `<cacheDir>/lockfile-verified.jsonl`.

The cache layer is policy-neutral. Today there's one verifier (`minimumReleaseAge`); future resolver-side verifiers (jsr trust, attestation, …) plug in by declaring their own `policy` slot and `canTrustPastCheck` comparator — no install-side changes.

## Why

#11583 re-hits the registry on every install for every locked (name, version) pair. On warm/repeat installs where the lockfile hasn't moved, that's a stack of per-package round trips with nothing to show for them. This change makes the steady-state case effectively free without weakening the protection — the gate still runs in full whenever the lockfile changes, any verifier's policy tightens, or no record exists.

## How

### Cache lookup, in order

The cache is **indexed by content hash** so git worktrees with identical lockfile bytes share a cache entry. A secondary path-keyed index drives the same-machine stat shortcut.

1. **`stat()` shortcut** — when a previous record for this exact `lockfilePath` matches today's `size + mtime + inode`, trust the cached hash without reading anything. Zero I/O beyond the stat. Microseconds.
2. **Content lookup** — hash the in-memory lockfile (not the file bytes — we already have the parsed object) and look up by content hash. Catches worktrees (same content, different path) and CI checkouts (same content, reset stat). On hit, append a refreshed path/stat entry so the next install at this path takes the stat shortcut.
3. **Any active verifier rejects the cached `policy`** — run the full gate.
4. **No record** — run the full gate.

The in-memory object is hashed with `hashObject` from `@pnpm/crypto.object-hasher` (streaming, key-order-stable).

### Record shape

```json
{
  "lockfile": {
    "hash": "<sha256 base64>",
    "path": "/abs/path/to/pnpm-lock.yaml",
    "size": 154,
    "mtimeNs": "1736245123000000000",
    "inode": "12345"
  },
  "verifiedAt": "2026-05-17T...",
  "policy": { "minimumReleaseAge": 1440 }
}
```

`policy` is the union of every active verifier's `policy` contribution. Verifiers checking the same logical policy (e.g. `minimumReleaseAge` honored by multiple registries) name it the same and share the slot — no resolver namespacing.

### File semantics

- **Sync fs throughout** — the cache is consulted once before verification fan-out and recorded once after. No concurrent install work to overlap with; keeping the call sites straight-line.
- **JSONL appends are atomic** on POSIX/NTFS, so parallel pnpm processes (monorepo installs, CI matrices sharing a cache) write without coordination. Latest record per `(path, hash)` tuple wins on read.
- **Bounded file** — capped at ~1000 entries; compaction is triggered by a single `stat()` of the cache file (1.5 MiB byte budget) so we never parse the file on the steady-state path. When triggered, the tail is rewritten via tempfile + rename.
- **No record on rejection** — a failing verification deliberately doesn't write a record; the next install must rerun the gate.
- **Single hash per install** — the in-memory hash is computed lazily and reused: `tryLockfileVerificationCache` returns the precomputed stat+hash to `recordVerification` on a miss, and the stat-shortcut hit forwards the cached record's hash unchanged.

## Plumbing

The verifier contract changed alongside the cache to make this composable without install-side knowledge of each policy:

- **`@pnpm/resolving.resolver-base`** — `ResolutionVerifier` is now `{ verify, policy, canTrustPastCheck }` (was a bare function in #11583). Each resolver-side verifier owns its policy snapshot and the comparator that decides whether a cached policy is still trustworthy.
- **`@pnpm/resolving.npm-resolver`** — `createNpmResolutionVerifier` returns the new shape: `policy: { minimumReleaseAge }`, `canTrustPastCheck` reads `minimumReleaseAge` from the merged cached bag.
- **`@pnpm/resolving.default-resolver`** — `createResolutionVerifier` (singular, returning a combined function) → `createResolutionVerifiers` (plural, returning a `ResolutionVerifier[]`). No combinator; each verifier handles its own protocol short-circuit inside `verify`, so dispatch happens naturally at the install side.
- **`@pnpm/installing.client`** — `Client.verifyResolution?` → `Client.resolutionVerifiers: ResolutionVerifier[]`. Same rename propagates through `@pnpm/store.connection-manager`, `@pnpm/testing.temp-store`, and `StrictInstallOptions`.
- **`@pnpm/installing.deps-installer`** — new `verifyLockfileResolutionsCache.ts` (`tryLockfileVerificationCache` + `recordVerification`). `verifyLockfileResolutions` takes the verifier list plus `cacheDir` + `lockfilePath` as flat options; the cache fires when both are present, otherwise the gate runs without memoization. The dedup key for in-flight candidates includes a serialization of `resolution` so two entries sharing a (name, version) but pinned via different protocols don't collapse.

Breaking but safe — `@pnpm/resolving.npm-resolver` hasn't been released since #11583 introduced the verifier abstraction, so no downstream consumer is on the old shape.

## Tests

- **17 unit tests** in `verifyLockfileResolutionsCache.ts`: cache miss/hit, stat shortcut, size mismatch falling through to hash lookup, hash-fallback on reset stat, content change with matching size, stricter/weaker policy, missing-field policy rejection, multi-verifier policy merge (shared field stored once), worktree case (same content, different path), JSONL append semantics, malformed-line tolerance.
- **12 integration tests** in `verifyLockfileResolutions.ts`: dedup of peer/patch-suffix variants, distinct-resolution dedup at the same (name, version), stable violation ordering, the 20-entry cap, multi-verifier fan-out (first failure wins), cache short-circuit on a passing run, no cache write on a rejecting run, empty-verifier-list passthrough.
- **1 e2e test** in `pnpm/test/install/minimumReleaseAge.ts`: bundled CLI plumbing — install once to seed the lockfile, enable `minimumReleaseAge` + `cacheDir`, install again, assert the cache file lands at `<cacheDir>/lockfile-verified.jsonl` with the documented record shape.
- Existing `minimumReleaseAge` (13) and `frozenLockfile` (12) suites still pass.
2026-05-17 13:07:24 +02:00
Ryo Matsukawa
31538bf8d2 fix: enforce minimumReleaseAge on existing lockfile entries (#11583)
Closes #10438.

## What

Re-verify every entry in `pnpm-lock.yaml` against the policies the resolver chain was configured with — today: `minimumReleaseAge` in strict mode — right after the lockfile is loaded from disk and before any tarball is fetched. A locked version that fails the policy aborts the install with `ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION`; `minimumReleaseAgeExclude` is honored.

## Why

The policy only fires while pnpm is *choosing* a version. Once a version is pinned in the lockfile — e.g. a developer disabled the policy locally and committed a fresh dependency, or a CI cache restored a stale lockfile — every later `pnpm install` (including `--frozen-lockfile` and `pnpm fetch`) installs it without re-checking, which defeats the supply-chain protection the setting is supposed to provide.

The threat model is **a lockfile someone else resolved**, not local resolution: local resolution is already covered by the resolver's own per-version filter. bun fixed the same shape of bug in [oven-sh/bun#30526](https://github.com/oven-sh/bun/pull/30526); this PR is the pnpm side.

## How

The fix introduces a generic `ResolutionVerifier` abstraction in the resolver chain — each resolver factory can ship a sibling verifier factory, exactly the way each resolver ships a `resolve` function. Today there's one verifier (npm); the shape leaves room for future ones (jsr, attestation-based, etc.) without changing the install-side interface.

- **`@pnpm/resolving.resolver-base`** exports the `ResolutionVerifier` / `ResolutionVerification` types — the shared contract.
- **`@pnpm/resolving.npm-resolver`** exports `createNpmResolutionVerifier`. Returns `undefined` when no policy is active, so callers can cheaply decide whether to iterate at all. When active, it inspects each lockfile entry, handles `minimumReleaseAgeExclude`, routes through named-registry prefixes (built-ins like `gh:` merged in), and uses `fetchFullMetadataCached` to fetch full registry metadata — decoupled from the resolver pipeline so neither `peekManifestFromStore` nor abbreviated metadata can hide the publish timestamp.
- **`@pnpm/resolving.default-resolver`** exports `createResolutionVerifier`, a combinator that asks each underlying verifier (today: npm) if it has work and returns `undefined` when none does. Designed so that adding more verifiers later doesn't change the install side.
- **`@pnpm/installing.client`** exposes `verifyResolution` on `Client`, built from the same `fetchFromRegistry` / `getAuthHeader` the resolver chain already uses — **no second fetcher is constructed**.
- **`@pnpm/store.connection-manager`** and **`@pnpm/testing.temp-store`** surface `verifyResolution` alongside the store controller they hand back, so it reaches `mutateModules` through the existing plumbing.
- **`@pnpm/installing.deps-installer`** gains one option on `StrictInstallOptions`: `verifyResolution?: ResolutionVerifier`. `mutateModules` invokes `verifyLockfileResolutions(ctx.wantedLockfile, opts.verifyResolution)` **once**, right after `getContext` returns the on-disk lockfile and before any path branches. When the verifier is `undefined`, the call is a no-op. The iteration is policy-neutral: dedupes by `(name, version)`, applies `pLimit(16)`, sorts violations stably, caps the printed list at 20 with an `…and N more` summary, throws a `PnpmError` carrying the verifier-supplied error code.

The error includes a recovery hint that points at `pnpm clean --lockfile` followed by `pnpm install` — the safe way to throw away a poisoned lockfile and rebuild from fresh resolution.

## Tests

- **9 unit tests** for `verifyLockfileResolutions` against a mock `ResolutionVerifier` — dedup, aggregation, stable ordering, the 20-entry cap, no-op behavior, the verifier-supplied error code surfacing in `PnpmError`.
- **13 integration tests** in `installing/deps-installer/test/install/minimumReleaseAge.ts` via the real `install()` entry — `testDefaults()` wires `verifyResolution` from `createTempStore` → `createClient`, so the npm verifier runs end-to-end at the install boundary. Covers the rejection scenario, `minimumReleaseAgeExclude`, the strict-mode toggle, the existing `minimumReleaseAge` resolver-side suite, and a `pnpm add` scenario where a pre-existing entry would otherwise survive resolution.
- **3 e2e tests** in `pnpm/test/install/minimumReleaseAge.ts` against the bundled CLI: rejection path with the right `ERR_PNPM_*` code and `pnpm clean --lockfile` hint in output, `minimumReleaseAgeExclude` honored, and the strict-off path (which now requires an explicit `minimumReleaseAgeStrict: false` since the config reader auto-enables strict mode when `minimumReleaseAge` is set).
- Existing `frozenLockfile` suite (12 tests) and npm-resolver suite (179 tests) still pass.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-05-16 21:38:06 +02:00
Zoltan Kochan
3ab403a1fa fix: route <alias>:@scope/pkg to the named-registry resolver (#11598)
## Summary

- The local resolver's path-shape match was claiming any specifier containing `/` as a local directory, so `pnpm add bit:@teambit/bit` (with `bit` configured under `namedRegistries`) installed a bogus link to `bit:@teambit/bit/` instead of resolving from the configured registry.
- Split the local resolver into two exports: `resolveFromLocalScheme` (handles `file:`/`link:`/`workspace:`/`path:`) and `resolveFromLocalPath` (path-shape match — tarball extension, `path.sep`, `isFilespec`). `resolveFromLocal` is removed.
- Re-order the default-resolver chain so the scheme pass runs *before* `resolveFromNamedRegistry` and the path pass runs *after*. Explicit local protocols still win even when a user configures a colliding `namedRegistries` alias; named-registry aliases reach their configured URL.

Repro before the fix:

```
$ cat pnpm-workspace.yaml
namedRegistries:
  bit: https://node-registry.bit.cloud/

$ pnpm add bit:@teambit/bit
[WARN] Installing a dependency from a non-existent directory: /private/tmp/.../bit:@teambit/bit

dependencies:
+ bit 0.0.0 <- bit:@teambit/bit
```

After the fix, the same command resolves `@teambit/bit 1.13.173` from `https://node-registry.bit.cloud/` and writes `"@teambit/bit": "bit:^1.13.173"` to `package.json`.
2026-05-12 12:50:17 +02:00
Igor Savin
b61e268d57 feat: add support for github prefix and named registries (#11324)
This is consistent with #9358, but implements support for the GitHub Packages npm registry and, more broadly, for vlt-style https://docs.vlt.sh/cli/registries for any registry.

This PR adds a built-in gh: specifier that resolves against the GitHub Packages npm registry, plus a namedRegistries config key so a project can map its own aliases to arbitrary registries. A project can mix public npm packages and private GitHub Packages (or self-hosted) ones without applying a scope-wide registry override to every @scope/* package.

- pnpm add gh:@acme/private writes "@acme/private": "gh:^1.0.0" and resolves from https://npm.pkg.github.com/.
- pnpm add gh:@acme/private@^1.0.0 (with or without an alias) is also supported. Aliased form writes "my-alias": "gh:@acme/private@^1.0.0".
- Auth comes from the existing per-URL .npmrc mechanism, e.g. //npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}. No new auth surface.
- @github is intentionally not defaulted to https://npm.pkg.github.com/ - hardcoding that would hijack installs of the public @github/* packages on npmjs.org (e.g. @github/relative-time-element) for users without a scope-wide override. Use gh: to install from GitHub Packages, or configure @github:registry=... yourself if that's really what you want.
- Additional named registries (a self-hosted proxy, GitHub Enterprise Server, etc.) can be configured in pnpm-workspace.yaml:

```yml
namedRegistries:
  gh: https://npm.pkg.github.example.com/   # optional: overrides the built-in `gh` alias for GHES
  work: https://npm.work.example.com/
```

- Then work:@corp/lib@^2.0.0 resolves against https://npm.work.example.com/, and the built-in gh alias can be redirected to a GHES host.
- Env-var substitution (${VAR}) is supported in namedRegistries values, mirroring the .npmrc convention.
- Reserved alias names (npm, jsr, github, workspace, catalog, file, git, http, https, link, patch, and related git host shorthands) cannot be redefined as user-named registries - the resolver throws ERR_PNPM_RESERVED_NAMED_REGISTRY_ALIAS at startup rather than silently shadowing another protocol. Malformed URLs throw ERR_PNPM_INVALID_NAMED_REGISTRY_URL at startup too, instead of failing as a confusing 404 during resolution.
- On publish, createExportableManifest strips any named-registry prefix (both the built-in gh: and any user-configured alias) so npm and yarn consumers can still resolve the dependency via their own scope-registry configuration - mirroring the user-facing requirement when installing such a dep without the prefix.

The prefix is gh: rather than github: because github: is reserved by npm-package-arg / hosted-git-info as a git host shorthand (e.g. github:owner/repo) - reusing it would be a deviation from the specs used by the npm CLI. gh: is  shorter, matches vlt's convention, and cannot collide with any existing npm scheme.

Unlike jsr:, gh: (and any other named-registry alias) does not rewrite the package name - gh:@acme/foo resolves @acme/foo from the GitHub Packages registry as-is. This also means npm/yarn consumers see the original name after the prefix is stripped on publish.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-04-29 12:38:56 +02:00
Zoltan Kochan
96704a1c58 refactor(config): rename rawConfig to authConfig, add nodeDownloadMirrors, simplify config reader (#11194)
Major cleanup of the config system after migrating settings from `.npmrc` to `pnpm-workspace.yaml`.

### Config reader simplification
- Remove `checkUnknownSetting` (dead code, always `false`)
- Trim `npmConfigTypes` from ~127 to ~67 keys (remove unused npm config keys)
- Replace `rcOptions` iteration over all type keys with direct construction from defaults + auth overlay
- Remove `rcOptionsTypes` parameter from `getConfig()` and its assembly chain

### Rename `rawConfig` to `authConfig`
- `rawConfig` was a confusing mix of auth data and general settings
- Non-auth settings are already on the typed `Config` object — stop duplicating them in `rawConfig`
- Rename `rawConfig` → `authConfig` across the codebase to clarify it only contains auth/registry data from `.npmrc`

### Remove `rawConfig` from non-auth consumers
- **Lifecycle hooks**: replace `rawConfig: object` with `userAgent?: string` — only user-agent was read
- **Fetchers**: remove unused `rawConfig` from git fetcher, binary fetcher, tarball fetcher, prepare-package
- **Update command**: use `opts.production/dev/optional` instead of `rawConfig.*`
- **`pnpm init`**: accept typed init properties instead of parsing `rawConfig`

### Add `nodeDownloadMirrors` setting
- New `nodeDownloadMirrors?: Record<string, string>` on `PnpmSettings` and `Config`
- Replaces the `node-mirror:<channel>` pattern that was stored in `rawConfig`
- Configured in `pnpm-workspace.yaml`:
  ```yaml
  nodeDownloadMirrors:
    release: https://my-mirror.example.com/download/release/
  ```
- Remove unused `rawConfig` from deno-resolver and bun-resolver

### Refactor `pnpm config get/list`
- New `configToRecord()` builds display data from typed Config properties on the fly
- Excludes sensitive internals (`authInfos`, `sslConfigs`, etc.)
- Non-types keys (e.g., `package-extensions`) resolve through `configToRecord` instead of direct property access
- Delete `processConfig.ts` (replaced by `configToRecord.ts`)

### Pre-push hook improvement
- Add `compile-only` (`tsgo --build`) to pre-push hook to catch type errors before push
2026-04-04 20:33:43 +02:00
Zoltan Kochan
dba4153767 refactor: rename packages and consolidate runtime resolvers (#10999)
* refactor: rename workspace.sort-packages and workspace.pkgs-graph

- workspace.sort-packages -> workspace.projects-sorter
- workspace.pkgs-graph -> workspace.projects-graph

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rename packages/ to core/ and pkg-manifest.read-package-json to reader

- Rename packages/ directory to core/ for clarity
- Rename pkg-manifest/read-package-json to pkg-manifest/reader (@pnpm/pkg-manifest.reader)
- Update all tsconfig, package.json, and lockfile references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: consolidate runtime resolvers under engine/runtime domain

- Remove unused @pnpm/engine.runtime.node.fetcher package
- Rename engine/runtime/node.resolver to node-resolver (dash convention)
- Move resolving/bun-resolver to engine/runtime/bun-resolver
- Move resolving/deno-resolver to engine/runtime/deno-resolver
- Update all package names, tsconfig paths, and lockfile references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update lockfile after removing node.fetcher

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sort tsconfig references and package.json deps alphabetically

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: auto-fix import sorting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update __typings__ paths in tsconfig.lint.json for moved resolvers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove deno-resolver from deps of bun-resolver

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 00:19:58 +01:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
1c8c4e49f5 style: add eslint-plugin-simple-import-sort (#10947)
Add eslint-plugin-simple-import-sort to enforce consistent import ordering:
- Node.js builtins first
- External packages second
- Relative imports last
- Named imports sorted alphabetically within each statement
2026-03-13 02:02:38 +01:00
Brandon Cheng
01914345d5 build: enable @typescript-eslint/no-import-type-side-effects (#10630)
* build: enable `@typescript-eslint/no-import-type-side-effects`

* build: disable `@typescript-eslint/consistent-type-imports`

* chore: apply fixes for `no-import-type-side-effects`

pnpm exec eslint "**/src/**/*.ts" "**/test/**/*.ts" --fix
2026-03-08 00:02:48 +01:00
Zoltan Kochan
ec7c5d7d1a feat: improve git URL detection to recognize plain HTTP/HTTPS URLs
Improve git URL detection to recognize plain HTTP/HTTPS URLs
ending in `.git` and prioritize git resolver over tarball resolver.

close #10468
2026-01-16 19:38:02 +01:00
btea
825b98a39d fix: make catalog protocol matching error messages clearer (#10052)
* fix: verify in advance whether the specifier that the catalog pkg is valid

* fix: update error message

* test: update

* Update resolving/default-resolver/src/index.ts

Co-authored-by: Brandon Cheng <gluxon@users.noreply.github.com>

---------

Co-authored-by: Brandon Cheng <gluxon@users.noreply.github.com>
2026-01-14 13:25:27 +01:00
Trevor Burnham
41664e83f5 feat: pass currentPkg to custom resolvers (#10440)
- Add currentPkg (with name/version) to custom resolver ResolveOptions
- Pass currentPkg through to custom resolvers in default-resolver
- Simplify checkCustomResolverForceResolve to use parseDepPath
2026-01-12 21:04:38 +01:00
Zoltan Kochan
0bcbaf9994 refactor: move out skip resolution logic from package requester (#10439) 2026-01-12 13:08:50 +01:00
Trevor Burnham
38b8e357b5 feat: add custom resolvers and fetchers (#10246) 2025-11-30 14:19:04 +01:00
Zoltan Kochan
86b33e91ea feat: support installing Bun runtime (#9815)
* feat: support installing Bun runtime

* feat: support installing Bun runtime

* fix: cache libc resolution

* refactor: shasum file fetching

* docs: add changesets

* feat: installing the right artifact

* test: supported architectures

* test: fix on Windows
2025-07-31 13:46:13 +02:00
Zoltan Kochan
d1edf732ad feat: support installing Deno runtime (#9791)
* feat: support installing Deno runtime

* refactor: use npm registry to resolve deno version

* feat: wip

* feat: installing deno runtime

* style: fix

* test: fix

* test: deno

* test: fix

* feat: deno

* feat: deno

* feat: create zip fetcher

* style: fix

* refactor: node fetch

* feat: support a new binary fetcher

* test: fix

* feat: rename zip-fetcher to binary-fetcher

* refactor: change naming

* fix: windows

* refactor: rename packages

* refactor: deno resolver

* refactor: runtime resolvers

* refactor: binary fetcher

* refactor: runtime resolvers

* refactor: runtime resolvers

* refactor: create SingleResolution

* refactor: remove not needed change

* refactor: package requester

* docs: add changesets

* refactor: use VariationsResolution and AtomicResolution

* refactor: implement CR suggestions

* docs: add changesets

* fix: address comment in CR

* feat: update formatting of pnpm-lock.yaml
2025-07-30 11:27:07 +02:00
Colin T.A. Gray
5dedadac76 fix: changes local-resolver to support absolute paths (#9761)
* feat: changes local-resolver to support absolute paths

Previously absolute paths were being turned to relative paths, but if the file:
specifier is 'file:/path/to/file', and the users are using a shared network
storage, this relative path requires that the users all use the same
local folder structure. Instead, using an absolute path as the specifier
allows them to have the source code anywhere, and the absolute path will
be resolved consistently.

Enabled via the `preserveAbsolutePaths` option.

* chore: changeset

* feat: add preserve absolute paths option

* docs: add changesets

* fix: also update the 'dependencyPath', add test for that case

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-07-23 23:54:49 +02:00
Zoltan Kochan
1a07b8fc7c feat: install Node.js runtime as a dependency (#9755) 2025-07-19 15:11:40 +02:00
Khải
272129193c refactor: improve resolver types (#9544) 2025-05-16 13:14:06 +02:00
Zoltan Kochan
8a9f3a4835 refactor: rename pref to bare specifier (#9445) 2025-04-20 22:58:08 +02:00
Khải
9c3dd03710 feat: add support for the jsr: protocol (#9358)
close #8941

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-04-20 11:51:51 +02:00
btea
b0f3c71b7b fix: get the final address by installing package through links (#8842)
close #8833

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2024-12-11 13:22:15 +01:00
Zoltan Kochan
0c08e1c50f fix: clear metadata cache after resolution is finished (#8109) 2024-05-22 00:23:14 +02:00
Zoltan Kochan
ee61ca4cb7 style: update eslint (#6236) 2023-03-18 14:57:29 +02:00
Zoltan Kochan
4ca53b0b50 refactor: group projects in different subdirectories (#5659) 2022-11-20 01:35:22 +02:00