Commit Graph

69 Commits

Author SHA1 Message Date
Zoltan Kochan
b741d91e67 chore(release): 11.5.0 (#12068) 2026-05-29 17:26:13 +02:00
Marvin Hagemeister
49e6074644 test: replace @pnpm/registry-mock with an in-repo in-process registry (#11927)
Replace the external `@pnpm/registry-mock` (Verdaccio) test dependency with an in-repo, in-process registry that serves package fixtures to **both** the pacquet Rust tests and the pnpm CLI (Jest) tests. No separately managed registry process is needed.

### How it works

- **Fixtures** live at `registry/.fixtures/packages/<name>/<version>/…`, moved verbatim from [`pnpm/registry-mock`](https://github.com/pnpm/registry-mock) (keyed by each `package.json`'s `name`+`version`).
- **`pnpm-registry-fixtures`** builds verdaccio-shaped storage from those fixtures; the in-tree **`pnpm-registry`** crate serves it.
  - Files whose names differ only by case (`@pnpm.e2e/with-same-file-in-different-cases`) and `bundleDependencies` trees are composed **in memory** by the builder, since neither can be committed to the working tree.
- **pacquet**: `pacquet-testing-utils`' `TestRegistry` starts the server lazily (once per process) in proxy mode, serving `@pnpm.e2e` fixtures locally and falling through to the npm uplink for real packages (`is-positive`, `is-negative`, …) — matching how registry-mock behaved.
- **pnpm CLI**: the `with-registry` Jest `globalSetup` builds storage from the fixtures via the new `pnpm-registry-prepare` binary (built from source in the Test CI job) and serves it with `pnpm-registry`. `REGISTRY_MOCK_PORT` / `REGISTRY_MOCK_CREDENTIALS` / `getIntegrity` now come from `@pnpm/testing.registry-mock`.

### Result

`@pnpm/registry-mock` is removed from every manifest, the catalog, and `packageExtensions`; `cargo test` / `cargo nextest run` / `just test` and the pnpm CLI Jest suites all run registry-backed tests without launching Verdaccio.
2026-05-29 14:35:45 +02:00
Zoltan Kochan
72d997cc34 chore(release): 11.4.0 (#11989) 2026-05-27 15:15:01 +02:00
Zoltan Kochan
f2a4d2caef chore(release): 11.3.0 (#11894) 2026-05-24 02:23:07 +02:00
Zoltan Kochan
11a43b15da chore(release): 11.2.1 (#11777) 2026-05-20 16:51:13 +02:00
Zoltan Kochan
0fb723323f chore(release): 11.2.0 (#11764) 2026-05-20 12:41:09 +02:00
Zoltan Kochan
cd80b2c8ae chore(release): 11.1.3 (#11717) 2026-05-18 15:42:32 +02:00
Ryo Matsukawa
31538bf8d2 fix: enforce minimumReleaseAge on existing lockfile entries (#11583)
Closes #10438.

## What

Re-verify every entry in `pnpm-lock.yaml` against the policies the resolver chain was configured with — today: `minimumReleaseAge` in strict mode — right after the lockfile is loaded from disk and before any tarball is fetched. A locked version that fails the policy aborts the install with `ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION`; `minimumReleaseAgeExclude` is honored.

## Why

The policy only fires while pnpm is *choosing* a version. Once a version is pinned in the lockfile — e.g. a developer disabled the policy locally and committed a fresh dependency, or a CI cache restored a stale lockfile — every later `pnpm install` (including `--frozen-lockfile` and `pnpm fetch`) installs it without re-checking, which defeats the supply-chain protection the setting is supposed to provide.

The threat model is **a lockfile someone else resolved**, not local resolution: local resolution is already covered by the resolver's own per-version filter. bun fixed the same shape of bug in [oven-sh/bun#30526](https://github.com/oven-sh/bun/pull/30526); this PR is the pnpm side.

## How

The fix introduces a generic `ResolutionVerifier` abstraction in the resolver chain — each resolver factory can ship a sibling verifier factory, exactly the way each resolver ships a `resolve` function. Today there's one verifier (npm); the shape leaves room for future ones (jsr, attestation-based, etc.) without changing the install-side interface.

- **`@pnpm/resolving.resolver-base`** exports the `ResolutionVerifier` / `ResolutionVerification` types — the shared contract.
- **`@pnpm/resolving.npm-resolver`** exports `createNpmResolutionVerifier`. Returns `undefined` when no policy is active, so callers can cheaply decide whether to iterate at all. When active, it inspects each lockfile entry, handles `minimumReleaseAgeExclude`, routes through named-registry prefixes (built-ins like `gh:` merged in), and uses `fetchFullMetadataCached` to fetch full registry metadata — decoupled from the resolver pipeline so neither `peekManifestFromStore` nor abbreviated metadata can hide the publish timestamp.
- **`@pnpm/resolving.default-resolver`** exports `createResolutionVerifier`, a combinator that asks each underlying verifier (today: npm) if it has work and returns `undefined` when none does. Designed so that adding more verifiers later doesn't change the install side.
- **`@pnpm/installing.client`** exposes `verifyResolution` on `Client`, built from the same `fetchFromRegistry` / `getAuthHeader` the resolver chain already uses — **no second fetcher is constructed**.
- **`@pnpm/store.connection-manager`** and **`@pnpm/testing.temp-store`** surface `verifyResolution` alongside the store controller they hand back, so it reaches `mutateModules` through the existing plumbing.
- **`@pnpm/installing.deps-installer`** gains one option on `StrictInstallOptions`: `verifyResolution?: ResolutionVerifier`. `mutateModules` invokes `verifyLockfileResolutions(ctx.wantedLockfile, opts.verifyResolution)` **once**, right after `getContext` returns the on-disk lockfile and before any path branches. When the verifier is `undefined`, the call is a no-op. The iteration is policy-neutral: dedupes by `(name, version)`, applies `pLimit(16)`, sorts violations stably, caps the printed list at 20 with an `…and N more` summary, throws a `PnpmError` carrying the verifier-supplied error code.

The error includes a recovery hint that points at `pnpm clean --lockfile` followed by `pnpm install` — the safe way to throw away a poisoned lockfile and rebuild from fresh resolution.

## Tests

- **9 unit tests** for `verifyLockfileResolutions` against a mock `ResolutionVerifier` — dedup, aggregation, stable ordering, the 20-entry cap, no-op behavior, the verifier-supplied error code surfacing in `PnpmError`.
- **13 integration tests** in `installing/deps-installer/test/install/minimumReleaseAge.ts` via the real `install()` entry — `testDefaults()` wires `verifyResolution` from `createTempStore` → `createClient`, so the npm verifier runs end-to-end at the install boundary. Covers the rejection scenario, `minimumReleaseAgeExclude`, the strict-mode toggle, the existing `minimumReleaseAge` resolver-side suite, and a `pnpm add` scenario where a pre-existing entry would otherwise survive resolution.
- **3 e2e tests** in `pnpm/test/install/minimumReleaseAge.ts` against the bundled CLI: rejection path with the right `ERR_PNPM_*` code and `pnpm clean --lockfile` hint in output, `minimumReleaseAgeExclude` honored, and the strict-off path (which now requires an explicit `minimumReleaseAgeStrict: false` since the config reader auto-enables strict mode when `minimumReleaseAge` is set).
- Existing `frozenLockfile` suite (12 tests) and npm-resolver suite (179 tests) still pass.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-05-16 21:38:06 +02:00
Zoltan Kochan
8a80235c7b chore(release): 11.1.2 2026-05-14 13:31:53 +02:00
Zoltan Kochan
9a327522ce chore(release): 11.1.1 2026-05-12 12:56:32 +02:00
Zoltan Kochan
732312f49e chore(release): 11.1.0 2026-05-11 19:56:10 +02:00
Zoltan Kochan
f2b28f85ff chore(release): 11.0.9 2026-05-09 02:06:35 +02:00
Zoltan Kochan
a516c24ce4 chore(release): 11.0.8 2026-05-07 08:35:07 +02:00
Zoltan Kochan
0c3ef0ec94 chore(release): 11.0.7 2026-05-07 00:21:03 +02:00
Zoltan Kochan
65f9327014 chore(release): 11.0.6 2026-05-05 19:50:32 +02:00
Zoltan Kochan
3a5534d75e chore(release): 11.0.4 2026-05-03 01:24:22 +02:00
Zoltan Kochan
6ef34b7a11 chore(release): 11.0.3 2026-04-30 23:03:46 +02:00
Zoltan Kochan
a53f78b111 chore(release): 11.0.2 2026-04-30 17:16:34 +02:00
Zoltan Kochan
38ffda2a18 chore(release): 11.0.1 2026-04-29 23:00:21 +02:00
Zoltan Kochan
8aeeff4c46 chore(release): 11.0.0 2026-04-28 11:27:43 +02:00
Zoltan Kochan
fd437ded13 chore(release): 11.0.0-rc.4 2026-04-21 15:03:02 +02:00
Zoltan Kochan
aa93759d9b chore(release): drop eslint from lib prepublishOnly (#11320)
Library packages had `prepublishOnly: pn compile`, which expands to
`tsgo --build && pn lint --fix`. During `pn release` that runs eslint
against ~150 packages for no benefit — the code has already been linted
in CI and the release flow's upfront compile has already built dist/.
Switch lib prepublishOnly to a bare `tsgo --build` so the safety-net
compile stays but the per-package eslint cost is gone.
2026-04-21 01:18:03 +02:00
Zoltan Kochan
fcdd50aaa7 chore(release): 11.0.0-rc.3 2026-04-21 00:17:38 +02:00
Zoltan Kochan
96ece9d736 chore(release): 11.0.0-rc.2 2026-04-17 18:21:35 +02:00
Zoltan Kochan
f7c23231a9 chore(release): 11.0.0-rc.1 2026-04-16 01:18:55 +02:00
Zoltan Kochan
06d6c2d405 chore(release): 11.0.0-rc.0 2026-04-10 18:30:33 +02:00
Zoltan Kochan
d6b8e281b6 chore: use pn instead of pnpm (#11124) 2026-03-28 11:55:51 +01:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
b7f0f21582 feat: use SQLite for storing package index in the content-addressable store (#10827)
## Summary

Replace individual `.mpk` (MessagePack) files under `$STORE/index/` with a single SQLite database at `$STORE/index.db` using Node.js 22's built-in `node:sqlite` module. This reduces filesystem syscall overhead and improves space efficiency for small metadata entries.

Closes #10826

## Design

### New package: `@pnpm/store.index`

A new `StoreIndex` class wraps a SQLite database with a simple key-value API (`get`, `set`, `delete`, `has`, `entries`). Data is serialized with msgpackr and stored as BLOBs. The table uses `WITHOUT ROWID` for compact storage.

Key design decisions:

- **WAL mode** enables concurrent reads from workers while the main process writes.
- **`busy_timeout=5000`** plus a retry loop with `Atomics.wait`-based `sleepSync` handles `SQLITE_BUSY` errors from concurrent access.
- **Performance PRAGMAs**: `synchronous=NORMAL`, `mmap_size=512MB`, `cache_size=32MB`, `temp_store=MEMORY`, `wal_autocheckpoint=10000`.
- **Write batching**: `queueWrites()` batches pre-packed entries from tarball extraction and flushes them in a single transaction on `process.nextTick`. `setRawMany()` writes immediate batches (e.g. from `addFilesFromDir`).
- **Lifecycle**: `close()` auto-flushes pending writes, runs `PRAGMA optimize`, and closes the DB. A `process.on('exit')` handler ensures cleanup even on unexpected exits.
- **`VACUUM` after `deleteMany`** (used by `pnpm store prune`) to reclaim disk space.

### Key format

Keys are `integrity\tpkgId` (tab-separated). Git-hosted packages use `pkgId\tbuilt` or `pkgId\tnot-built`.

### Shared StoreIndex instance

A single `StoreIndex` instance is threaded through the entire install lifecycle — from `createNewStoreController` through the fetcher chain, package requester, license scanner, SBOM collector, and dependencies hierarchy. This replaces the previous pattern of each component creating its own file-based index access.

### Worker architecture

Index writes are performed in the main process, not in worker threads. Workers send pre-packed `{ key, buffer }` pairs back to the main process via `postMessage`, where they are batched and flushed to SQLite. This avoids SQLite write contention between threads.

### SQLite ExperimentalWarning suppression

`node:sqlite` emits an `ExperimentalWarning` on first load. This is suppressed via a `process.emitWarning` override injected through esbuild's `banner` option, which runs on line 1 of both `dist/pnpm.mjs` and `dist/worker.js` — before any module that loads `node:sqlite`.

### No migration from `.mpk` files

Old `.mpk` index files are not migrated. Packages missing from the new SQLite index are re-fetched on demand (the same behavior as a fresh store).

## Changed packages

121 files changed across these areas:

- **`store/index/`** — New `@pnpm/store.index` package
- **`worker/`** — Write batching moved from worker module into `StoreIndex` class; workers send pre-packed buffers to main process
- **`store/package-store/`** — StoreIndex creation and lifecycle management
- **`store/cafs/`** — Removed `getFilePathInCafs` index-file utilities (no longer needed)
- **`store/pkg-finder/`** — Reads from StoreIndex instead of `.mpk` files
- **`store/plugin-commands-store/`** — `store status` uses StoreIndex
- **`store/plugin-commands-store-inspecting/`** — `cat-index` and `find-hash` use StoreIndex
- **`fetching/tarball-fetcher/`** — Threads StoreIndex through fetchers; git-hosted fetcher flushes before reading
- **`fetching/git-fetcher/`, `binary-fetcher/`, `pick-fetcher/`** — Accept StoreIndex parameter
- **`pkg-manager/`** — `client`, `core`, `headless`, `package-requester` thread StoreIndex
- **`reviewing/`** — `license-scanner`, `sbom`, `dependencies-hierarchy` accept StoreIndex
- **`cache/api/`** — Cache view uses StoreIndex
- **`pnpm/bundle.ts`** — esbuild banner for ExperimentalWarning suppression

## Test plan

- [x] `pnpm --filter @pnpm/store.index test` — Unit tests for StoreIndex CRUD and batching
- [x] `pnpm --filter @pnpm/package-store test` — Store controller lifecycle
- [x] `pnpm --filter @pnpm/package-requester test` — Package requester reads from SQLite index
- [x] `pnpm --filter @pnpm/tarball-fetcher test` — Tarball and git-hosted fetcher writes
- [x] `pnpm --filter @pnpm/headless test` — Headless install
- [x] `pnpm --filter @pnpm/core test` — Core install, side effects, patching
- [x] `pnpm --filter @pnpm/plugin-commands-rebuild test` — Rebuild reads from index
- [x] `pnpm --filter @pnpm/license-scanner test` — License scanning
- [x] e2e tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-06 12:59:04 +01:00
Zoltan Kochan
e18a879d72 feat!: drop Node.js 22.12 support 2026-02-18 14:54:09 +01:00
Zoltan Kochan
1b4df57a01 feat!: drop Node.js 20 and 21 support (#10569) 2026-02-08 19:16:24 +01:00
Zoltan Kochan
a00f9e515c chore: use typescript-go (#10452) 2026-01-14 01:18:13 +01:00
Zoltan Kochan
7e2910e70f chore(release): 11.0.0-alpha.0 2025-11-13 15:44:27 +01:00
Zoltan Kochan
3ce5f82bd7 Merge remote-tracking branch 'origin/main' into v11 2025-10-28 18:40:05 +01:00
Zoltan Kochan
49f03d14ee chore(release): 10.20.0 2025-10-28 17:35:21 +01:00
Zoltan Kochan
dab9abef5c Merge remote-tracking branch 'origin/main' into v11 2025-10-24 14:19:07 +02:00
Zoltan Kochan
0cde1287c8 chore: update repository fields 2025-10-23 11:57:12 +02:00
Zoltan Kochan
43d7b18c2f chore(release): 10.19.0 2025-10-21 15:30:20 +02:00
Zoltan Kochan
1bfc105da0 chore(release): 10.18.3 2025-10-14 11:27:45 +02:00
Zoltan Kochan
a43166624e Merge remote-tracking branch 'origin/main' into v11 2025-10-10 10:01:19 +02:00
Zoltan Kochan
1b15e45ae9 chore(release): 10.18.2 2025-10-09 16:56:04 +02:00
Zoltan Kochan
6618431aee chore(release): libs 2025-09-29 11:56:00 +02:00
Zoltan Kochan
f307b9a130 Merge remote-tracking branch 'origin/main' into v11 2025-09-24 10:51:53 +02:00
Zoltan Kochan
f6242c333b chore(release): 10.17.1 2025-09-22 15:09:34 +02:00
Zoltan Kochan
cb0c5b94c6 Merge remote-tracking branch 'origin/main' into v11 2025-09-19 14:31:36 +02:00
Zoltan Kochan
1a1a272fa4 chore(release): 10.17.0 2025-09-17 16:48:25 +02:00
Zoltan Kochan
ea44ff979f chore(release): 10.16.1 2025-09-13 19:20:27 +02:00
Zoltan Kochan
6f861bccaa Merge remote-tracking branch 'origin/main' into v11 2025-09-12 22:35:14 +02:00
Zoltan Kochan
a3c1498403 chore(release): 10.16.0 2025-09-12 14:24:30 +02:00
Zoltan Kochan
3d1711a439 chore(release): 10.15.1 2025-09-01 11:41:05 +02:00