## Summary
Replace individual `.mpk` (MessagePack) files under `$STORE/index/` with a single SQLite database at `$STORE/index.db` using Node.js 22's built-in `node:sqlite` module. This reduces filesystem syscall overhead and improves space efficiency for small metadata entries.
Closes#10826
## Design
### New package: `@pnpm/store.index`
A new `StoreIndex` class wraps a SQLite database with a simple key-value API (`get`, `set`, `delete`, `has`, `entries`). Data is serialized with msgpackr and stored as BLOBs. The table uses `WITHOUT ROWID` for compact storage.
Key design decisions:
- **WAL mode** enables concurrent reads from workers while the main process writes.
- **`busy_timeout=5000`** plus a retry loop with `Atomics.wait`-based `sleepSync` handles `SQLITE_BUSY` errors from concurrent access.
- **Performance PRAGMAs**: `synchronous=NORMAL`, `mmap_size=512MB`, `cache_size=32MB`, `temp_store=MEMORY`, `wal_autocheckpoint=10000`.
- **Write batching**: `queueWrites()` batches pre-packed entries from tarball extraction and flushes them in a single transaction on `process.nextTick`. `setRawMany()` writes immediate batches (e.g. from `addFilesFromDir`).
- **Lifecycle**: `close()` auto-flushes pending writes, runs `PRAGMA optimize`, and closes the DB. A `process.on('exit')` handler ensures cleanup even on unexpected exits.
- **`VACUUM` after `deleteMany`** (used by `pnpm store prune`) to reclaim disk space.
### Key format
Keys are `integrity\tpkgId` (tab-separated). Git-hosted packages use `pkgId\tbuilt` or `pkgId\tnot-built`.
### Shared StoreIndex instance
A single `StoreIndex` instance is threaded through the entire install lifecycle — from `createNewStoreController` through the fetcher chain, package requester, license scanner, SBOM collector, and dependencies hierarchy. This replaces the previous pattern of each component creating its own file-based index access.
### Worker architecture
Index writes are performed in the main process, not in worker threads. Workers send pre-packed `{ key, buffer }` pairs back to the main process via `postMessage`, where they are batched and flushed to SQLite. This avoids SQLite write contention between threads.
### SQLite ExperimentalWarning suppression
`node:sqlite` emits an `ExperimentalWarning` on first load. This is suppressed via a `process.emitWarning` override injected through esbuild's `banner` option, which runs on line 1 of both `dist/pnpm.mjs` and `dist/worker.js` — before any module that loads `node:sqlite`.
### No migration from `.mpk` files
Old `.mpk` index files are not migrated. Packages missing from the new SQLite index are re-fetched on demand (the same behavior as a fresh store).
## Changed packages
121 files changed across these areas:
- **`store/index/`** — New `@pnpm/store.index` package
- **`worker/`** — Write batching moved from worker module into `StoreIndex` class; workers send pre-packed buffers to main process
- **`store/package-store/`** — StoreIndex creation and lifecycle management
- **`store/cafs/`** — Removed `getFilePathInCafs` index-file utilities (no longer needed)
- **`store/pkg-finder/`** — Reads from StoreIndex instead of `.mpk` files
- **`store/plugin-commands-store/`** — `store status` uses StoreIndex
- **`store/plugin-commands-store-inspecting/`** — `cat-index` and `find-hash` use StoreIndex
- **`fetching/tarball-fetcher/`** — Threads StoreIndex through fetchers; git-hosted fetcher flushes before reading
- **`fetching/git-fetcher/`, `binary-fetcher/`, `pick-fetcher/`** — Accept StoreIndex parameter
- **`pkg-manager/`** — `client`, `core`, `headless`, `package-requester` thread StoreIndex
- **`reviewing/`** — `license-scanner`, `sbom`, `dependencies-hierarchy` accept StoreIndex
- **`cache/api/`** — Cache view uses StoreIndex
- **`pnpm/bundle.ts`** — esbuild banner for ExperimentalWarning suppression
## Test plan
- [x] `pnpm --filter @pnpm/store.index test` — Unit tests for StoreIndex CRUD and batching
- [x] `pnpm --filter @pnpm/package-store test` — Store controller lifecycle
- [x] `pnpm --filter @pnpm/package-requester test` — Package requester reads from SQLite index
- [x] `pnpm --filter @pnpm/tarball-fetcher test` — Tarball and git-hosted fetcher writes
- [x] `pnpm --filter @pnpm/headless test` — Headless install
- [x] `pnpm --filter @pnpm/core test` — Core install, side effects, patching
- [x] `pnpm --filter @pnpm/plugin-commands-rebuild test` — Rebuild reads from index
- [x] `pnpm --filter @pnpm/license-scanner test` — License scanning
- [x] e2e tests pass
🤖 Generated with [Claude Code](https://claude.com/claude-code)
This PR overhauls `pnpm env` use to route through pnpm's own install machinery instead of maintaining a parallel code path with manual symlink/shim/hardlink logic.
```
pnpm env use -g <version>
```
now runs:
```
pnpm add --global node@runtime:<version>
```
via `@pnpm/exec.pnpm-cli-runner`. All manual symlink, hardlink, and cmd-shim code in `envUse.ts` is gone (~1000 lines removed across the package).
### Changes
**npm and npx shims on all platforms**
Added `getNodeBinsForCurrentOS(platform)` to `@pnpm/constants`, returning a `Record<string, string>` with the correct relative paths for `node`, `npm`, and `npx` inside a Node.js distribution. `BinaryResolution.bin` is widened from `string` to `string | Record<string, string>` in `@pnpm/resolver-base` and `@pnpm/lockfile.types`, so the node resolver can set all three entries and pnpm's bin-linker creates shims for each automatically.
**Windows npm/npx fix**
`addFilesFromDir` was skipping root-level `node_modules/` (to avoid storing a package's own dependencies), which stripped the bundled `npm` from Node.js Windows zip archives. Added an `includeNodeModules` option and enabled it from the binary fetcher so Windows distributions keep their full contents.
**Removed subcommands**
`pnpm env add` and `pnpm env remove` are removed. `pnpm env use` handles both installing and activating a version. `pnpm env list` now always lists remote versions (the `--remote` flag is no longer required, though it is kept for backwards compatibility).
**musl support**
On Alpine Linux and other musl-based systems, the musl variant of Node.js is automatically downloaded from [unofficial-builds.nodejs.org](https://unofficial-builds.nodejs.org).
