mirror/pnpm
1
0
Fork 0
mirror of https://github.com/pnpm/pnpm.git synced 2026-06-29 18:35:18 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat: add --frozen-store for installs against a read-only store (#12190)

Browse Source 
## What

Adds an opt-in `frozenStore` / `--frozen-store` setting (default `false`) that lets `pnpm install --offline --frozen-lockfile` run against a package store that lives on a **read-only filesystem** — a Nix store, a read-only bind mount, an OCI layer.

## Why

A normal install fails against such a store **not** because it writes package content, but because it unconditionally:

1. opens the SQLite `index.db` in WAL mode, which needs to create `-shm`/`-wal` sidecars in the store directory; and
2. writes a project-registry entry under the store.

Both fail with `attempt to write a readonly database` / `EROFS` on a read-only store directory, even when the store is complete and the lockfile is frozen. This blocks any deployment that wants an immutable, content-addressed store.

## How

When `frozenStore` is enabled, pnpm opens `index.db` through the SQLite **`immutable=1`** URI — which tells SQLite the file cannot change underneath it, so it bypasses the WAL/`-shm` sidecar machinery entirely and reads the raw file with zero sidecar creation — and suppresses every store-write path. Pair it with `--offline --frozen-lockfile` against a fully-populated store. It is incompatible with two settings that would write into the store, and each throws a clear config-conflict error before any network or store access: **`--force`** (which bypasses the no-write-on-hit skip → `ERR_PNPM_CONFIG_CONFLICT_FROZEN_STORE_WITH_FORCE`) and a configured **pnpr server** (which would fetch and write packages into the store → `ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR`).

> Plain `SQLITE_OPEN_READ_ONLY` is **not** sufficient: opening a WAL-mode db read-only still tries to create the `-shm` sidecar, which fails on a read-only *directory*. `immutable=1` is the load-bearing piece.

> **Node.js requirement:** `node:sqlite` only passes `SQLITE_OPEN_URI` to SQLite (so the `immutable=1` query is honored rather than treated as part of a literal filename) starting in **v22.15.0** (22.x line), **v23.11.0**, and every **v24+**. pnpm's `engines` floor is `>=22.13`, so on a runtime older than that the frozen open is detected up front and fails with a clear `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` instead of SQLite's cryptic "unable to open database file". (pacquet uses rusqlite with an explicit `SQLITE_OPEN_URI` flag, so it has no such floor.)

> The `immutable=1` URI path is also percent-encoded (`%`→`%25`, `?`→`%3f`, `#`→`%23`, in that order, leaving `/` literal) so a store path containing those characters doesn't truncate the path or inject a spurious query parameter — applied identically in both stacks.

### Build backstop under the global virtual store

Under the global virtual store (default), a package's directory lives **inside** the store (`{storeDir}/links/...`). Applying a patch or running an allowlisted lifecycle script writes into that directory — so on a frozen store it would crash mid-build with a raw `EROFS`. A fully-seeded store never reaches the build step (patched/built packages are imported from the side-effects cache and filtered out by the `isBuilt` gate), so any residual build candidate means the seed is **missing that package's build output**.

`buildModules` now refuses up front with `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` and actionable guidance ("rebuild the seed with their scripts enabled, or remove them from `onlyBuiltDependencies`") instead of failing cryptically once a script starts. The check is gated on the global virtual store — under the isolated linker, slot directories live in the writable project-local store, so builds there are fine. Non-allowlisted scripts never run, so they are not treated as a blocking write.

Bin-linking has its own read-only-store edge under the global virtual store. On a **warm** checkout (the project's `.bin/<name>` already points at the seed target) `linkBin` returns before touching the store, so it is write-free. But on a **fresh** checkout it (re)creates the bin and calls `fixBin`, whose `chmod` targets the bin's **source file inside the store** (`{storeDir}/links/...`) — which is refused with `EPERM`/`EACCES` on a read-only store, even though a complete seed already ships that bin executable (so the `chmod` is redundant). `@pnpm/bins.linker` now wraps that call in `ensureExecutable`: it swallows the refusal when the target is already executable and rethrows otherwise, so bin-linking is write-free against a frozen store on a cold checkout too, while a genuinely non-executable bin (a broken seed) still surfaces as an error.

The blocking predicate distinguishes the two write kinds: a **patch** is applied regardless of `ignoreScripts`, so a patched package is always blocked; a **lifecycle script** is suppressed under `ignoreScripts`, so an allowlisted build-requiring package is *not* blocked when scripts are off (it would write nothing). This avoids falsely rejecting a valid `--ignore-scripts` frozen install. **Optional dependencies are exempt**: a build or patch failure on an optional dependency is non-fatal at runtime, so a seed missing an optional package's build output skips that build (emitting the `skipped-optional-dependency` log) instead of blocking the install — in both stacks.

### Both stacks (parity rule)

**TypeScript pnpm CLI**
- Config plumbing (`@pnpm/config.reader`): `frozen-store` type, config-file key, default, `Config.frozenStore`.
- The read-only open branch (`@pnpm/store.index`): `immutable=1`, read-only statements, throwing mutators.
- Wiring through `@pnpm/store.controller` and `@pnpm/store.connection-manager` to the sole `StoreIndex` construction site.
- Gating the project-registry write (`@pnpm/installing.context`).
- The `--force` / pnpr-server conflict guards (`@pnpm/installing.deps-installer`) and CLI surface (`@pnpm/installing.commands`).
- **After-install rebuild** (`@pnpm/building.after-install`): the post-install rebuild opens its `StoreIndex` immutably under the flag, so re-reading the store for a rebuild never attempts a writable open against the frozen store.
- **Worker fix:** `@pnpm/worker` opens its *own* writable `StoreIndex` on every `readPkgFromCafs` cache hit, so a pure read crashed on a frozen store. `frozenStore` is threaded through to `getStoreIndex` and keyed into its connection cache.
- **Build backstop** (`@pnpm/building.during-install`): `buildModules` throws `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` for a GVS slot that would build/patch on a frozen store, honoring `ignoreScripts`; threaded from `@pnpm/installing.deps-installer`.
- **Bin-linking on a read-only store** (`@pnpm/bins.linker`): `linkBin` wraps the `fixBin` chmod in `ensureExecutable`, which tolerates `EPERM`/`EACCES` when the bin's store-resident source is already executable (a complete seed) and rethrows otherwise — so a fresh checkout against a frozen store links bins without crashing on the redundant chmod. Catch-on-failure keeps the writable hot path at zero added syscalls.

**Rust pacquet**
- A dedicated `open_immutable` / `shared_immutable_in` opens via `immutable=1`, selected only under the flag. Plain `open_readonly` keeps the ordinary `SQLITE_OPEN_READ_ONLY` open (WAL locking intact) because normal installs read the index while the same process's `StoreIndexWriter` writes it concurrently — an immutable connection skips all locking and change detection, so a concurrent writer would make those reads undefined.
- `--frozen-store` CLI flag + `frozenStore` workspace-yaml setting.
- The store-index writer is replaced with a drain-and-drop stub (`spawn_disabled`) and `init_store_dir_best_effort` is skipped under the flag.
- **Build backstop:** `build_modules` returns `BuildModulesError::FrozenStoreNeedsBuild` (`ERR_PNPM_FROZEN_STORE_NEEDS_BUILD`) under the same GVS + frozen-store condition, threaded from `config.frozen_store`. The gate keys off `should_run_scripts` (which already folds the allow-build policy), so it is correct without an explicit ignore-scripts branch — pacquet has no configurable ignore-scripts mode yet.

pacquet already separated read-only index access (`shared_readonly_in`, or `shared_immutable_in` under the flag) from writes, so it never had the worker-conflation bug; the flag makes the "no writes attempted" contract explicit and gates the remaining best-effort write attempts.

## Testing

- **TS:** `store.index` frozen-mode-on-`0555`-directory test (reads work, writes throw `ERR_PNPM_FROZEN_STORE_WRITE`) plus a path-with-`?` open test — both gated on the runtime's immutable-URI support, with a complementary test asserting `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` fires where that support is absent (the CI Node 22.13.0 path); `config.reader` round-trip; `deps-installer` `--force` and pnpr-server conflict guards; `worker`/`package-requester` unit tests. End-to-end on a `chmod -R 0555` store: install succeeds, `node_modules` materializes, no `-shm`/`-wal`/`-journal` sidecars; negative control without the flag fails as expected; incomplete store → clean offline error.
- **pacquet:** `open_immutable_reads_wal_db_on_readonly_directory` unit test plus the `immutable_sqlite_uri` encoding test and a path-with-`?` open test; yaml + CLI fold tests; **integration test** `frozen_store_installs_against_a_read_only_store` — primes a store, `chmod 0555` the tree, runs `install --frozen-lockfile --frozen-store --offline`, asserts success + materialized `node_modules` + zero sidecars. Confirmed load-bearing by reverting the `immutable=1` fix (test then fails).
- **Build backstop (both stacks):** `building/during-install` unit tests — approved-build-not-cached and patched-not-cached refuse; cached, non-allowlisted, and non-GVS cases pass through; **approved-build-under-`ignoreScripts` passes through while patched-under-`ignoreScripts` still refuses** — and the matching pacquet `build_modules` tests (`frozen_store_gvs_patch_not_seeded_refuses` + GVS-off / frozen-off controls). Each confirmed load-bearing by disabling the relevant guard and watching the corresponding test fail.
- **Bin-linking (`bins/linker`):** `ensureExecutable` tests with `fixBin` mocked to reject with `EPERM` — an already-executable bin source resolves (and `fixBin` is asserted called, so it isn't the warm skip-guard passing), a non-executable one rethrows `EPERM`. Confirmed end-to-end by running the built `linkBins` against a real `chflags uchg`-immutable store: the executable-seed case resolves and links the bin, the non-executable-seed control throws `EPERM`.

A changeset is included with `"pnpm": minor` and `"@pnpm/bins.linker": patch` (the read-only-store bin-linking fix).
This commit is contained in:
Victor Sumner
2026-06-12 02:36:08 -04:00
committed by GitHub
parent a31faa7c19
commit 61810aa684
50 changed files with 1546 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
.changeset/frozen-store.md Normal file
View File

@@ -0,0 +1,18 @@
---
"@pnpm/config.reader": minor
"@pnpm/store.index": minor
"@pnpm/store.controller": minor
"@pnpm/store.connection-manager": minor
"@pnpm/building.after-install": patch
"@pnpm/building.during-install": patch
"@pnpm/bins.linker": patch
"@pnpm/resolving.npm-resolver": patch
"@pnpm/worker": minor
"@pnpm/installing.package-requester": minor
"@pnpm/installing.context": patch
"@pnpm/installing.deps-installer": minor
"@pnpm/installing.commands": minor
"pnpm": minor
---
Added a new setting `frozenStore` (`--frozen-store`) that lets `pnpm install` run against a package store on a read-only filesystem (e.g. a Nix store, a read-only bind mount, an OCI layer). When enabled, pnpm opens the store's SQLite `index.db` through the `immutable=1` URI — bypassing the WAL/`-shm` sidecar creation that otherwise fails on a read-only directory — and suppresses every store-write path (the `index.db` writer and the project-registry write). Pair it with `--offline --frozen-lockfile` against a fully-populated store. Under the global virtual store, package directories live inside the store, so if the store is missing the build output of a package whose lifecycle scripts are approved (or that has a patch), pnpm fails up front with `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` rather than crashing mid-build on a read-only write — seed the store with those builds first. Incompatible with `--force` and with a configured pnpr server, since both write into the store; the side-effects cache is likewise not written under `frozenStore`. If the store is missing its content directory, the install fails fast with `ERR_PNPM_FROZEN_STORE_INCOMPLETE` rather than attempting to initialize it. The read-only `immutable=1` open requires Node.js >=22.15.0, >=23.11.0, or >=24.0.0; on older runtimes `--frozen-store` fails with a clear `ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE` error. Bin-linking also tolerates a read-only store: under the global virtual store a package's bin source lives inside the store, so the `chmod` that makes it executable would be refused — with `EPERM`/`EACCES`, or with `EROFS` on a genuinely read-only filesystem. That `chmod` is redundant when the seed already ships its bins executable with a normalized shebang, so it is now skipped in that case, while a non-executable bin (or one still carrying a Windows CRLF shebang) on a read-only store still errors.

1
Cargo.lock generated
View File

@@ -4344,6 +4344,7 @@ dependencies = [
"tempfile",
"tokio",
"tracing",
"url",
]
[[package]]

1
Cargo.toml
View File

@@ -152,6 +152,7 @@ tower-http = { version = "0.6.11", features = ["compression-gzip", "trac
tracing = { version = "0.1.44" }
tracing-subscriber = { version = "0.3.23", features = ["env-filter"] }
tokio = { version = "1", features = ["rt", "rt-multi-thread", "macros", "fs", "io-util", "net", "signal", "sync"] }
url = { version = "2.5.8" }
walkdir = { version = "2.5.0" }
yaml_serde = { version = "0.10.4" }
yamlpatch = { version = "1.25.2" }

45
bins/linker/src/index.ts
View File

@@ -310,7 +310,7 @@ async function linkBin (cmd: CommandInfo, binsDir: string, opts?: LinkBinOptions
if (opts?.preferSymlinkedExecutables && !IS_WINDOWS && cmd.nodeExecPath == null) {
try {
await symlinkDir(cmd.path, externalBinPath)
await fixBin(cmd.path, 0o755)
await ensureExecutable(cmd.path, 0o755)
} catch (err: any) { // eslint-disable-line
if (err.code !== 'ENOENT' && err.code !== 'EISDIR') {
throw err
@@ -357,7 +357,48 @@ async function linkBin (cmd: CommandInfo, binsDir: string, opts?: LinkBinOptions
// ensure that bin are executable and not containing
// windows line-endings(CRLF) on the hashbang line
if (EXECUTABLE_SHEBANG_SUPPORTED) {
await fixBin(cmd.path, 0o755)
await ensureExecutable(cmd.path, 0o755)
}
}
// `fixBin` chmods the bin's source file (which lives inside the store) to make
// it executable and rewrites a Windows CRLF shebang to LF. Under the global
// virtual store that source is `{storeDir}/links/...`, so on a read-only store
// (e.g. `frozenStore`) the chmod is refused — with EPERM/EACCES when the file is
// owned but permissions forbid it, or EROFS on a genuinely read-only filesystem
// (Nix store, RO bind mount, OCI layer). A complete seed already ships its bins
// executable and shebang-normalized by the writable seed-build, so that work is
// redundant: treat an already-correct target as a no-op, keeping bin-linking
// write-free (see building/during-install: "Bin-linking reuses existing symlinks
// write-free"). A non-executable bin — or one still carrying a CRLF shebang that
// `fixBin` could not rewrite here — still throws, because that means the seed is
// broken and the bin would not run.
async function ensureExecutable (file: string, mode: number): Promise<void> {
try {
await fixBin(file, mode)
} catch (err: any) { // eslint-disable-line
if (err.code === 'EPERM' || err.code === 'EACCES' || err.code === 'EROFS') {
const stat = await fs.stat(file).catch(() => undefined)
if (stat != null && (stat.mode & 0o111) !== 0 && !(await hasWindowsShebang(file))) return
}
throw err
}
}
// Detects a `#!`-shebang line terminated by CRLF, which fails to execute on
// POSIX. Mirrors bin-links' own fix-bin detection so a chmod failure on a
// read-only store is only swallowed when the bin is genuinely already correct.
async function hasWindowsShebang (file: string): Promise<boolean> {
const fh = await fs.open(file, 'r').catch(() => undefined)
if (fh == null) return false
try {
const buf = Buffer.alloc(2048)
await fh.read(buf, 0, 2048, 0)
return buf[0] === 0x23 /* # */ && buf[1] === 0x21 /* ! */ && /^#![^\n]+\r\n/.test(buf.toString())
} catch {
return false
} finally {
await fh.close().catch(() => {})
}
}

104
bins/linker/test/ensureExecutable.ts Normal file
View File

@@ -0,0 +1,104 @@
/// <reference path="../../../__typings__/index.d.ts"/>
import fs from 'node:fs'
import path from 'node:path'
import { beforeEach, expect, jest, test } from '@jest/globals'
import { fixtures } from '@pnpm/test-fixtures'
import isWindows from 'is-windows'
import { temporaryDirectory } from 'tempy'
// `linkBins` calls `fixBin` to make a package's bin source executable. Under the
// global virtual store that source lives inside the (potentially read-only) store,
// so the chmod is refused with EPERM/EACCES. The linker wraps the call in
// `ensureExecutable`, which treats an already-executable target as a no-op (the
// chmod was redundant) but still surfaces the error for a genuinely
// non-executable bin. These tests drive that wrapper by forcing `fixBin` to throw.
const fixBinMock = jest.fn<(file: string, mode: number) => Promise<void>>()
jest.unstable_mockModule('bin-links/lib/fix-bin.js', () => ({
default: fixBinMock,
}))
jest.unstable_mockModule('@pnpm/logger', () => ({
logger: () => ({ debug: jest.fn() }),
globalWarn: jest.fn(),
}))
const { linkBins } = await import('@pnpm/bins.linker')
const f = fixtures(import.meta.dirname)
beforeEach(() => {
fixBinMock.mockReset()
})
// `fixBin` chmods the source file; on Windows there is no executable bit to assert
// against, so the read-only-store reasoning these tests cover does not apply.
const testOnPosix = isWindows() ? test.skip : test
testOnPosix('linkBins() tolerates EPERM from fixBin when the bin source is already executable', async () => {
const eperm = Object.assign(new Error('EPERM: operation not permitted, chmod'), { code: 'EPERM' })
fixBinMock.mockRejectedValue(eperm)
const binTarget = temporaryDirectory()
const fixture = f.prepare('simple-fixture')
const binSource = path.join(fixture, 'node_modules', 'simple', 'index.js')
// Mimic a complete seed: the bin already ships executable, so the refused chmod
// is redundant and must be swallowed.
fs.chmodSync(binSource, 0o755)
const warn = jest.fn()
await expect(linkBins(path.join(fixture, 'node_modules'), binTarget, { warn })).resolves.toBeDefined()
expect(fixBinMock).toHaveBeenCalledWith(binSource, 0o755)
expect(fs.existsSync(path.join(binTarget, 'simple'))).toBe(true)
})
testOnPosix('linkBins() rethrows EPERM from fixBin when the bin source is not executable', async () => {
const eperm = Object.assign(new Error('EPERM: operation not permitted, chmod'), { code: 'EPERM' })
fixBinMock.mockRejectedValue(eperm)
const binTarget = temporaryDirectory()
const fixture = f.prepare('simple-fixture')
const binSource = path.join(fixture, 'node_modules', 'simple', 'index.js')
// A broken seed: the bin is not executable, so the refused chmod is a real
// problem and must surface rather than be silently swallowed.
fs.chmodSync(binSource, 0o644)
const warn = jest.fn()
await expect(linkBins(path.join(fixture, 'node_modules'), binTarget, { warn })).rejects.toHaveProperty('code', 'EPERM')
})
testOnPosix('linkBins() tolerates EROFS from fixBin when the bin source is already executable', async () => {
// A genuinely read-only filesystem (the primary frozenStore target) refuses
// chmod with EROFS rather than EPERM/EACCES.
const erofs = Object.assign(new Error('EROFS: read-only file system, chmod'), { code: 'EROFS' })
fixBinMock.mockRejectedValue(erofs)
const binTarget = temporaryDirectory()
const fixture = f.prepare('simple-fixture')
const binSource = path.join(fixture, 'node_modules', 'simple', 'index.js')
fs.chmodSync(binSource, 0o755)
const warn = jest.fn()
await expect(linkBins(path.join(fixture, 'node_modules'), binTarget, { warn })).resolves.toBeDefined()
expect(fixBinMock).toHaveBeenCalledWith(binSource, 0o755)
expect(fs.existsSync(path.join(binTarget, 'simple'))).toBe(true)
})
testOnPosix('linkBins() rethrows a chmod failure when the bin still has a CRLF shebang', async () => {
// fixBin chmods *before* normalizing the shebang, so a chmod failure means the
// CRLF was never rewritten. An executable-but-CRLF bin would not run on POSIX,
// so the failure must surface even though the execute bit is set.
const erofs = Object.assign(new Error('EROFS: read-only file system, chmod'), { code: 'EROFS' })
fixBinMock.mockRejectedValue(erofs)
const binTarget = temporaryDirectory()
const fixture = f.prepare('simple-fixture')
const binSource = path.join(fixture, 'node_modules', 'simple', 'index.js')
fs.writeFileSync(binSource, '#!/usr/bin/env node\r\nconsole.log("hi")\n')
fs.chmodSync(binSource, 0o755)
const warn = jest.fn()
await expect(linkBins(path.join(fixture, 'node_modules'), binTarget, { warn })).rejects.toHaveProperty('code', 'EROFS')
})

1
building/after-install/src/extendBuildOptions.ts
View File

@@ -23,6 +23,7 @@ export type StrictBuildOptions = {
scriptsPrependNodePath: boolean | 'warn-only'
shellEmulator: boolean
skipIfHasSideEffectsCache?: boolean
frozenStore?: boolean
storeDir: string // TODO: remove this property
storeController: StoreController
force: boolean

13
building/after-install/src/index.ts
View File

@@ -34,7 +34,7 @@ import npa from '@pnpm/npm-package-arg'
import { safeReadPackageJsonFromDir } from '@pnpm/pkg-manifest.reader'
import type { PackageFilesIndex } from '@pnpm/store.cafs'
import { createStoreController } from '@pnpm/store.connection-manager'
import { pickStoreIndexKey, StoreIndex } from '@pnpm/store.index'
import { pickStoreIndexKey, ReadOnlyStoreIndex, StoreIndex } from '@pnpm/store.index'
import type {
DepPath,
IgnoredBuilds,
@@ -358,7 +358,16 @@ async function _rebuild (
return false
}
const builtDepPaths = new Set<string>()
const storeIndex = opts.skipIfHasSideEffectsCache ? new StoreIndex(opts.storeDir) : undefined
// This handle is read-only in practice (only `.get()` below); the
// side-effects upload writes through `storeController`, not here. Open it
// immutable under `frozenStore` so the read works against a read-only store
// — a writable open would fail creating the WAL/`-shm` sidecar there. The
// immutable open is gated on `frozenStore` because on a normal install the
// concurrent side-effects uploads mutate `index.db`, which immutable reads
// would not see.
const storeIndex = opts.skipIfHasSideEffectsCache
? (opts.frozenStore ? new ReadOnlyStoreIndex(opts.storeDir) : new StoreIndex(opts.storeDir))
: undefined
// Under GVS, packages live at `<globalVirtualStoreDir>/<hash>/node_modules/<name>`,
// not the classic virtualStoreDir layout. The hash is computed with the same inputs

69
building/during-install/src/index.ts
View File

@@ -53,6 +53,7 @@ export async function buildModules<T extends string> (
rootModulesDir: string
hoistedLocations?: Record<string, string[]>
enableGlobalVirtualStore?: boolean
frozenStore?: boolean
}
): Promise<{ ignoredBuilds?: IgnoredBuilds }> {
if (!rootDepPaths.length) return {}
@@ -76,6 +77,22 @@ export async function buildModules<T extends string> (
if (!chunks.length) return {}
const ignoredBuilds = new Set<DepPath>()
const allowBuild = opts.allowBuild ?? (() => undefined)
// Under the global virtual store a package's directory lives inside the store
// (`{storeDir}/links/...`), so applying a patch or running an allowlisted
// lifecycle script writes into it. On a read-only `frozenStore` that write
// would crash mid-build with a raw `EROFS`. A complete seed never reaches the
// build step — built and patched packages are imported from the side-effects
// cache with `isBuilt` set and filtered out just below — so any package still
// wanting to write means the seed is missing its build output. We collect
// those off the same filtered chunk and refuse up front (see
// `throwFrozenStoreNeedsBuild`) instead of failing cryptically once a script
// starts. Bin-linking reuses existing symlinks write-free, and non-allowlisted
// scripts never run, so neither counts as a blocking write. Optional
// dependencies don't block either — their build failures are non-fatal at
// runtime, so their builds are skipped instead.
const frozenStoreBlocked = (opts.frozenStore && opts.enableGlobalVirtualStore)
? new Set<string>()
: undefined
const groups = chunks.map((chunk) => {
chunk = chunk.filter((depPath) => {
const node = depGraph[depPath]
@@ -84,6 +101,36 @@ export async function buildModules<T extends string> (
if (opts.depsToBuild != null) {
chunk = chunk.filter((depPath) => opts.depsToBuild!.has(depPath))
}
if (frozenStoreBlocked != null) {
chunk = chunk.filter((depPath) => {
const node = depGraph[depPath]
// A patch is applied even under `ignoreScripts`, but a lifecycle script
// is not — so only the patch write counts as blocking when scripts are
// suppressed.
const willPatch = node.patch != null
const willRunScripts = !opts.ignoreScripts && Boolean(node.requiresBuild) && allowBuild(node.depPath) === true
if (!willPatch && !willRunScripts) return true
if (node.optional) {
// A build/patch failure on an optional dependency is non-fatal at
// runtime (see the catch in `buildDependency`), so a seed missing an
// optional package's build output skips that build instead of
// blocking the install.
skippedOptionalDependencyLogger.debug({
details: `The read-only store (frozenStore) is missing the build output of ${node.name}@${node.version}.`,
package: {
id: node.dir,
name: node.name,
version: node.version,
},
prefix: opts.lockfileDir,
reason: 'build_failure',
})
return false
}
frozenStoreBlocked.add(`${node.name}@${node.version}`)
return true
})
}
return chunk.map((depPath) =>
() => {
@@ -113,6 +160,9 @@ export async function buildModules<T extends string> (
}
)
})
if (frozenStoreBlocked?.size) {
throwFrozenStoreNeedsBuild(frozenStoreBlocked)
}
const patchErrors: Error[] = []
const groupsWithPatchErrors = groups.map((group) =>
group.map((task) => async () => {
@@ -134,6 +184,18 @@ export async function buildModules<T extends string> (
return { ignoredBuilds }
}
/** Refuse a build under a read-only global virtual store. See the call site. */
function throwFrozenStoreNeedsBuild (blocked: Set<string>): never {
const list = Array.from(blocked).sort()
throw new PnpmError(
'FROZEN_STORE_NEEDS_BUILD',
`Cannot build the following ${list.length === 1 ? 'package' : 'packages'} because the store is read-only (frozenStore is enabled): ${list.join(', ')}`,
{
hint: 'This read-only store was not seeded with these packages\' build output. Rebuild the seed with their scripts enabled so the side-effects cache is populated, or remove them from onlyBuiltDependencies.',
}
)
}
async function buildDependency<T extends string> (
depPath: T,
depGraph: DependenciesGraph<T>,
@@ -157,6 +219,7 @@ async function buildDependency<T extends string> (
hoistedLocations?: Record<string, string[]>
builtHoistedDeps?: Record<string, DeferredPromise<void>>
enableGlobalVirtualStore?: boolean
frozenStore?: boolean
/** Resolved `engines.runtime` Node version — see [`buildModules`]. */
nodeVersion?: string
warn: (message: string) => void
@@ -203,7 +266,11 @@ async function buildDependency<T extends string> (
if (opts.enableGlobalVirtualStore) {
await fs.unlink(path.join(depNode.dir, '.pnpm-needs-build')).catch(() => {})
}
if ((isPatched || hasSideEffects) && opts.sideEffectsCacheWrite) {
// frozenStore opens the store read-only, so the side-effects cache (which
// lives in the store) cannot be written. extendInstallOptions already forces
// sideEffectsCacheWrite off under frozenStore; this guards callers that
// bypass it.
if ((isPatched || hasSideEffects) && opts.sideEffectsCacheWrite && !opts.frozenStore) {
try {
const sideEffectsCacheKey = calcDepState(depGraph, opts.depsStateCache, depPath, {
patchFileHash: depNode.patch?.hash,

147
building/during-install/test/buildModules.test.ts Normal file
View File

@@ -0,0 +1,147 @@
import { expect, test } from '@jest/globals'
import type { DependenciesGraph } from '../lib/buildSequence.js'
import { buildModules } from '../lib/index.js'
const baseOpts = {
depsStateCache: {},
lockfileDir: '/project',
optional: true,
rootModulesDir: '/project/node_modules',
sideEffectsCacheWrite: false,
storeController: {} as never,
unsafePerm: false,
userAgent: 'pnpm',
}
interface NodeOverrides {
requiresBuild?: boolean
patch?: object
isBuilt?: boolean
optional?: boolean
}
function singlePkgGraph (depPath: string, overrides: NodeOverrides): DependenciesGraph<string> {
return {
[depPath]: {
children: {},
depPath,
name: 'foo',
version: '1.0.0',
dir: '/store/links/hash/node_modules/foo',
hasBin: false,
hasBundledDependencies: false,
optional: false,
optionalDependencies: new Set<string>(),
...overrides,
},
} as unknown as DependenciesGraph<string>
}
const allowFoo = (depPath: string): boolean => depPath === 'foo@1.0.0'
test('frozenStore + GVS: an approved build that is not cached refuses up front', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { requiresBuild: true, isBuilt: false }), ['foo@1.0.0'], {
...baseOpts,
allowBuild: allowFoo,
enableGlobalVirtualStore: true,
frozenStore: true,
})
).rejects.toMatchObject({
code: 'ERR_PNPM_FROZEN_STORE_NEEDS_BUILD',
})
})
test('frozenStore + GVS: a patched package that is not cached refuses up front', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { patch: { hash: 'h', path: '/p' }, isBuilt: false }), ['foo@1.0.0'], {
...baseOpts,
enableGlobalVirtualStore: true,
frozenStore: true,
})
).rejects.toMatchObject({
code: 'ERR_PNPM_FROZEN_STORE_NEEDS_BUILD',
})
})
test('frozenStore + GVS: an already-built (cached) package does not trip the backstop', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { requiresBuild: true, isBuilt: true }), ['foo@1.0.0'], {
...baseOpts,
allowBuild: allowFoo,
enableGlobalVirtualStore: true,
frozenStore: true,
})
).resolves.toBeDefined()
})
test('frozenStore + GVS: a build-requiring package that is not approved does not trip the backstop', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { requiresBuild: true, isBuilt: false }), ['foo@1.0.0'], {
...baseOpts,
allowBuild: () => false,
enableGlobalVirtualStore: true,
frozenStore: true,
ignoreScripts: true,
})
).resolves.toBeDefined()
})
test('frozenStore + GVS: an approved build under ignoreScripts is not blocked (the script never runs)', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { requiresBuild: true, isBuilt: false }), ['foo@1.0.0'], {
...baseOpts,
allowBuild: allowFoo,
enableGlobalVirtualStore: true,
frozenStore: true,
ignoreScripts: true,
})
).resolves.toBeDefined()
})
test('frozenStore + GVS: a patched package under ignoreScripts still refuses (the patch is applied regardless)', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { patch: { hash: 'h', path: '/p' }, isBuilt: false }), ['foo@1.0.0'], {
...baseOpts,
enableGlobalVirtualStore: true,
frozenStore: true,
ignoreScripts: true,
})
).rejects.toMatchObject({
code: 'ERR_PNPM_FROZEN_STORE_NEEDS_BUILD',
})
})
test('frozenStore + GVS: an optional approved build that is not cached is skipped, not blocked', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { requiresBuild: true, isBuilt: false, optional: true }), ['foo@1.0.0'], {
...baseOpts,
allowBuild: allowFoo,
enableGlobalVirtualStore: true,
frozenStore: true,
})
).resolves.toBeDefined()
})
test('frozenStore + GVS: an optional patched package that is not cached is skipped, not blocked', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { patch: { hash: 'h', path: '/p' }, isBuilt: false, optional: true }), ['foo@1.0.0'], {
...baseOpts,
enableGlobalVirtualStore: true,
frozenStore: true,
})
).resolves.toBeDefined()
})
test('frozenStore without GVS: an approved build is not blocked (builds write to the writable project store)', async () => {
await expect(
buildModules(singlePkgGraph('foo@1.0.0', { requiresBuild: true, isBuilt: false }), ['foo@1.0.0'], {
...baseOpts,
allowBuild: allowFoo,
enableGlobalVirtualStore: false,
frozenStore: true,
ignoreScripts: true,
})
).resolves.toBeDefined()
})

1
config/reader/src/Config.ts
View File

@@ -176,6 +176,7 @@ export interface Config extends OptionsFromRootManifest {
virtualStoreOnly?: boolean
enableGlobalVirtualStore?: boolean
verifyStoreIntegrity?: boolean
frozenStore?: boolean
maxSockets?: number
networkConcurrency?: number
fetchingConcurrency?: number

1
config/reader/src/configFileKey.ts
View File

@@ -22,6 +22,7 @@ export const pnpmConfigFileKeys = [
'fetch-warn-timeout-ms',
'fetch-min-speed-ki-bps',
'fetching-concurrency',
'frozen-store',
'git-checks',
'git-shallow-hosts',
'global-bin-dir',

1
config/reader/src/index.ts
View File

@@ -207,6 +207,7 @@ export async function getConfig (opts: {
userconfig: npmDefaults.userconfig,
'verify-deps-before-run': 'install',
'verify-store-integrity': true,
'frozen-store': false,
'workspace-concurrency': getDefaultWorkspaceConcurrency(),
'workspace-prefix': opts.workspaceDir,
'embed-readme': false,

1
config/reader/src/types.ts
View File

@@ -128,6 +128,7 @@ export const pnpmTypes = {
'use-stderr': Boolean,
'verify-deps-before-run': Boolean,
'verify-store-integrity': Boolean,
'frozen-store': Boolean,
'global-virtual-store-dir': String,
'virtual-store-dir': String,
'virtual-store-only': Boolean,

2
config/reader/test/index.ts
View File

@@ -2823,6 +2823,7 @@ describe('global config.yaml', () => {
useStderr: true,
verifyDepsBeforeRun: 'error',
verifyStoreIntegrity: false,
frozenStore: true,
virtualStoreDir: '/custom/.pnpm',
virtualStoreDirMaxLength: 80,
})
@@ -2850,6 +2851,7 @@ describe('global config.yaml', () => {
expect(config.useStderr).toBe(true)
expect(config.verifyDepsBeforeRun).toBe('error')
expect(config.verifyStoreIntegrity).toBe(false)
expect(config.frozenStore).toBe(true)
expect(config.virtualStoreDir).toBe('/custom/.pnpm')
expect(config.virtualStoreDirMaxLength).toBe(80)
expect(warnings.find((w) => w.includes('global config file'))).toBeUndefined()

3
cspell.json
View File

@@ -40,6 +40,7 @@
"canva",
"cerbos",
"certfile",
"chmods",
"clonedeep",
"cmds",
"Codeberg",
@@ -75,6 +76,7 @@
"dpkg",
"drivelist",
"duplexify",
"eacces",
"eagain",
"ebadplatform",
"ebusy",
@@ -95,6 +97,7 @@
"eotp",
"eperm",
"epipe",
"erofs",
"errcode",
"esac",
"etamponi",

5
installing/commands/src/install.ts
View File

@@ -75,6 +75,7 @@ export function rcOptionsTypes (): Record<string, unknown> {
'optional',
'unsafe-perm',
'verify-store-integrity',
'frozen-store',
'virtual-store-dir',
'virtual-store-only',
], allTypes)
@@ -221,6 +222,10 @@ by any dependencies, so it is an emulation of a flat node_modules',
description: 'If false, skips store integrity checks. These checks detect accidental corruption, not tampering by untrusted users with write access to the store',
name: '--[no-]verify-store-integrity',
},
{
description: 'Open the package store read-only (immutable) and skip all store writes. For installs against a store on a read-only filesystem (e.g. a Nix store); pair with --offline --frozen-lockfile. Incompatible with --force',
name: '--frozen-store',
},
{
description: 'Fail on missing or invalid peer dependencies',
name: '--strict-peer-dependencies',

18
installing/context/src/index.ts
View File

@@ -92,6 +92,7 @@ export interface GetContextOptions {
confirmModulesPurge?: boolean
force: boolean
frozenLockfile?: boolean
frozenStore?: boolean
enableGlobalVirtualStore?: boolean
extraBinPaths: string[]
extendNodePath?: boolean
@@ -122,10 +123,12 @@ export async function getContext (
const importersContext = await readProjectsContext(opts.allProjects, { lockfileDir: opts.lockfileDir, modulesDir })
const virtualStoreDir = pathAbsolute(opts.virtualStoreDir ?? path.join(modulesDir, '.pnpm'), opts.lockfileDir)
await fs.mkdir(opts.storeDir, { recursive: true })
if (!opts.frozenStore) {
await fs.mkdir(opts.storeDir, { recursive: true })
// Register this project for store prune tracking
await registerProject(opts.storeDir, opts.lockfileDir)
// Register this project for store prune tracking
await registerProject(opts.storeDir, opts.lockfileDir)
}
for (const project of opts.allProjects) {
packageManifestLogger.debug({
@@ -243,6 +246,7 @@ export async function getContextForSingleImporter (
excludeLinksFromLockfile: boolean
peersSuffixMaxLength: number
force: boolean
frozenStore?: boolean
confirmModulesPurge?: boolean
extraBinPaths: string[]
extendNodePath?: boolean
@@ -293,10 +297,12 @@ export async function getContextForSingleImporter (
const importerId = importer.id
const virtualStoreDir = pathAbsolute(opts.virtualStoreDir ?? 'node_modules/.pnpm', opts.lockfileDir)
await fs.mkdir(storeDir, { recursive: true })
if (!opts.frozenStore) {
await fs.mkdir(storeDir, { recursive: true })
// Register this project for store prune tracking
await registerProject(storeDir, opts.lockfileDir)
// Register this project for store prune tracking
await registerProject(storeDir, opts.lockfileDir)
}
const extraBinPaths = [
...opts.extraBinPaths || [],
]

14
installing/deps-installer/src/install/extendInstallOptions.ts
View File

@@ -37,6 +37,7 @@ export interface StrictInstallOptions {
cleanupUnusedCatalogs: boolean
frozenLockfile: boolean
frozenLockfileIfExists: boolean
frozenStore: boolean
enableGlobalVirtualStore: boolean
enablePnp: boolean
extraBinPaths: string[]
@@ -289,6 +290,7 @@ const defaults = (opts: InstallOptions): StrictInstallOptions => {
force: false,
forceFullResolution: false,
frozenLockfile: false,
frozenStore: false,
hoistPattern: undefined,
publicHoistPattern: undefined,
hooks: {},
@@ -417,6 +419,18 @@ export function extendOptions (
`Cannot generate a ${WANTED_LOCKFILE} because lockfile is set to false`)
}
}
if (extendedOpts.frozenStore && extendedOpts.force) {
throw new PnpmError('CONFIG_CONFLICT_FROZEN_STORE_WITH_FORCE',
'Cannot use force together with frozenStore: --force re-imports packages into the store, which is opened read-only when frozenStore is enabled')
}
if (extendedOpts.frozenStore) {
// The side-effects cache is written into the store, which frozenStore opens
// read-only. Caching is an optimization, not a correctness requirement, so
// force it off rather than failing (the writable seed-build already
// populated it). Without this, a build under frozenStore (e.g. with the
// global virtual store disabled) would attempt a store write.
extendedOpts.sideEffectsCacheWrite = false
}
if (extendedOpts.userAgent.startsWith('npm/')) {
extendedOpts.userAgent = `${extendedOpts.packageManager.name}/${extendedOpts.packageManager.version} ${extendedOpts.userAgent}`
}

14
installing/deps-installer/src/install/index.ts
View File

@@ -1648,6 +1648,7 @@ const _installInContext: InstallFunction = async (projects, ctx, opts) => {
unsafePerm: opts.unsafePerm,
userAgent: opts.userAgent,
enableGlobalVirtualStore: opts.enableGlobalVirtualStore,
frozenStore: opts.frozenStore,
})).ignoredBuilds
if (ctx.modulesFile?.ignoredBuilds?.size) {
ignoredBuilds ??= new Set()
@@ -2335,6 +2336,19 @@ async function installViaPnprServer (
opts: Opts,
allInstallProjects?: Array<{ rootDir: ProjectRootDir, manifest: ProjectManifest }>
): Promise<InstallResult & { stats: InstallationResultStats, lockfile: LockfileObject }> {
// The pnpr server path re-resolves and persists new `index.db` entries plus a
// freshly written lockfile, so it inherently writes the store. `frozenStore`
// promises the store is complete and read-only, so the two are mutually
// exclusive — and the unconditional pnpr gate means this path runs even under
// `--offline --frozen-lockfile`, so refuse up front with guidance instead of
// crashing later on the read-only `index.db` open.
if (opts.frozenStore) {
throw new PnpmError(
'FROZEN_STORE_INCOMPATIBLE_WITH_PNPR',
'The pnpr server resolves dependencies and writes new entries into the store, which is opened read-only when frozenStore is enabled.',
{ hint: 'Disable the pnpr server (unset `--pnpr-server` / `pnprServer` in pnpm-workspace.yaml) so the install reads from the existing store, or unset `frozenStore` to allow store writes.' }
)
}
// The pnpr server path skips client-side resolution, so resolver-side policies
// can't be enforced locally. `minimumReleaseAge` is forwarded to the
// pnpr server and enforced server-side. `trustPolicy` has no server-side

29
installing/deps-installer/test/install/frozenStore.ts Normal file
View File

@@ -0,0 +1,29 @@
import { expect, test } from '@jest/globals'
import { install } from '@pnpm/installing.deps-installer'
import { prepareEmpty } from '@pnpm/prepare'
import { testDefaults } from '../utils/index.js'
test('frozenStore together with force throws a config conflict error', async () => {
prepareEmpty()
await expect(
install({}, testDefaults({
frozenStore: true,
force: true,
}))
).rejects.toMatchObject({
code: 'ERR_PNPM_CONFIG_CONFLICT_FROZEN_STORE_WITH_FORCE',
})
})
test('frozenStore together with a configured pnpr server throws before any store write', async () => {
prepareEmpty()
await expect(
install({}, testDefaults({
frozenStore: true,
pnprServer: 'http://localhost:0',
}))
).rejects.toMatchObject({
code: 'ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR',
})
})

2
installing/package-requester/src/packageRequester.ts
View File

@@ -86,6 +86,7 @@ export function createPackageRequester (
virtualStoreDirMaxLength: number
strictStorePkgContentCheck?: boolean
customFetchers?: CustomFetcher[]
frozenStore?: boolean
}
): RequestPackageFunction & {
fetchPackageToStore: FetchPackageToStoreFunction
@@ -108,6 +109,7 @@ export function createPackageRequester (
storeDir: opts.storeDir,
verifyStoreIntegrity: opts.verifyStoreIntegrity,
strictStorePkgContentCheck: opts.strictStorePkgContentCheck,
frozenStore: opts.frozenStore,
})
const fetchPackageToStore = fetchToStore.bind(null, {
readPkgFromCafs,

1
pacquet/crates/cli/src/cli_args.rs
View File

@@ -243,6 +243,7 @@ impl CliArgs {
let cfg = config()?;
cfg.offline = cfg.offline || args.offline;
cfg.prefer_offline = cfg.prefer_offline || args.prefer_offline;
cfg.frozen_store = cfg.frozen_store || args.frozen_store;
cfg.workspace_concurrency =
args.resolve_workspace_concurrency(cfg.workspace_concurrency);
// Network overrides: a passed `--network-concurrency` /

44
pacquet/crates/cli/src/cli_args/install.rs
View File

@@ -1,6 +1,7 @@
use crate::{State, cli_args::supported_architectures::SupportedArchitecturesArgs};
use clap::{Args, ValueEnum};
use miette::Context;
use derive_more::{Display, Error};
use miette::{Context, Diagnostic};
use pacquet_config::NodeLinker;
use pacquet_lockfile::{Lockfile, LockfileResolution};
use pacquet_package_manager::{Install, TarballPrefetcher, UpdateSeedPolicy};
@@ -165,6 +166,18 @@ pub struct InstallArgs {
#[clap(long)]
pub offline: bool,
/// Open the package store read-only (immutable) and skip all store
/// writes. For installs against a store on a read-only filesystem
/// (e.g. a Nix store); pair with `--offline --frozen-lockfile`.
/// Mirrors pnpm's `--frozen-store`. Overrides `frozenStore` from
/// `pnpm-workspace.yaml`: any `--frozen-store` upgrades a yaml
/// `false` to `true`, but cannot turn an explicit yaml `true` back
/// off. (pnpm additionally rejects `--frozen-store` combined with
/// `--force`; pacquet has no `force` flow yet, so there is nothing
/// to conflict with — the guard ports alongside `force`.)
#[clap(long = "frozen-store")]
pub frozen_store: bool,
/// Prefer cached artifacts over network fetches when both have
/// what's needed. Mirrors pnpm's
/// [`--prefer-offline`](https://github.com/pnpm/pnpm/blob/94240bc046/resolving/npm-resolver/src/pickPackage.ts)
@@ -253,6 +266,9 @@ impl InstallArgs {
no_runtime,
node_linker,
offline: _,
// Read from `config.frozen_store` (the CLI flag was already
// merged in by the dispatch in `cli_args.rs`), not from here.
frozen_store: _,
prefer_offline: _,
trust_lockfile,
update_checksums,
@@ -426,6 +442,22 @@ struct PnprLink<'a> {
lockfile_path: Option<&'a std::path::Path>,
}
/// `frozenStore` was enabled together with a configured `pnprServer`.
/// The pnpr path writes resolved files into the store, which `frozenStore`
/// opens read-only, so the combination can't proceed. Mirrors pnpm's
/// `ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR`.
#[derive(Debug, Display, Error, Diagnostic)]
#[display(
"The pnpr server resolves dependencies and writes new entries into the store, which is opened read-only when frozenStore is enabled."
)]
#[diagnostic(
code(ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR),
help(
"Disable the pnpr server (unset `--pnpr-server` / `pnprServer` in pnpm-workspace.yaml) so the install reads from the existing store, or unset `frozenStore` to allow store writes."
)
)]
struct FrozenStoreIncompatibleWithPnpr;
/// Resolve a single project through a `pnpr` server, then link it.
///
/// Sends the client's registries to the server, which resolves against
@@ -441,6 +473,16 @@ async fn install_via_pnpr<Reporter: self::Reporter + 'static>(
pnpr_server: &str,
link: PnprLink<'_>,
) -> miette::Result<()> {
// The pnpr server resolves dependencies and streams missing files
// straight into the store, so this path inherently writes the store.
// `frozenStore` promises the store is complete and read-only, so the
// two are mutually exclusive — refuse up front instead of failing on
// the read-only write. Mirrors pnpm's `FROZEN_STORE_INCOMPATIBLE_WITH_PNPR`
// guard in `installFromPnpmRegistry`.
if state.config.frozen_store {
return Err(FrozenStoreIncompatibleWithPnpr.into());
}
let dependencies = state
.manifest
.dependencies([DependencyGroup::Prod])

14
pacquet/crates/cli/src/cli_args/install/tests.rs
View File

@@ -129,6 +129,20 @@ fn ignore_manifest_check_flag_parses() {
assert!(parsed.args.ignore_manifest_check, "flag present → true");
}
/// `--frozen-store` parses to `true`. Absent → `false`. The flag is
/// folded into `config.frozen_store` at the dispatch in `cli_args.rs`
/// (any `--frozen-store` upgrades a yaml `false` to `true`), so the
/// install path reads the effective value off the config.
#[test]
fn frozen_store_flag_parses() {
let parsed = InstallArgsHarness::try_parse_from(["pacquet-test"]).expect("parses");
assert!(!parsed.args.frozen_store, "flag absent → false");
let parsed = InstallArgsHarness::try_parse_from(["pacquet-test", "--frozen-store"])
.expect("parses --frozen-store");
assert!(parsed.args.frozen_store, "flag present → true");
}
/// `--workspace-concurrency` is absent by default, so the override
/// is `None` and the config-resolved value stays in effect.
#[test]

153
pacquet/crates/cli/tests/install.rs
View File

@@ -804,6 +804,143 @@ fn frozen_install_short_circuits_when_node_modules_is_up_to_date() {
drop((root, mock_instance)); // cleanup
}
/// The reason `--frozen-store` exists: install against a package store that
/// lives on a read-only filesystem (a Nix store, a read-only bind mount, an
/// OCI layer). A complete store plus an up-to-date lockfile is all a
/// `--frozen-lockfile` install needs, yet the install would still fail
/// because opening the WAL-mode `index.db` tries to create `-wal`/`-shm`
/// sidecars in the store directory. `--frozen-store` opens the index through
/// the `immutable=1` URI ([`StoreIndex::open_immutable`]) and replaces the
/// store-index writer with a drain-and-drop stub
/// ([`StoreIndexWriter::spawn_disabled`]), so the install reads from the
/// store and materializes `node_modules` without creating a single file under
/// the (here `0555`) store root.
///
/// This is the Rust parallel to the TypeScript end-to-end coverage that
/// caught the equivalent worker-thread regression in pnpm
/// (`@pnpm/worker` opened its own *writable* `StoreIndex` on every cache hit).
/// pacquet has no analogous bug — frozen-store reads go through the immutable
/// [`StoreIndex::shared_immutable_in`] and every warm-path store write is
/// either gated under `frozenStore` or best-effort — so there is no clean
/// hard-fail negative control here; the load-bearing assertion is that the
/// install *succeeds* against a genuinely read-only store and mutates nothing.
#[cfg(unix)]
#[test]
fn frozen_store_installs_against_a_read_only_store() {
let CommandTempCwd { pacquet, root, workspace, npmrc_info, .. } =
CommandTempCwd::init().add_mocked_registry();
let AddMockedRegistry { store_dir, mock_instance, .. } = npmrc_info;
eprintln!("Creating package.json...");
let manifest_path = workspace.join("package.json");
let package_json = serde_json::json!({
"dependencies": {
"@pnpm.e2e/hello-world-js-bin-parent": "1.0.0",
},
});
fs::write(&manifest_path, package_json.to_string()).expect("write to package.json");
eprintln!("Priming the store and lockfile with a writable install...");
pacquet.with_arg("install").assert().success();
// Drop node_modules so the frozen run cannot take the up-to-date
// short-circuit — it must re-materialize from the store, which
// exercises the read path against the now read-only index.
eprintln!("Removing node_modules so the frozen install re-materializes...");
fs::remove_dir_all(workspace.join("node_modules")).expect("remove node_modules");
eprintln!("Making every directory in the store tree read-only (0555)...");
set_dir_modes(&store_dir, 0o555);
// The store root is `<store-dir>/v11` (the `STORE_VERSION` suffix), which
// is where `index.db` and the CAFS shards live.
let store_root = store_dir.join("v11");
// Guard: prove the chmod actually took. A green result below would be a
// false pass if the store dir were somehow still writable.
assert!(
fs::write(store_root.join("pacquet-write-probe"), b"x").is_err(),
"the store root must be read-only for this test to mean anything",
);
eprintln!("Running install --frozen-lockfile --frozen-store --offline...");
let output = new_pacquet_command(&workspace)
.with_args(["install", "--frozen-lockfile", "--frozen-store", "--offline"])
.output()
.expect("run pacquet install --frozen-store");
assert!(
output.status.success(),
"frozen-store install against a read-only store must succeed: stderr={}",
String::from_utf8_lossy(&output.stderr),
);
eprintln!("node_modules must be materialized from the read-only store...");
let symlink_path = workspace.join("node_modules/@pnpm.e2e/hello-world-js-bin-parent");
assert!(
is_symlink_or_junction(&symlink_path).expect("stat the dependency symlink"),
"the direct dependency must be linked into node_modules",
);
assert!(
workspace.join("node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0").exists(),
"the virtual-store entry must be present",
);
eprintln!("No WAL/SHM/journal sidecars may have been created under the store...");
for sidecar in ["index.db-wal", "index.db-shm", "index.db-journal"] {
assert!(
!store_root.join(sidecar).exists(),
"frozen-store must not create the {sidecar} sidecar under the read-only store",
);
}
// Restore writability so the TempDir can clean itself up — unlinking a
// file needs write permission on its *parent* directory.
set_dir_modes(&store_dir, 0o755);
drop((root, mock_instance)); // cleanup
}
/// `--frozen-store` with a configured `pnprServer` is a hard config conflict:
/// the pnpr path resolves and streams missing files straight into the store,
/// which `frozenStore` opens read-only. pacquet must refuse up front with
/// `ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR` (before any network), matching
/// pnpm's guard in `installFromPnpmRegistry`. The server URL points at a closed
/// port precisely to prove the guard fires before any connection is attempted.
#[test]
fn frozen_store_with_a_pnpr_server_is_a_config_conflict() {
let CommandTempCwd { pacquet, root, workspace, npmrc_info, .. } =
CommandTempCwd::init().add_mocked_registry();
let AddMockedRegistry { mock_instance, .. } = npmrc_info;
let manifest_path = workspace.join("package.json");
let package_json = serde_json::json!({
"dependencies": {
"@pnpm.e2e/hello-world-js-bin-parent": "1.0.0",
},
});
fs::write(&manifest_path, package_json.to_string()).expect("write to package.json");
let output = pacquet
.with_args(["install", "--frozen-store", "--pnpr-server", "http://127.0.0.1:0"])
.assert()
.failure();
let stderr = String::from_utf8_lossy(&output.get_output().stderr);
eprintln!("stderr={stderr}");
// The miette report hard-wraps the message and inserts box-drawing
// characters on wrapped lines; strip whitespace and those glyphs before
// substring-matching so wrap position can't make the assertion brittle.
let flattened: String = stderr
.chars()
.filter(|ch| !ch.is_whitespace() && !matches!(ch, '│' | '├' | '╰' | '─' | '▶' | '×'))
.collect();
assert!(
flattened.contains("ERR_PNPM_FROZEN_STORE_INCOMPATIBLE_WITH_PNPR"),
"stderr did not carry the frozen-store/pnpr conflict code: {stderr}",
);
drop((root, mock_instance)); // cleanup
}
/// `resolutionMode: highest` (the default) resolves a direct dependency
/// to the highest version satisfying its range. `@pnpm.e2e/foo`
/// publishes `100.0.0` and `100.1.0`; `^100.0.0` therefore lands on
@@ -994,3 +1131,19 @@ fn new_pacquet_command(workspace: &std::path::Path) -> std::process::Command {
.expect("find the pacquet binary")
.with_current_dir(workspace)
}
/// Recursively set `mode` on `path` and every directory beneath it. Children
/// are re-permissioned before their parent so each `read_dir` runs while the
/// directory is still traversable, which lets the same helper both lock a
/// tree down to `0555` and restore it to `0755`.
#[cfg(unix)]
fn set_dir_modes(path: &std::path::Path, mode: u32) {
use std::os::unix::fs::PermissionsExt;
for entry in fs::read_dir(path).expect("read directory while setting modes") {
let entry = entry.expect("read directory entry");
if entry.file_type().expect("stat directory entry").is_dir() {
set_dir_modes(&entry.path(), mode);
}
}
fs::set_permissions(path, fs::Permissions::from_mode(mode)).expect("set directory mode");
}

19
pacquet/crates/config/src/lib.rs
View File

@@ -828,6 +828,25 @@ pub struct Config {
#[default = true]
pub verify_store_integrity: bool,
/// Opt-in assertion that the package store is complete and will not
/// be written during this install — for running against a store on a
/// read-only filesystem (a Nix store, a read-only bind mount, an OCI
/// layer). When `true`, pacquet opens `index.db` through the
/// `immutable=1` URI (see `StoreIndex::open_immutable`) and suppresses
/// every store-write path: the batched `index.db` writer is replaced
/// with a drain-and-drop stub that never opens the DB, and
/// `init_store_dir_best_effort` is skipped so no directory creation is
/// attempted under the store root. Pair with `--offline
/// --frozen-lockfile` against a fully-populated store.
///
/// pnpm rejects `frozenStore` combined with `force` (force re-imports
/// packages into the store, which a read-only store cannot accept).
/// pacquet has no `force` flow yet, so there is no conflict to guard;
/// the guard ports alongside `force`.
///
/// Matches pnpm's `frozenStore` / `--frozen-store` (default `false`).
pub frozen_store: bool,
/// Whether to consult the side-effects cache
/// (`PackageFilesIndex.sideEffects`) when importing a package
/// and whether to populate it after a successful postinstall.

1
pacquet/crates/config/src/pnpm_default_parity.rs
View File

@@ -147,6 +147,7 @@ fn mapped_rows(cfg: &Config) -> Vec<(&'static str, Scalar)> {
("fetch-retry-maxtimeout", Int(cfg.fetch_retry_maxtimeout as i64)),
("fetch-retry-mintimeout", Int(cfg.fetch_retry_mintimeout as i64)),
("fetch-timeout", Int(cfg.fetch_timeout as i64)),
("frozen-store", Bool(cfg.frozen_store)),
(
"minimum-release-age",
Int(cfg.minimum_release_age.expect("pacquet defaults minimum-release-age to Some")

8
pacquet/crates/config/src/workspace_yaml.rs
View File

@@ -171,6 +171,12 @@ pub struct WorkspaceSettings {
pub resolve_peers_from_workspace_root: Option<bool>,
pub block_exotic_subdeps: Option<bool>,
pub verify_store_integrity: Option<bool>,
/// `frozenStore` from `pnpm-workspace.yaml`. Opens the store
/// read-only and suppresses every store write — see
/// [`Config::frozen_store`]. Default `false`.
///
/// [`Config::frozen_store`]: crate::Config::frozen_store
pub frozen_store: Option<bool>,
pub side_effects_cache: Option<bool>,
pub side_effects_cache_readonly: Option<bool>,
pub fetch_retries: Option<u32>,
@@ -722,7 +728,7 @@ impl WorkspaceSettings {
hoisting_limits, external_dependencies,
dedupe_peer_dependents, dedupe_peers, dedupe_direct_deps, dedupe_injected_deps,
strict_peer_dependencies,
resolve_peers_from_workspace_root, verify_store_integrity,
resolve_peers_from_workspace_root, verify_store_integrity, frozen_store,
block_exotic_subdeps,
link_workspace_packages,
inject_workspace_packages,

23
pacquet/crates/config/src/workspace_yaml/tests.rs
View File

@@ -1457,3 +1457,26 @@ fn script_shell_and_node_options_null_clears_inherited_value() {
assert_eq!(config.script_shell, None, "explicit null must clear the inherited shell");
assert_eq!(config.node_options, None, "explicit null must clear inherited NODE_OPTIONS");
}
/// `frozenStore` parses from `pnpm-workspace.yaml` as a camelCase
/// boolean and `apply_to` pushes it onto the `Config`. Defaults to
/// `false` when the key is absent, matching pnpm's `frozen-store`
/// default. Drives the read-only-store open path (`immutable=1`) and
/// the disabled `index.db` writer.
#[test]
fn parses_frozen_store_from_yaml_and_applies() {
// Absent → config default stays `false`.
let absent: WorkspaceSettings = serde_saphyr::from_str("hoist: true").unwrap();
assert_eq!(absent.frozen_store, None);
let mut config = Config::new();
assert!(!config.frozen_store, "frozen_store must default to false");
absent.apply_to(&mut config, Path::new("/irrelevant"));
assert!(!config.frozen_store, "absent frozenStore must leave the default in place");
// Explicit `true` parses and applies.
let enabled: WorkspaceSettings = serde_saphyr::from_str("frozenStore: true").unwrap();
assert_eq!(enabled.frozen_store, Some(true));
let mut config = Config::new();
enabled.apply_to(&mut config, Path::new("/irrelevant"));
assert!(config.frozen_store, "frozenStore: true must apply onto the config");
}

74
pacquet/crates/package-manager/src/build_modules.rs
View File

@@ -62,6 +62,25 @@ pub enum BuildModulesError {
#[error(source)]
source: rayon::ThreadPoolBuildError,
},
/// Under the global virtual store a package's directory lives
/// inside the store, so applying a patch or running an approved
/// lifecycle script writes into the store. `frozen_store` promises
/// the store is complete and read-only, so the build cannot run.
/// A complete seed never reaches here — patched and built packages
/// are imported from the side-effects cache and skipped by the
/// `is_built` gate — so this means the seed is missing build
/// output. Mirrors the TS `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD`
/// thrown from
/// [`building/during-install`](https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts).
#[display("Cannot build {package} because the store is read-only (frozenStore is enabled)")]
#[diagnostic(
code(ERR_PNPM_FROZEN_STORE_NEEDS_BUILD),
help(
"This read-only store was not seeded with this package's build output. Rebuild the seed with its scripts enabled so the side-effects cache is populated, or remove it from onlyBuiltDependencies."
)
)]
FrozenStoreNeedsBuild { package: String },
}
/// Build policy derived from `allowBuilds` and
@@ -374,6 +393,15 @@ pub struct BuildModules<'a> {
/// front by [`crate::LinkVirtualStoreBins`], and the script
/// executor adds that path itself.
pub gather_ancestor_bin_paths: bool,
/// Mirrors `config.frozen_store`. When `true` together with the
/// global virtual store, a snapshot that would apply a patch or
/// run an approved lifecycle script is refused with
/// [`BuildModulesError::FrozenStoreNeedsBuild`] before the write
/// is attempted — the store is read-only, so the build cannot run.
/// Has no effect under the isolated linker, whose slot directories
/// live in the writable project store.
pub frozen_store: bool,
}
impl BuildModules<'_> {
@@ -405,6 +433,7 @@ impl BuildModules<'_> {
skipped,
pkg_root_by_key,
gather_ancestor_bin_paths,
frozen_store,
} = self;
let Some(snapshots) = snapshots else { return Ok(Vec::new()) };
@@ -552,6 +581,7 @@ impl BuildModules<'_> {
&extra_env,
scripts_prepend_node_path,
unsafe_perm,
frozen_store,
)
})
})?;
@@ -603,6 +633,7 @@ fn build_one_snapshot<Reporter: self::Reporter>(
extra_env: &HashMap<String, String>,
scripts_prepend_node_path: ScriptsPrependNodePath,
unsafe_perm: bool,
frozen_store: bool,
) -> Result<(), BuildModulesError> {
let metadata_key = snapshot_key.without_peer();
// Look up against the peer-stripped key because patches are
@@ -727,6 +758,47 @@ fn build_one_snapshot<Reporter: self::Reporter>(
return Ok(());
}
let optional = snapshots.get(snapshot_key).is_some_and(|entry| entry.optional);
// Frozen-store backstop. Under the global virtual store the slot
// directory lives inside the read-only store, so applying a patch
// or running an approved lifecycle script (the two writes below)
// would fail with a raw `EROFS`. Refuse up front with guidance.
// We're past the `is_built` gate, so a cached build has already
// returned — reaching here means the seed is genuinely missing
// this package's build output. Mirrors the TS backstop in
// <https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts>.
// Bin-linking (the other write) reuses existing symlinks
// write-free on a complete seed, so only patch/script writes gate.
if frozen_store && layout.enable_global_virtual_store() && (has_patch || should_run_scripts) {
if optional {
// A build/patch failure on an optional dependency is non-fatal
// (see the lifecycle-script arm below), so a seed missing an
// optional package's build output skips that build instead of
// blocking the install.
Reporter::emit(&LogEvent::SkippedOptionalDependency(SkippedOptionalDependencyLog {
level: LogLevel::Debug,
details: Some(format!(
"The read-only store (frozenStore) is missing the build output of {name}@{version}.",
)),
package: SkippedOptionalPackage::Installed {
id: pkg_root_for_key(layout, pkg_root_by_key, snapshot_key).map_or_else(
|| snapshot_key.to_string(),
|dir| dir.to_string_lossy().into_owned(),
),
name,
version,
},
prefix: lockfile_dir.to_string_lossy().into_owned(),
reason: SkippedOptionalReason::BuildFailure,
}));
return Ok(());
}
return Err(BuildModulesError::FrozenStoreNeedsBuild {
package: format!("{name}@{version}"),
});
}
// Hoisted snapshots without a recorded `pkgRoot` (the walker
// dropped them — pre-skipped, optional skip, etc.) take the
// same exit as the isolated path's `!pkg_dir.exists()` skip.
@@ -747,8 +819,6 @@ fn build_one_snapshot<Reporter: self::Reporter>(
Vec::new()
};
let optional = snapshots.get(snapshot_key).is_some_and(|entry| entry.optional);
// Apply the patch before running postinstall hooks. Mirrors
// upstream at
// <https://github.com/pnpm/pnpm/blob/b4f8f47ac2/building/during-install/src/index.ts#L171-L178>:

160
pacquet/crates/package-manager/src/build_modules/tests.rs
View File

@@ -345,6 +345,7 @@ fn build_modules_collects_ignored_builds() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("run BuildModules");
@@ -417,6 +418,7 @@ fn build_modules_collects_ignored_builds_under_concurrency() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("run BuildModules under concurrency");
@@ -477,6 +479,7 @@ fn build_modules_excludes_explicit_deny_from_ignored() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("run BuildModules");
@@ -560,6 +563,7 @@ fn do_not_fail_on_optional_dep_with_failing_postinstall() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<RecordingReporter>()
.expect("optional build failure must NOT abort the install");
@@ -698,6 +702,7 @@ fn using_side_effects_cache_skips_rebuild() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<RecordingReporter>()
.expect("install must succeed when the cache hit skips the rebuild");
@@ -765,6 +770,7 @@ fn side_effects_cache_disabled_bypasses_the_gate() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect_err("with cache disabled, the failing postinstall must run and the install must fail");
@@ -825,6 +831,7 @@ fn fail_when_failing_postinstall_is_required() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect_err("required build failure must propagate");
@@ -832,6 +839,153 @@ fn fail_when_failing_postinstall_is_required() {
assert!(matches!(err, crate::build_modules::BuildModulesError::LifecycleScript(_)));
}
// --- frozen-store build backstop ---------------------------------------
/// A `pnpm-workspace.yaml`-shaped patch entry for one package. The
/// `patch_file_path` is left `None` because the frozen-store backstop
/// fires before any patch application — the entry only has to make
/// `has_patch` true and land the snapshot in the build sequence.
fn single_patch(key: &PackageKey) -> HashMap<PackageKey, pacquet_patching::ExtendedPatchInfo> {
HashMap::from([(
key.without_peer(),
pacquet_patching::ExtendedPatchInfo {
hash: "deadbeef".to_string(),
patch_file_path: None,
key: key.without_peer().to_string(),
},
)])
}
/// Build a global-virtual-store [`VirtualStoreLayout`] for a backstop
/// test. `snapshots: None` short-circuits [`VirtualStoreLayout::new`]
/// to an empty `gvs_suffixes` map, which is all the backstop needs —
/// it queries [`VirtualStoreLayout::enable_global_virtual_store`]
/// (true iff `gvs_suffixes.is_some()`) and fires before any slot-dir
/// lookup.
fn gvs_layout(dir: &Path) -> &'static VirtualStoreLayout {
let mut config = Config::new();
config.enable_global_virtual_store = true;
config.store_dir = dir.join("store").into();
config.global_virtual_store_dir = dir.join("store/links");
config.virtual_store_dir = dir.join("node_modules/.pacquet");
let config = config.leak();
Box::leak(Box::new(VirtualStoreLayout::new(config, None, None, None, None)))
}
/// Run [`BuildModules`] over a single patched `is-positive@1.0.0`
/// snapshot, varying only `layout`, `frozen_store`, and the snapshot's
/// `optional` flag — the three backstop inputs. No on-disk fixture: the
/// snapshot's slot never materializes, so on paths that skip the
/// backstop the run no-ops at the `!pkg_dir.exists()` guard rather than
/// touching the (absent) patch file.
fn frozen_backstop_run(
layout: &VirtualStoreLayout,
frozen_store: bool,
optional: bool,
) -> Result<Vec<String>, crate::build_modules::BuildModulesError> {
let pkg_key = key("is-positive", "1.0.0");
let snapshots =
HashMap::from([(pkg_key.clone(), SnapshotEntry { optional, ..SnapshotEntry::default() })]);
let patches = single_patch(&pkg_key);
let importers = root_importers(&[("is-positive", "1.0.0")]);
let policy = policy_from_specs([], false);
let modules_dir = tempdir().expect("create temp dir");
let lockfile_dir = tempdir().expect("create temp dir");
BuildModules {
layout,
modules_dir: modules_dir.path(),
lockfile_dir: lockfile_dir.path(),
snapshots: Some(&snapshots),
packages: None,
importers: &importers,
allow_build_policy: &policy,
side_effects_maps_by_snapshot: None,
engine_name: None,
side_effects_cache: false,
side_effects_cache_write: false,
store_dir: None,
store_index_writer: None,
patches: Some(&patches),
scripts_prepend_node_path: ScriptsPrependNodePath::Never,
unsafe_perm: true,
child_concurrency: 1,
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store,
}
.run::<SilentReporter>()
}
/// Positive: under the global virtual store, a patched package whose
/// build output is missing from the read-only store refuses up front
/// with `ERR_PNPM_FROZEN_STORE_NEEDS_BUILD` rather than crashing on a
/// raw `EROFS` when `apply_patch_to_dir` tries to write into the store.
/// Mirrors the TS unit test
/// `frozenStore + GVS: a patched package that is not cached refuses up front`.
#[test]
fn frozen_store_gvs_patch_not_seeded_refuses() {
let store_dir = tempdir().expect("create temp dir");
let layout = gvs_layout(store_dir.path());
let err = frozen_backstop_run(layout, true, false)
.expect_err("a missing patched build under a frozen GVS store must refuse up front");
assert!(
matches!(err, crate::build_modules::BuildModulesError::FrozenStoreNeedsBuild { .. }),
"expected FrozenStoreNeedsBuild, got {err:?}",
);
}
/// An optional patched snapshot must not block the install: a build or
/// patch failure on an optional dependency is non-fatal at runtime, so
/// the backstop skips its build (emitting the skipped-optional log)
/// instead of refusing. Mirrors the TS unit test
/// `frozenStore + GVS: an optional patched package that is not cached is skipped, not blocked`.
#[test]
fn frozen_store_gvs_optional_not_seeded_skips() {
let store_dir = tempdir().expect("create temp dir");
let layout = gvs_layout(store_dir.path());
let ignored = frozen_backstop_run(layout, true, true)
.expect("an optional un-seeded build must be skipped, not refused");
assert!(ignored.is_empty(), "no scripts to ignore for a patched-only snapshot: {ignored:?}");
}
/// Negative control: the same patched, un-seeded snapshot does NOT trip
/// the backstop when `frozen_store` is off — proving the flag is
/// load-bearing. With no fixture on disk the run no-ops at the
/// `!pkg_dir.exists()` guard, so it returns `Ok` rather than attempting
/// to apply the (absent) patch file.
#[test]
fn gvs_without_frozen_store_does_not_trip_backstop() {
let store_dir = tempdir().expect("create temp dir");
let layout = gvs_layout(store_dir.path());
let ignored = frozen_backstop_run(layout, false, false)
.expect("without frozen_store the backstop must not fire");
assert!(ignored.is_empty(), "no scripts to ignore for a patched-only snapshot: {ignored:?}");
}
/// Negative control: under the legacy (non-GVS) layout, package
/// directories live under the writable project-local virtual store, so
/// the backstop is correctly inert even with `frozen_store` enabled —
/// builds and patches there never touch the read-only store. Mirrors
/// the TS unit test `frozenStore without GVS: ... is not blocked`.
#[test]
fn frozen_store_without_gvs_does_not_trip_backstop() {
let virtual_store_dir = tempdir().expect("create temp dir");
let layout = VirtualStoreLayout::legacy(
virtual_store_dir.path(),
pacquet_config::default_virtual_store_dir_max_length() as usize,
);
let ignored = frozen_backstop_run(&layout, true, false)
.expect("the non-GVS layout writes to the project store, so the backstop must not fire");
assert!(ignored.is_empty(), "no scripts to ignore for a patched-only snapshot: {ignored:?}");
}
/// Materialize a package fixture whose contents are byte-identical
/// to upstream's `@pnpm.e2e/failing-postinstall@1.0.0` at
/// `/Volumes/src/pnpm/registry-mock/packages/failing-postinstall/package.json`.
@@ -1078,6 +1232,7 @@ async fn write_path_populates_side_effects_row() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("build modules must complete cleanly");
@@ -1192,6 +1347,7 @@ async fn write_path_disabled_skips_upload() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("build modules must complete cleanly");
@@ -1314,6 +1470,7 @@ async fn upload_error_does_not_interrupt_install() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("upload failure must not propagate; install continues");
@@ -1547,6 +1704,7 @@ new file mode 100644
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("build modules must complete cleanly");
@@ -1656,6 +1814,7 @@ new file mode 100644
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect("build modules must complete cleanly");
@@ -1736,6 +1895,7 @@ async fn missing_patch_file_path_errors_with_diagnostic() {
skipped: &SkippedSnapshots::default(),
pkg_root_by_key: None,
gather_ancestor_bin_paths: false,
frozen_store: false,
}
.run::<SilentReporter>()
.expect_err("missing patch_file_path must surface as PatchFilePathMissing");

17
pacquet/crates/package-manager/src/create_virtual_store.rs
View File

@@ -266,13 +266,20 @@ impl CreateVirtualStore<'_> {
// tarball CAFS writes never pay a `create_dir_all` syscall on the
// hot path. Ports pnpm's `initStore` in `worker/src/start.ts`.
// See [`init_store_dir_best_effort`] for the error-degradation
// policy shared with `install_without_lockfile.rs`.
init_store_dir_best_effort(store_dir).await;
// policy shared with `install_without_lockfile.rs`. Skipped under
// `frozenStore`: the store is read-only and complete, so no
// directory creation is attempted under its root.
if !config.frozen_store {
init_store_dir_best_effort(store_dir).await;
}
let open_store_index = if config.frozen_store {
StoreIndex::shared_immutable_in
} else {
StoreIndex::shared_readonly_in
};
let store_index =
match tokio::task::spawn_blocking(move || StoreIndex::shared_readonly_in(store_dir))
.await
{
match tokio::task::spawn_blocking(move || open_store_index(store_dir)).await {
Ok(store_index) => store_index,
Err(error) => {
tracing::warn!(

10
pacquet/crates/package-manager/src/install_frozen_lockfile.rs
View File

@@ -312,7 +312,14 @@ where
// been processed. A writer open / task failure is degraded
// to a `warn!` and the install still succeeds — pacquet's
// existing best-effort stance on cache writes.
let (store_index_writer, writer_task) = StoreIndexWriter::spawn(&config.store_dir);
// Under `frozenStore` the store is opened read-only, so the
// writer is replaced with a drain-and-drop stub that never opens
// `index.db` (no WAL / SHM sidecar under the read-only root).
let (store_index_writer, writer_task) = if config.frozen_store {
StoreIndexWriter::spawn_disabled()
} else {
StoreIndexWriter::spawn(&config.store_dir)
};
// Caller-side fast-path for the installability check. The
// common case (no lockfile metadata row declares an
@@ -985,6 +992,7 @@ where
skipped: &skipped,
pkg_root_by_key: hoisted_pkg_root_by_key.as_ref(),
gather_ancestor_bin_paths: is_hoisted,
frozen_store: config.frozen_store,
}
.run::<Reporter>()
.map_err(InstallFrozenLockfileError::BuildModules)?;

1
pacquet/crates/package-manager/src/install_package_from_registry/tests.rs
View File

@@ -68,6 +68,7 @@ fn create_config(store_dir: &Path, modules_dir: &Path, virtual_store_dir: &Path)
resolve_peers_from_workspace_root: false,
block_exotic_subdeps: false,
verify_store_integrity: true,
frozen_store: false,
side_effects_cache: true,
side_effects_cache_readonly: false,
fetch_retries: 2,

26
pacquet/crates/package-manager/src/install_with_fresh_lockfile.rs
View File

@@ -474,8 +474,12 @@ impl<DependencyGroupList> InstallWithFreshLockfile<'_, DependencyGroupList> {
// tarball CAFS writes never pay a `create_dir_all` syscall on the
// hot path. Ports pnpm's `initStore` in `worker/src/start.ts`.
// See [`init_store_dir_best_effort`] for the error-degradation
// policy shared with `create_virtual_store.rs`.
init_store_dir_best_effort(store_dir).await;
// policy shared with `create_virtual_store.rs`. Skipped under
// `frozenStore`: the store is read-only and complete, so no
// directory creation is attempted under its root.
if !config.frozen_store {
init_store_dir_best_effort(store_dir).await;
}
// Resolve pass: walk the manifest's dependencies through the
// npm resolver chain and produce a flat tree keyed by
@@ -538,10 +542,13 @@ impl<DependencyGroupList> InstallWithFreshLockfile<'_, DependencyGroupList> {
// once resolution is done. Mirrors pnpm's `packageRequester`
// shape: the fetch begins as soon as the resolver returns,
// before any further tree walk.
let open_store_index = if config.frozen_store {
StoreIndex::shared_immutable_in
} else {
StoreIndex::shared_readonly_in
};
let store_index =
match tokio::task::spawn_blocking(move || StoreIndex::shared_readonly_in(store_dir))
.await
{
match tokio::task::spawn_blocking(move || open_store_index(store_dir)).await {
Ok(store_index) => store_index,
Err(error) => {
tracing::warn!(
@@ -554,7 +561,14 @@ impl<DependencyGroupList> InstallWithFreshLockfile<'_, DependencyGroupList> {
};
let store_index_ref = store_index.as_ref();
let (store_index_writer, writer_task) = StoreIndexWriter::spawn(store_dir);
// Under `frozenStore` the store is opened read-only, so the
// writer is replaced with a drain-and-drop stub that never opens
// `index.db` (no WAL / SHM sidecar under the read-only root).
let (store_index_writer, writer_task) = if config.frozen_store {
StoreIndexWriter::spawn_disabled()
} else {
StoreIndexWriter::spawn(store_dir)
};
let verified_files_cache = SharedVerifiedFilesCache::default();

1
pacquet/crates/store-dir/Cargo.toml
View File

@@ -27,6 +27,7 @@ smart-default = { workspace = true }
ssri = { workspace = true }
tokio = { workspace = true, features = ["sync"] }
tracing = { workspace = true }
url = { workspace = true }
# Enable `sha2`'s `asm` feature on non-MSVC targets only. The feature
# activates the `sha2-asm` crate (hand-written `.S` assembly files for

106
pacquet/crates/store-dir/src/store_index.rs
View File

@@ -11,6 +11,7 @@ use std::{
atomic::{AtomicBool, Ordering},
},
};
use url::Url;
/// SQLite-backed per-package index that pnpm v11 stores alongside the CAFS
/// blobs. In the pacquet layout the file lives at
@@ -174,6 +175,28 @@ impl StoreIndexWriter {
});
(Arc::new(StoreIndexWriter { tx, warn_on_send_failure: AtomicBool::new(true) }), handle)
}
/// Spawn a writer that never opens `index.db` — it drains every
/// queued message and drops it.
///
/// Used when `frozenStore` is enabled: the store is opened read-only,
/// so there is nothing to write back. Producers keep queuing rows
/// through the normal handle (the call sites stay identical), but the
/// task touches no `SQLite` connection, so no `index.db` / WAL / SHM
/// sidecar is opened or created under the read-only store root.
///
/// Returns the same `(handle, JoinHandle)` shape as [`Self::spawn`] so
/// call sites can branch on `frozen_store` without diverging.
#[must_use]
pub fn spawn_disabled()
-> (Arc<StoreIndexWriter>, tokio::task::JoinHandle<Result<(), StoreIndexError>>) {
let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel::<WriteMsg>();
let handle = tokio::spawn(async move {
while rx.recv().await.is_some() {}
Ok::<(), StoreIndexError>(())
});
(Arc::new(StoreIndexWriter { tx, warn_on_send_failure: AtomicBool::new(true) }), handle)
}
}
/// Fold one queued `WriteMsg` into the batch's in-flight
@@ -323,6 +346,16 @@ pub enum StoreIndexError {
source: rusqlite::Error,
},
/// The store path could not be turned into the `file:` URI that the
/// immutable open requires (see [`StoreIndex::open_immutable`]).
#[display("Failed to build a file: URI for index.db at {path:?}")]
#[diagnostic(code(pacquet_store_dir::store_index::file_uri))]
FileUri {
path: PathBuf,
#[error(source)]
source: Option<std::io::Error>,
},
#[display("Failed to initialize index.db schema: {source}")]
#[diagnostic(code(pacquet_store_dir::store_index::init_schema))]
InitSchema {
@@ -408,8 +441,12 @@ impl StoreIndex {
/// Open an existing `index.db` read-only. Skips the schema-mutating
/// PRAGMAs (`journal_mode=WAL`, `synchronous`, `wal_autocheckpoint`)
/// and `CREATE TABLE IF NOT EXISTS`, so the call cannot create WAL /
/// SHM sidecar files or otherwise mutate the store.
/// and `CREATE TABLE IF NOT EXISTS`. The connection still participates
/// in WAL locking (it may create the `-shm` sidecar in the store
/// directory), so it stays consistent while a concurrent writer — this
/// process's [`StoreIndexWriter`] or another pnpm/pacquet process —
/// mutates the same database. For a store on a read-only filesystem use
/// [`StoreIndex::open_immutable`] instead.
///
/// We *do* set `busy_timeout`: it's a connection-local wait, not a
/// DB mutation, and without it a concurrent writer (pnpm or another
@@ -425,6 +462,34 @@ impl StoreIndex {
Ok(StoreIndex { conn })
}
/// Open an existing `index.db` from a store that is complete and
/// read-only (`frozenStore`): a Nix store, a read-only bind mount, an
/// OCI layer. `index.db` is a WAL-mode database, and even a plain
/// `SQLITE_OPEN_READ_ONLY` open needs to create the `-shm` sidecar in
/// the store directory — which fails with "attempt to write a readonly
/// database" when the directory is not writable. Opening through the
/// `file:…?immutable=1` URI tells `SQLite` the file cannot change
/// underneath it, so it bypasses the WAL/shm machinery entirely and
/// reads the raw file with zero sidecar creation.
///
/// The immutability assertion is load-bearing: `SQLite` skips all locking
/// and change detection on this connection, so a concurrent writer makes
/// reads undefined (stale or corrupt results). Only open this way under
/// `frozenStore`, which disables every store-write path — for a store
/// that other connections may write, use [`StoreIndex::open_readonly`].
/// No `busy_timeout` is set because an immutable connection takes no
/// locks for it to wait on.
pub fn open_immutable(store_dir: &Path) -> Result<Self, StoreIndexError> {
let db_path = store_dir.join("index.db");
let uri = immutable_sqlite_uri(&db_path)?;
let conn = Connection::open_with_flags(
&uri,
OpenFlags::SQLITE_OPEN_READ_ONLY | OpenFlags::SQLITE_OPEN_URI,
)
.map_err(|source| StoreIndexError::Open { path: db_path, source })?;
Ok(StoreIndex { conn })
}
/// Read-only counterpart to [`StoreIndex::open_in`].
pub fn open_readonly_in(store_dir: &StoreDir) -> Result<Self, StoreIndexError> {
StoreIndex::open_readonly(store_dir.root())
@@ -440,11 +505,25 @@ impl StoreIndex {
/// redoing its PRAGMAs) on every package, which otherwise scales
/// linearly with the snapshot count.
pub fn shared_readonly_in(store_dir: &StoreDir) -> Option<SharedReadonlyStoreIndex> {
StoreIndex::shared_in(store_dir, StoreIndex::open_readonly)
}
/// [`StoreIndex::shared_readonly_in`] for a frozen store — opens via
/// [`StoreIndex::open_immutable`], with its no-concurrent-writer
/// contract.
pub fn shared_immutable_in(store_dir: &StoreDir) -> Option<SharedReadonlyStoreIndex> {
StoreIndex::shared_in(store_dir, StoreIndex::open_immutable)
}
fn shared_in(
store_dir: &StoreDir,
open: fn(&Path) -> Result<Self, StoreIndexError>,
) -> Option<SharedReadonlyStoreIndex> {
let store_root = store_dir.root();
if !store_root.join("index.db").exists() {
return None;
}
StoreIndex::open_readonly(store_root).ok().map(|index| Arc::new(Mutex::new(index)))
open(store_root).ok().map(|index| Arc::new(Mutex::new(index)))
}
/// Look up a package-files index by key. Returns `Ok(None)` if no row exists.
@@ -847,5 +926,26 @@ pub struct SideEffectsDiff {
pub deleted: Option<Vec<String>>,
}
/// Build the `file://…?immutable=1` URI used to open `index.db` read-only (see
/// [`StoreIndex::open_immutable`] for why immutable). [`Url::from_file_path`]
/// yields a canonical file URL on every platform: it percent-encodes the URI
/// delimiters that would otherwise truncate the path or inject a query/fragment
/// (`?`, `#`, `%`, spaces) and, on Windows, maps the drive letter and
/// backslashes into a valid `file:///C:/…` form. See <https://sqlite.org/uri.html>.
///
/// [`Url::from_file_path`] only accepts an absolute path, so a relative store
/// path is first absolutized against the current directory — the same
/// resolution Node's `pathToFileURL` applies on the pnpm side.
fn immutable_sqlite_uri(db_path: &Path) -> Result<String, StoreIndexError> {
let absolute = std::path::absolute(db_path).map_err(|source| StoreIndexError::FileUri {
path: db_path.to_path_buf(),
source: Some(source),
})?;
let mut url = Url::from_file_path(&absolute)
.map_err(|()| StoreIndexError::FileUri { path: absolute, source: None })?;
url.query_pairs_mut().append_pair("immutable", "1");
Ok(url.into())
}
#[cfg(test)]
mod tests;

104
pacquet/crates/store-dir/src/store_index/tests.rs
View File

@@ -1,12 +1,40 @@
use super::{
CafsFileInfo, GET_MANY_CHUNK, PackageFilesIndex, StoreIndex, git_hosted_store_index_key,
pick_store_index_key, store_index_key,
immutable_sqlite_uri, pick_store_index_key, store_index_key,
};
use crate::StoreDir;
use pretty_assertions::assert_eq;
use std::collections::HashMap;
use std::{collections::HashMap, path::Path};
use tempfile::tempdir;
// `Url::from_file_path` only accepts a platform-absolute path: a POSIX
// `/store/...` path is not a valid Windows file path, so these construct
// Unix paths and are gated to Unix.
#[cfg(unix)]
#[test]
fn immutable_uri_percent_encodes_sqlite_path_delimiters() {
// `/` stays literal; `?`, `#`, and `%` (the escape introducer) are
// percent-encoded so they cannot truncate the path or inject a query.
assert_eq!(
immutable_sqlite_uri(Path::new("/store/index.db")).unwrap(),
"file:///store/index.db?immutable=1",
);
assert_eq!(
immutable_sqlite_uri(Path::new("/a?b/c#d/100%/index.db")).unwrap(),
"file:///a%3Fb/c%23d/100%25/index.db?immutable=1",
);
}
/// A relative store path is absolutized against the current directory
/// instead of failing — the resolution Node's `pathToFileURL` applies on
/// the pnpm side.
#[test]
fn immutable_uri_absolutizes_a_relative_path() {
let uri = immutable_sqlite_uri(Path::new("relative-store/index.db")).unwrap();
assert!(uri.starts_with("file:///"), "{uri}");
assert!(uri.ends_with("/relative-store/index.db?immutable=1"), "{uri}");
}
fn sample_index() -> PackageFilesIndex {
let mut files = HashMap::new();
files.insert(
@@ -129,6 +157,24 @@ fn reopening_the_same_db_sees_prior_writes() {
assert_eq!(idx.get(&key).unwrap().unwrap(), payload);
}
/// `?` is a legal filename byte on Unix but a `SQLite` URI delimiter, so a raw
/// `file:{path}?immutable=1` would truncate the path here. (`?` is illegal in
/// Windows filenames, so this case cannot arise there.)
#[cfg(unix)]
#[test]
fn open_immutable_handles_a_store_path_containing_a_question_mark() {
let root = tempdir().unwrap();
let store_dir = root.path().join("weird?store");
std::fs::create_dir_all(&store_dir).unwrap();
let key = store_index_key("sha512-q", "q-pkg@1.0.0");
let payload = sample_index();
StoreIndex::open(&store_dir).unwrap().set(&key, &payload).unwrap();
let idx = StoreIndex::open_immutable(&store_dir).unwrap();
assert_eq!(idx.get(&key).unwrap().unwrap(), payload);
}
#[test]
fn index_db_lives_at_store_dir_v11() {
let root = tempdir().unwrap();
@@ -294,3 +340,57 @@ fn get_many_handles_more_keys_than_chunk_size() {
assert_eq!(out.get(key), Some(&payload));
}
}
/// `open_immutable` must read a WAL-mode `index.db` that lives on a
/// read-only *directory* — the `frozenStore` / read-only-store
/// scenario (a Nix store, a read-only bind mount, an OCI layer).
///
/// This is the regression guard for the `immutable=1` open: a plain
/// `SQLITE_OPEN_READ_ONLY` open of a WAL database still tries to create
/// the `-shm` sidecar in the directory, which fails with "attempt to
/// write a readonly database" when the directory is not writable.
/// Revert `open_immutable` to plain `SQLITE_OPEN_READ_ONLY` and this
/// test fails — confirming the URI is load-bearing.
#[cfg(unix)]
#[test]
fn open_immutable_reads_wal_db_on_readonly_directory() {
use std::fs;
use std::os::unix::fs::PermissionsExt;
let dir = tempdir().unwrap();
let key = store_index_key("sha512-ro", "frozen@1.0.0");
let payload = sample_index();
// Seed the WAL db while the directory is still writable, then close
// the writer connection so the on-disk file is a settled WAL db —
// exactly what a Nix seed-build would leave behind.
{
let idx = StoreIndex::open(dir.path()).unwrap();
idx.set(&key, &payload).unwrap();
}
// Drop the directory to read + execute only: no writes permitted,
// so SQLite cannot create any `-shm` / `-wal` / `-journal` sidecar.
let original = fs::metadata(dir.path()).unwrap().permissions();
fs::set_permissions(dir.path(), fs::Permissions::from_mode(0o555)).unwrap();
let read_result = StoreIndex::open_immutable(dir.path()).map(|idx| idx.get(&key));
// Restore write permission before assertions so the tempdir can be
// cleaned up regardless of the outcome.
fs::set_permissions(dir.path(), original).unwrap();
let loaded = read_result
.expect("open_immutable must succeed on a read-only directory")
.expect("get must not error")
.expect("the seeded row must be readable");
assert_eq!(loaded, payload);
// No sidecar may have been created under the read-only directory.
for sidecar in ["index.db-shm", "index.db-wal", "index.db-journal"] {
assert!(
!dir.path().join(sidecar).exists(),
"immutable open must not create the {sidecar} sidecar",
);
}
}

3
pnpm-lock.yaml generated
View File

@@ -9304,6 +9304,9 @@ importers:
store/index:
dependencies:
'@pnpm/error':
specifier: workspace:*
version: link:../../core/error
msgpackr:
specifier: 'catalog:'
version: 2.0.4

2
resolving/npm-resolver/src/index.ts
View File

@@ -132,6 +132,7 @@ export { whichVersionIsPinned } from './whichVersionIsPinned.js'
export interface ResolverFactoryOptions {
cacheDir: string
storeDir?: string
frozenStore?: boolean
fullMetadata?: boolean
filterMetadata?: boolean
offline?: boolean
@@ -238,6 +239,7 @@ export function createNpmResolver (
{
storeDir,
verifyStoreIntegrity: false,
frozenStore: opts.frozenStore,
},
filesIndexFile,
{

11
store/connection-manager/src/createNewStoreController.ts
View File

@@ -5,7 +5,7 @@ import type { Config, ConfigContext } from '@pnpm/config.reader'
import { type ClientOptions, createClient } from '@pnpm/installing.client'
import type { ResolutionVerifier } from '@pnpm/resolving.resolver-base'
import { type CafsLocker, createPackageStore, type StoreController } from '@pnpm/store.controller'
import { StoreIndex } from '@pnpm/store.index'
import { ReadOnlyStoreIndex, StoreIndex } from '@pnpm/store.index'
type CreateResolverOptions = Pick<Config,
| 'fetchRetries'
@@ -22,6 +22,7 @@ export type CreateNewStoreControllerOptions = CreateResolverOptions & Pick<Confi
| 'cert'
| 'engineStrict'
| 'force'
| 'frozenStore'
| 'nodeDownloadMirrors'
| 'nodeVersion'
| 'fetchTimeout'
@@ -72,8 +73,10 @@ export async function createNewStoreController (
opts.trustPolicy === 'no-downgrade'
) && !opts.registrySupportsTimeField
)
await fs.mkdir(opts.storeDir, { recursive: true })
const storeIndex = new StoreIndex(opts.storeDir)
if (!opts.frozenStore) {
await fs.mkdir(opts.storeDir, { recursive: true })
}
const storeIndex = opts.frozenStore ? new ReadOnlyStoreIndex(opts.storeDir) : new StoreIndex(opts.storeDir)
const { resolve, fetchers, clearResolutionCache, resolutionVerifiers } = createClient({
customResolvers: opts.hooks?.customResolvers,
customFetchers: opts.hooks?.customFetchers,
@@ -82,6 +85,7 @@ export async function createNewStoreController (
cacheDir: opts.cacheDir,
storeDir: opts.storeDir,
cert: opts.cert,
frozenStore: opts.frozenStore,
fetchWarnTimeoutMs: opts.fetchWarnTimeoutMs,
fetchMinSpeedKiBps: opts.fetchMinSpeedKiBps,
fullMetadata,
@@ -146,6 +150,7 @@ export async function createNewStoreController (
strictStorePkgContentCheck: opts.strictStorePkgContentCheck,
clearResolutionCache,
customFetchers: opts.hooks?.customFetchers,
frozenStore: opts.frozenStore,
storeIndex,
}),
dir: opts.storeDir,

11
store/controller/src/storeController/index.ts
View File

@@ -1,6 +1,7 @@
import fs from 'node:fs'
import path from 'node:path'
import { PnpmError } from '@pnpm/error'
import type { Fetchers } from '@pnpm/fetching.fetcher-base'
import type { CustomFetcher } from '@pnpm/hooks.types'
import { createPackageRequester } from '@pnpm/installing.package-requester'
@@ -34,6 +35,7 @@ export interface CreatePackageStoreOptions {
strictStorePkgContentCheck?: boolean
clearResolutionCache: () => void
customFetchers?: CustomFetcher[]
frozenStore?: boolean
storeIndex: StoreIndex
}
@@ -44,6 +46,14 @@ export function createPackageStore (
): StoreController {
const storeDir = initOpts.storeDir
if (!fs.existsSync(path.join(storeDir, 'files'))) {
// A missing `{storeDir}/files` means the store has no content directory yet.
// Under frozenStore the store is meant to be a complete, read-only seed, so
// this is a setup error: initializing it would be a write into a read-only
// store. Fail fast with actionable guidance instead of swallowing the write.
if (initOpts.frozenStore) {
throw new PnpmError('FROZEN_STORE_INCOMPLETE',
`frozenStore is enabled but the store at ${storeDir} is missing its content directory (${path.join(storeDir, 'files')}). The store must be fully seeded before it can be used read-only.`)
}
initStoreDir(storeDir).catch(() => {})
}
const cafs = createCafsStore(storeDir, {
@@ -65,6 +75,7 @@ export function createPackageStore (
virtualStoreDirMaxLength: initOpts.virtualStoreDirMaxLength,
strictStorePkgContentCheck: initOpts.strictStorePkgContentCheck,
customFetchers: initOpts.customFetchers,
frozenStore: initOpts.frozenStore,
})
return {

1
store/index/package.json
View File

@@ -32,6 +32,7 @@
".test": "cross-env NODE_OPTIONS=\"$NODE_OPTIONS --experimental-vm-modules --disable-warning=ExperimentalWarning --disable-warning=DEP0169\" jest"
},
"dependencies": {
"@pnpm/error": "workspace:*",
"msgpackr": "catalog:"
},
"devDependencies": {

161
store/index/src/index.ts
View File

@@ -1,9 +1,13 @@
import fs from 'node:fs'
import { createRequire } from 'node:module'
import type { DatabaseSync as DatabaseSyncType, StatementSync } from 'node:sqlite'
import { pathToFileURL } from 'node:url'
import { PnpmError } from '@pnpm/error'
import { Packr } from 'msgpackr'
const FROZEN_STORE_WRITE_MESSAGE = 'Cannot write to the package store because frozenStore is enabled (the store is opened read-only). This indicates the store is missing content the install needs.'
// Use createRequire to load node:sqlite because it is a prefix-only builtin
// that Jest's ESM module resolver cannot handle.
const req = createRequire(import.meta.url)
@@ -106,22 +110,37 @@ export function closeAllStoreIndexes (): void {
}
export class StoreIndex {
private db: DatabaseSyncType
private closed = false
protected db!: DatabaseSyncType
protected closed = false
private pendingWrites: Array<{ key: string, buffer: Uint8Array }> = []
private flushScheduled = false
private stmtGet: StatementSync
private stmtSet: StatementSync
private stmtDel: StatementSync
private stmtHas: StatementSync
private stmtAll: StatementSync
private stmtKeys: StatementSync
protected stmtGet!: StatementSync
protected stmtSet!: StatementSync
protected stmtDel!: StatementSync
protected stmtHas!: StatementSync
protected stmtAll!: StatementSync
protected stmtKeys!: StatementSync
private readonly exitHandler: () => void
constructor (storeDir: string) {
const dbPath = `${storeDir}/index.db`
this.openDatabase(storeDir)
this.prepareStatements()
this.exitHandler = () => this.close()
// Multiple StoreIndex instances may be created (e.g. in tests), each adding
// an exit listener. Raise the limit to avoid MaxListenersExceededWarning.
// Skip when maxListeners is 0 (unlimited).
const currentMax = process.getMaxListeners()
if (currentMax !== 0 && currentMax < openInstances.size + 11) {
process.setMaxListeners(Math.max(currentMax + 10, openInstances.size + 11))
}
process.on('exit', this.exitHandler)
openInstances.add(this)
}
/** Open the SQLite connection. Overridden by {@link ReadOnlyStoreIndex}. */
protected openDatabase (storeDir: string): void {
fs.mkdirSync(storeDir, { recursive: true })
this.db = new DatabaseSync(dbPath)
this.db = new DatabaseSync(`${storeDir}/index.db`)
// Set busy_timeout FIRST so SQLite's internal busy handler is active
// during all subsequent operations. On Windows, file locking is mandatory
// and concurrent processes (e.g. parallel dlx calls) will contend.
@@ -143,22 +162,16 @@ export class StoreIndex {
) WITHOUT ROWID
`)
})
}
/** Prepare the prepared statements. Overridden by {@link ReadOnlyStoreIndex} to skip the write statements. */
protected prepareStatements (): void {
this.stmtGet = this.db.prepare('SELECT data FROM package_index WHERE key = ?')
this.stmtSet = this.db.prepare('INSERT OR REPLACE INTO package_index (key, data) VALUES (?, ?)')
this.stmtDel = this.db.prepare('DELETE FROM package_index WHERE key = ?')
this.stmtHas = this.db.prepare('SELECT 1 FROM package_index WHERE key = ?')
this.stmtAll = this.db.prepare('SELECT key, data FROM package_index')
this.stmtKeys = this.db.prepare('SELECT key FROM package_index')
this.exitHandler = () => this.close()
// Multiple StoreIndex instances may be created (e.g. in tests), each adding
// an exit listener. Raise the limit to avoid MaxListenersExceededWarning.
// Skip when maxListeners is 0 (unlimited).
const currentMax = process.getMaxListeners()
if (currentMax !== 0 && currentMax < openInstances.size + 11) {
process.setMaxListeners(Math.max(currentMax + 10, openInstances.size + 11))
}
process.on('exit', this.exitHandler)
openInstances.add(this)
}
get (key: string): unknown | undefined {
@@ -317,15 +330,113 @@ export class StoreIndex {
this.closed = true
openInstances.delete(this)
process.removeListener('exit', this.exitHandler)
try {
this.db.exec('PRAGMA optimize')
} catch {
// PRAGMA optimize is a performance hint; safe to ignore if the DB is locked.
}
this.optimizeBeforeClose()
try {
this.db.close()
} catch {
// The DB may be locked by another connection; the OS will reclaim it on process exit.
}
}
/** Run `PRAGMA optimize` before closing. Overridden by {@link ReadOnlyStoreIndex} to skip it (the DB is immutable). */
protected optimizeBeforeClose (): void {
try {
this.db.exec('PRAGMA optimize')
} catch {
// PRAGMA optimize is a performance hint; safe to ignore if the DB is locked.
}
}
}
/**
* A {@link StoreIndex} opened read-only for installs against a store on a
* read-only filesystem (`frozenStore`). The index is a WAL-mode database, and a
* normal WAL read creates an `index.db-shm` sidecar in the store directory —
* which fails on a read-only directory and surfaces as "attempt to write a
* readonly database" on the first query. Opening via the SQLite `immutable=1`
* URI tells SQLite the file cannot change, so it bypasses the WAL/shm machinery
* and reads the file directly, creating no sidecars.
*
* The store is assumed complete; every write is a programming error and throws.
*/
export class ReadOnlyStoreIndex extends StoreIndex {
protected override openDatabase (storeDir: string): void {
if (!nodeSupportsImmutableSqliteUri()) {
throw new PnpmError(
'FROZEN_STORE_UNSUPPORTED_NODE',
`frozenStore opens the store index read-only via a SQLite "immutable" URI, which requires Node.js >=22.15.0, >=23.11.0, or >=24.0.0, but the current version is ${process.versions.node}. Upgrade Node.js, or run without frozenStore.`
)
}
this.db = new DatabaseSync(immutableSqliteUri(`${storeDir}/index.db`))
}
protected override prepareStatements (): void {
this.stmtGet = this.db.prepare('SELECT data FROM package_index WHERE key = ?')
this.stmtHas = this.db.prepare('SELECT 1 FROM package_index WHERE key = ?')
this.stmtAll = this.db.prepare('SELECT key, data FROM package_index')
this.stmtKeys = this.db.prepare('SELECT key FROM package_index')
}
protected override optimizeBeforeClose (): void {}
override set (_key: string, _data: unknown): void {
this.throwReadOnly()
}
override delete (_key: string): boolean {
this.throwReadOnly()
}
override queueWrites (_writes: Array<{ key: string, buffer: Uint8Array }>): void {
this.throwReadOnly()
}
override setRawMany (_entries: Array<{ key: string, buffer: Uint8Array }>): void {
this.throwReadOnly()
}
override deleteMany (_keys: string[]): void {
this.throwReadOnly()
}
override checkpoint (): void {
this.throwReadOnly()
}
private throwReadOnly (): never {
throw new PnpmError('FROZEN_STORE_WRITE', FROZEN_STORE_WRITE_MESSAGE)
}
}
/**
* Build the `file://…?immutable=1` URI used to open `index.db` read-only (see
* the frozen-store rationale at the call site). `pathToFileURL` yields a
* canonical file URL on every platform: it percent-encodes the URI delimiters
* that could otherwise truncate the path or inject a query/fragment (`?`, `#`,
* `%`, spaces) and, on Windows, maps the drive letter and backslashes into a
* valid `file:///C:/…` form. A raw `file:${path}` concatenation would mis-parse
* those. See https://sqlite.org/uri.html.
*/
function immutableSqliteUri (dbPath: string): string {
const url = pathToFileURL(dbPath)
url.searchParams.set('immutable', '1')
return url.href
}
/**
* Whether the running Node.js can open a `file:…?immutable=1` SQLite URI.
*
* `node:sqlite` only passes `SQLITE_OPEN_URI` to SQLite — so the `immutable=1`
* query is honored rather than treated as part of a literal filename — starting
* in v22.15.0 (22.x line), v23.11.0 (23.x line), and every v24+. On older
* runtimes the URI is opened as a literal path and fails with a cryptic
* "unable to open database file"; we detect that up front to give actionable
* guidance instead.
*/
function nodeSupportsImmutableSqliteUri (): boolean {
const [major, minor] = process.versions.node.split('.', 2).map(Number)
if (major < 22) return false
if (major === 22) return minor >= 15
if (major === 23) return minor >= 11
return true
}

101
store/index/test/index.ts
View File

@@ -1,7 +1,8 @@
import fs from 'node:fs'
import path from 'node:path'
import { expect, test } from '@jest/globals'
import { StoreIndex, storeIndexKey } from '@pnpm/store.index'
import { ReadOnlyStoreIndex, StoreIndex, storeIndexKey } from '@pnpm/store.index'
import { temporaryDirectory } from 'tempy'
test('StoreIndex round-trips data via SQLite key', () => {
@@ -46,3 +47,101 @@ test('StoreIndex entries() iterates all SQLite entries', () => {
idx.close()
}
})
// The immutable open only works on a runtime that honors the immutable URI;
// this is purely a Node-version property, independent of platform.
const supportsImmutableUri = nodeSupportsImmutableSqliteUri()
// chmod 0555 has no effect on Windows (and `?` is illegal in filenames there),
// so the read-only-directory tests below cannot hold on win32.
const canAssertReadonlyDir = process.platform !== 'win32'
const testFrozenOpen = (canAssertReadonlyDir && supportsImmutableUri) ? test : test.skip
testFrozenOpen('StoreIndex frozen mode reads a WAL db on a read-only directory and refuses writes', () => {
const storeDir = path.join(temporaryDirectory(), 'store', 'v11')
const key = storeIndexKey('sha512-frozen', 'frozen-pkg@1.0.0')
const data = { algo: 'sha512', files: new Map([['index.js', { digest: 'abc', size: 100, mode: 0o644 }]]) }
// Seed the WAL db while the directory is still writable, then close
// the connection so the on-disk file is a settled WAL db — what a
// read-only-store seed-build would leave behind.
const seed = new StoreIndex(storeDir)
seed.set(key, data)
seed.close()
// Drop the store dir to read + execute only: no writes permitted, so
// SQLite cannot create any -shm / -wal sidecar.
fs.chmodSync(storeDir, 0o555)
try {
const idx = new ReadOnlyStoreIndex(storeDir)
try {
const result = idx.get(key) as typeof data
expect(result).toBeDefined()
expect(result.algo).toBe('sha512')
expect(result.files.get('index.js')?.digest).toBe('abc')
expect(idx.has(key)).toBe(true)
expect(() => {
idx.set(key, data)
}).toThrow(expect.objectContaining({ code: 'ERR_PNPM_FROZEN_STORE_WRITE' }))
expect(() => {
idx.delete(key)
}).toThrow(expect.objectContaining({ code: 'ERR_PNPM_FROZEN_STORE_WRITE' }))
// The immutable open must not create any sidecar under the
// read-only directory.
for (const sidecar of ['index.db-shm', 'index.db-wal', 'index.db-journal']) {
expect(fs.existsSync(path.join(storeDir, sidecar))).toBe(false)
}
} finally {
idx.close()
}
} finally {
// Restore write permission so the tempdir can be cleaned up.
fs.chmodSync(storeDir, 0o755)
}
})
// `?` is a legal filename character on POSIX but a SQLite URI delimiter, so a
// raw `file:${path}?immutable=1` would truncate the path here. (`?` is illegal
// in Windows filenames, so this case cannot arise there.)
testFrozenOpen('StoreIndex frozen mode opens under a store path containing a "?"', () => {
const storeDir = path.join(temporaryDirectory(), 'weird?store', 'v11')
const key = storeIndexKey('sha512-q', 'q-pkg@1.0.0')
const data = { algo: 'sha512', files: new Map([['index.js', { digest: 'q', size: 1, mode: 0o644 }]]) }
const seed = new StoreIndex(storeDir)
seed.set(key, data)
seed.close()
const idx = new ReadOnlyStoreIndex(storeDir)
try {
expect(idx.has(key)).toBe(true)
expect((idx.get(key) as typeof data).algo).toBe('sha512')
} finally {
idx.close()
}
})
// On a runtime that cannot honor the immutable URI, a frozen open must fail
// fast with actionable guidance rather than SQLite's cryptic "unable to open
// database file". This is keyed only off the Node version (the error is
// platform-independent), so it runs on Windows too when the runtime is old.
const testUnsupportedNode = supportsImmutableUri ? test.skip : test
testUnsupportedNode('StoreIndex frozen mode refuses to open on a Node.js without immutable-URI support', () => {
const storeDir = path.join(temporaryDirectory(), 'store', 'v11')
expect(() => new ReadOnlyStoreIndex(storeDir))
.toThrow(expect.objectContaining({ code: 'ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE' }))
})
// The `immutable=1` URI open only works on Node.js that passes
// SQLITE_OPEN_URI to SQLite: v22.15.0+, v23.11.0+, and every v24+. On older
// runtimes (including pnpm's `engines` floor of 22.13) the open throws a clear
// ERR_PNPM_FROZEN_STORE_UNSUPPORTED_NODE instead — asserted separately below.
function nodeSupportsImmutableSqliteUri (): boolean {
const [major, minor] = process.versions.node.split('.', 2).map(Number)
if (major < 22) return false
if (major === 22) return minor >= 15
if (major === 23) return minor >= 11
return true
}

6
store/index/tsconfig.json
View File

@@ -8,5 +8,9 @@
"src/**/*.ts",
"../../__typings__/**/*.d.ts"
],
"references": []
"references": [
{
"path": "../../core/error"
}
]
}

19
worker/src/index.ts
View File

@@ -100,7 +100,15 @@ export async function addFilesFromDir (opts: AddFilesFromDirOptions): Promise<Ad
if (indexWrites) {
// Write immediately so that subsequent worker reads (e.g. side effects)
// see the committed data without waiting for nextTick.
opts.storeIndex.setRawMany(indexWrites)
// A throw must reject rather than escape the message callback, where it
// would surface as an uncaughtException and leave this promise pending —
// e.g. ReadOnlyStoreIndex refusing the write under frozenStore.
try {
opts.storeIndex.setRawMany(indexWrites)
} catch (err: unknown) {
reject(err as Error)
return
}
}
resolve(value)
})
@@ -181,7 +189,13 @@ export async function addFilesFromTarball (opts: AddFilesFromTarballOptions): Pr
return
}
if (indexWrites) {
opts.storeIndex.queueWrites(indexWrites)
// See addFilesFromDir: a throw must reject, not escape the callback.
try {
opts.storeIndex.queueWrites(indexWrites)
} catch (err: unknown) {
reject(err as Error)
return
}
}
resolve(value)
})
@@ -204,6 +218,7 @@ export interface ReadPkgFromCafsContext {
storeDir: string
verifyStoreIntegrity: boolean
strictStorePkgContentCheck?: boolean
frozenStore?: boolean
}
export interface ReadPkgFromCafsOptions {

17
worker/src/start.ts
View File

@@ -22,7 +22,7 @@ import {
} from '@pnpm/store.cafs'
import type { Cafs, FilesMap, PackageFiles, SideEffectsDiff } from '@pnpm/store.cafs-types'
import { createCafsStore } from '@pnpm/store.create-cafs-store'
import { packForStorage, StoreIndex } from '@pnpm/store.index'
import { packForStorage, ReadOnlyStoreIndex, StoreIndex } from '@pnpm/store.index'
import type { BundledManifest, DependencyManifest } from '@pnpm/types'
import { equalOrSemverEqual } from './equalOrSemverEqual.js'
@@ -48,11 +48,14 @@ const cafsStoreCache = new Map<string, Cafs>()
const cafsLocker = new Map<string, number>()
const storeIndexCache = new Map<string, StoreIndex>()
function getStoreIndex (storeDir: string): StoreIndex {
if (!storeIndexCache.has(storeDir)) {
storeIndexCache.set(storeDir, new StoreIndex(storeDir))
function getStoreIndex (storeDir: string, frozen = false): StoreIndex {
// A frozen store is opened immutable (read-only), so it cannot share a
// cached handle with a writable open of the same directory. Key on both.
const cacheKey = frozen ? `${storeDir}\0frozen` : storeDir
if (!storeIndexCache.has(cacheKey)) {
storeIndexCache.set(cacheKey, frozen ? new ReadOnlyStoreIndex(storeDir) : new StoreIndex(storeDir))
}
return storeIndexCache.get(storeDir)!
return storeIndexCache.get(cacheKey)!
}
async function handleMessage (
@@ -96,8 +99,8 @@ async function handleMessage (
break
}
case 'readPkgFromCafs': {
const { storeDir, filesIndexFile, verifyStoreIntegrity, expectedPkg, strictStorePkgContentCheck } = message
const pkgFilesIndex = getStoreIndex(storeDir).get(filesIndexFile) as PackageFilesIndex | undefined
const { storeDir, filesIndexFile, verifyStoreIntegrity, expectedPkg, strictStorePkgContentCheck, frozenStore } = message
const pkgFilesIndex = getStoreIndex(storeDir, frozenStore).get(filesIndexFile) as PackageFilesIndex | undefined
if (!pkgFilesIndex) {
parentPort!.postMessage({
status: 'success',

1
worker/src/types.ts
View File

@@ -72,6 +72,7 @@ export interface ReadPkgFromCafsMessage {
verifyStoreIntegrity: boolean
expectedPkg?: PkgNameVersion
strictStorePkgContentCheck?: boolean
frozenStore?: boolean
}
export interface HardLinkDirMessage {

32
worker/test/addFilesFromDir.test.ts Normal file
View File

@@ -0,0 +1,32 @@
import fs from 'node:fs'
import os from 'node:os'
import path from 'node:path'
import { afterAll, expect, test } from '@jest/globals'
import { PnpmError } from '@pnpm/error'
import type { StoreIndex } from '@pnpm/store.index'
import { addFilesFromDir, finishWorkers } from '../lib/index.js'
afterAll(() => finishWorkers())
test('addFilesFromDir() rejects when committing the index writes throws (e.g. a read-only store index)', async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'pnpm-worker-test-'))
const dir = path.join(tmp, 'pkg')
fs.mkdirSync(dir)
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'frozen-test-pkg', version: '1.0.0' }))
const storeDir = path.join(tmp, 'store')
const storeIndex = {
setRawMany () {
throw new PnpmError('FROZEN_STORE_WRITE', 'Cannot write to the package store because frozenStore is enabled')
},
} as unknown as StoreIndex
await expect(addFilesFromDir({
storeDir,
dir,
filesIndexFile: path.join(storeDir, 'frozen-test-pkg.json'),
storeIndex,
})).rejects.toMatchObject({ code: 'ERR_PNPM_FROZEN_STORE_WRITE' })
})