mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-01 20:49:45 -04:00
dependabot/github_actions/github-actions-48d2ea223b
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Kochan
|
fcf95c7faa |
perf: cache the post-resolution lockfile verification gate (#11691)
Closes #11687. ## What Cache the result of the post-resolution lockfile verification gate (#11583) so repeat installs against an unchanged lockfile skip the per-package registry round trips entirely. Persisted as JSON Lines at `<cacheDir>/lockfile-verified.jsonl`. The cache layer is policy-neutral. Today there's one verifier (`minimumReleaseAge`); future resolver-side verifiers (jsr trust, attestation, …) plug in by declaring their own `policy` slot and `canTrustPastCheck` comparator — no install-side changes. ## Why #11583 re-hits the registry on every install for every locked (name, version) pair. On warm/repeat installs where the lockfile hasn't moved, that's a stack of per-package round trips with nothing to show for them. This change makes the steady-state case effectively free without weakening the protection — the gate still runs in full whenever the lockfile changes, any verifier's policy tightens, or no record exists. ## How ### Cache lookup, in order The cache is **indexed by content hash** so git worktrees with identical lockfile bytes share a cache entry. A secondary path-keyed index drives the same-machine stat shortcut. 1. **`stat()` shortcut** — when a previous record for this exact `lockfilePath` matches today's `size + mtime + inode`, trust the cached hash without reading anything. Zero I/O beyond the stat. Microseconds. 2. **Content lookup** — hash the in-memory lockfile (not the file bytes — we already have the parsed object) and look up by content hash. Catches worktrees (same content, different path) and CI checkouts (same content, reset stat). On hit, append a refreshed path/stat entry so the next install at this path takes the stat shortcut. 3. **Any active verifier rejects the cached `policy`** — run the full gate. 4. **No record** — run the full gate. The in-memory object is hashed with `hashObject` from `@pnpm/crypto.object-hasher` (streaming, key-order-stable). ### Record shape ```json { "lockfile": { "hash": "<sha256 base64>", "path": "/abs/path/to/pnpm-lock.yaml", "size": 154, "mtimeNs": "1736245123000000000", "inode": "12345" }, "verifiedAt": "2026-05-17T...", "policy": { "minimumReleaseAge": 1440 } } ``` `policy` is the union of every active verifier's `policy` contribution. Verifiers checking the same logical policy (e.g. `minimumReleaseAge` honored by multiple registries) name it the same and share the slot — no resolver namespacing. ### File semantics - **Sync fs throughout** — the cache is consulted once before verification fan-out and recorded once after. No concurrent install work to overlap with; keeping the call sites straight-line. - **JSONL appends are atomic** on POSIX/NTFS, so parallel pnpm processes (monorepo installs, CI matrices sharing a cache) write without coordination. Latest record per `(path, hash)` tuple wins on read. - **Bounded file** — capped at ~1000 entries; compaction is triggered by a single `stat()` of the cache file (1.5 MiB byte budget) so we never parse the file on the steady-state path. When triggered, the tail is rewritten via tempfile + rename. - **No record on rejection** — a failing verification deliberately doesn't write a record; the next install must rerun the gate. - **Single hash per install** — the in-memory hash is computed lazily and reused: `tryLockfileVerificationCache` returns the precomputed stat+hash to `recordVerification` on a miss, and the stat-shortcut hit forwards the cached record's hash unchanged. ## Plumbing The verifier contract changed alongside the cache to make this composable without install-side knowledge of each policy: - **`@pnpm/resolving.resolver-base`** — `ResolutionVerifier` is now `{ verify, policy, canTrustPastCheck }` (was a bare function in #11583). Each resolver-side verifier owns its policy snapshot and the comparator that decides whether a cached policy is still trustworthy. - **`@pnpm/resolving.npm-resolver`** — `createNpmResolutionVerifier` returns the new shape: `policy: { minimumReleaseAge }`, `canTrustPastCheck` reads `minimumReleaseAge` from the merged cached bag. - **`@pnpm/resolving.default-resolver`** — `createResolutionVerifier` (singular, returning a combined function) → `createResolutionVerifiers` (plural, returning a `ResolutionVerifier[]`). No combinator; each verifier handles its own protocol short-circuit inside `verify`, so dispatch happens naturally at the install side. - **`@pnpm/installing.client`** — `Client.verifyResolution?` → `Client.resolutionVerifiers: ResolutionVerifier[]`. Same rename propagates through `@pnpm/store.connection-manager`, `@pnpm/testing.temp-store`, and `StrictInstallOptions`. - **`@pnpm/installing.deps-installer`** — new `verifyLockfileResolutionsCache.ts` (`tryLockfileVerificationCache` + `recordVerification`). `verifyLockfileResolutions` takes the verifier list plus `cacheDir` + `lockfilePath` as flat options; the cache fires when both are present, otherwise the gate runs without memoization. The dedup key for in-flight candidates includes a serialization of `resolution` so two entries sharing a (name, version) but pinned via different protocols don't collapse. Breaking but safe — `@pnpm/resolving.npm-resolver` hasn't been released since #11583 introduced the verifier abstraction, so no downstream consumer is on the old shape. ## Tests - **17 unit tests** in `verifyLockfileResolutionsCache.ts`: cache miss/hit, stat shortcut, size mismatch falling through to hash lookup, hash-fallback on reset stat, content change with matching size, stricter/weaker policy, missing-field policy rejection, multi-verifier policy merge (shared field stored once), worktree case (same content, different path), JSONL append semantics, malformed-line tolerance. - **12 integration tests** in `verifyLockfileResolutions.ts`: dedup of peer/patch-suffix variants, distinct-resolution dedup at the same (name, version), stable violation ordering, the 20-entry cap, multi-verifier fan-out (first failure wins), cache short-circuit on a passing run, no cache write on a rejecting run, empty-verifier-list passthrough. - **1 e2e test** in `pnpm/test/install/minimumReleaseAge.ts`: bundled CLI plumbing — install once to seed the lockfile, enable `minimumReleaseAge` + `cacheDir`, install again, assert the cache file lands at `<cacheDir>/lockfile-verified.jsonl` with the documented record shape. - Existing `minimumReleaseAge` (13) and `frozenLockfile` (12) suites still pass. |
||
|
Ryo Matsukawa
|
31538bf8d2 |
fix: enforce minimumReleaseAge on existing lockfile entries (#11583)
Closes #10438. ## What Re-verify every entry in `pnpm-lock.yaml` against the policies the resolver chain was configured with — today: `minimumReleaseAge` in strict mode — right after the lockfile is loaded from disk and before any tarball is fetched. A locked version that fails the policy aborts the install with `ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION`; `minimumReleaseAgeExclude` is honored. ## Why The policy only fires while pnpm is *choosing* a version. Once a version is pinned in the lockfile — e.g. a developer disabled the policy locally and committed a fresh dependency, or a CI cache restored a stale lockfile — every later `pnpm install` (including `--frozen-lockfile` and `pnpm fetch`) installs it without re-checking, which defeats the supply-chain protection the setting is supposed to provide. The threat model is **a lockfile someone else resolved**, not local resolution: local resolution is already covered by the resolver's own per-version filter. bun fixed the same shape of bug in [oven-sh/bun#30526](https://github.com/oven-sh/bun/pull/30526); this PR is the pnpm side. ## How The fix introduces a generic `ResolutionVerifier` abstraction in the resolver chain — each resolver factory can ship a sibling verifier factory, exactly the way each resolver ships a `resolve` function. Today there's one verifier (npm); the shape leaves room for future ones (jsr, attestation-based, etc.) without changing the install-side interface. - **`@pnpm/resolving.resolver-base`** exports the `ResolutionVerifier` / `ResolutionVerification` types — the shared contract. - **`@pnpm/resolving.npm-resolver`** exports `createNpmResolutionVerifier`. Returns `undefined` when no policy is active, so callers can cheaply decide whether to iterate at all. When active, it inspects each lockfile entry, handles `minimumReleaseAgeExclude`, routes through named-registry prefixes (built-ins like `gh:` merged in), and uses `fetchFullMetadataCached` to fetch full registry metadata — decoupled from the resolver pipeline so neither `peekManifestFromStore` nor abbreviated metadata can hide the publish timestamp. - **`@pnpm/resolving.default-resolver`** exports `createResolutionVerifier`, a combinator that asks each underlying verifier (today: npm) if it has work and returns `undefined` when none does. Designed so that adding more verifiers later doesn't change the install side. - **`@pnpm/installing.client`** exposes `verifyResolution` on `Client`, built from the same `fetchFromRegistry` / `getAuthHeader` the resolver chain already uses — **no second fetcher is constructed**. - **`@pnpm/store.connection-manager`** and **`@pnpm/testing.temp-store`** surface `verifyResolution` alongside the store controller they hand back, so it reaches `mutateModules` through the existing plumbing. - **`@pnpm/installing.deps-installer`** gains one option on `StrictInstallOptions`: `verifyResolution?: ResolutionVerifier`. `mutateModules` invokes `verifyLockfileResolutions(ctx.wantedLockfile, opts.verifyResolution)` **once**, right after `getContext` returns the on-disk lockfile and before any path branches. When the verifier is `undefined`, the call is a no-op. The iteration is policy-neutral: dedupes by `(name, version)`, applies `pLimit(16)`, sorts violations stably, caps the printed list at 20 with an `…and N more` summary, throws a `PnpmError` carrying the verifier-supplied error code. The error includes a recovery hint that points at `pnpm clean --lockfile` followed by `pnpm install` — the safe way to throw away a poisoned lockfile and rebuild from fresh resolution. ## Tests - **9 unit tests** for `verifyLockfileResolutions` against a mock `ResolutionVerifier` — dedup, aggregation, stable ordering, the 20-entry cap, no-op behavior, the verifier-supplied error code surfacing in `PnpmError`. - **13 integration tests** in `installing/deps-installer/test/install/minimumReleaseAge.ts` via the real `install()` entry — `testDefaults()` wires `verifyResolution` from `createTempStore` → `createClient`, so the npm verifier runs end-to-end at the install boundary. Covers the rejection scenario, `minimumReleaseAgeExclude`, the strict-mode toggle, the existing `minimumReleaseAge` resolver-side suite, and a `pnpm add` scenario where a pre-existing entry would otherwise survive resolution. - **3 e2e tests** in `pnpm/test/install/minimumReleaseAge.ts` against the bundled CLI: rejection path with the right `ERR_PNPM_*` code and `pnpm clean --lockfile` hint in output, `minimumReleaseAgeExclude` honored, and the strict-off path (which now requires an explicit `minimumReleaseAgeStrict: false` since the config reader auto-enables strict mode when `minimumReleaseAge` is set). - Existing `frozenLockfile` suite (12 tests) and npm-resolver suite (179 tests) still pass. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> |
||
|
Zoltan Kochan
|
3033bee430 |
refactor(config): split Config interface into settings + runtime context (#11197)
* refactor(config): split Config interface into settings + runtime context
Create ConfigContext for runtime state (hooks, finders, workspace graph,
CLI metadata) and keep Config for user-facing settings only. Functions
use Pick<Config, ...> & Pick<ConfigContext, ...> to express which fields
they need from each interface.
getConfig() now returns { config, context, warnings }. The CLI wrapper
returns { config, context } and spreads both when calling command
handlers (to be refactored to separate params in follow-up PRs).
Closes #11195
* fix: address review feedback
- Initialize cliOptions on pnpmConfig so context.cliOptions is never undefined
- Move rootProjectManifestDir assignment before ignoreLocalSettings guard
- Add allProjectsGraph to INTERNAL_CONFIG_KEYS
* refactor: remove INTERNAL_CONFIG_KEYS from configToRecord
configToRecord now accepts Config and ConfigContext separately, so
context fields are never in scope. Only auth-related Config fields
(authConfig, authInfos, sslConfigs) need filtering.
* refactor: eliminate INTERNAL_CONFIG_KEYS from configToRecord
configToRecord now receives the clean Config object and explicitlySetKeys
separately (via opts.config and opts.context), so context fields are
never in scope. main.ts passes the original split objects alongside
the spread for command handlers that need them.
* fix: spelling
* fix: import sorting
* fix: --config.xxx nconf overrides conflicting with --config CLI flag
When `pnpm add` registers `config: Boolean`, nopt captures
--config.xxx=yyy as the --config flag value instead of treating it
as a nconf-style config override. Fix by extracting --config.xxx args
before nopt parsing and re-parsing them separately.
Also rename the split config/context properties on the command opts
object to _config/_context to avoid clashing with the --config CLI option.
|
||
|
Devendr Mishra
|
74cdef5e46 |
fix: resolve patch file paths during pnpm fetch (#11054)
* fix: ensure patches are applied during pnpm fetch * test: add coverage for patch file resolution during pnpm fetch fallback * fix(test): remove invalid pnpm property in fetch tests * fix: resolve lint errors in fetch test |
||
|
Zoltan Kochan
|
303ca410f5 |
feat!: stop reading settings from the pnpm field of package.json (#10086)
Settings should be read from pnpm-workspace.yaml |
||
|
Zoltan Kochan
|
4a36b9a110 |
refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity. ### Bulk rename (94 packages) All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only. ### Directory renames (removing redundant prefixes) - `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils` - `config/config` → `config/reader`, `config/config-writer` → `config/writer` - `fetching/fetching-types` → `fetching/types` - `lockfile/lockfile-to-pnp` → `lockfile/to-pnp` - `store/store-connection-manager` → `store/connection-manager` - `store/store-controller-types` → `store/controller-types` - `store/store-path` → `store/path` ### Targeted renames (clarity improvements) - `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`) - `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`) - `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`) - `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver` - `installing/get-context` → `installing/context` - `store/package-store` → `store/controller` - `pkg-manifest/manifest-utils` → `pkg-manifest/utils` ### Manifest reader/writer renames - `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`) - `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`) - `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`) - `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`) ### Workspace package renames - `workspace/find-packages` → `workspace/projects-reader` - `workspace/find-workspace-dir` → `workspace/root-finder` - `workspace/resolve-workspace-range` → `workspace/range-resolver` - `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter` ### Domain moves - `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader` - `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer` - `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest` ### Scope - 1206 files changed - Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering |
||
|
Zoltan Kochan
|
7a304b17c4 |
refactor: rename directories and unify command packages per domain (#10993)
- Rename `installing/core` → `installing/deps-installer` and `installing/headless` → `installing/deps-restorer` for clearer naming
- Rename all `plugin-commands-*` directories to use `-commands` suffix convention
- Merge multiple command packages per domain into a single `commands/` directory (one commands package per domain rule):
- `building/{build-commands,policy-commands}` → `building/commands`
- `deps/compliance/{audit-commands,licenses-commands,sbom-commands}` → `deps/compliance/commands`
- `deps/inspection/{listing-commands,outdated-commands}` → `deps/inspection/commands`
- `store/{store-commands,inspecting-commands}` → `store/commands`
- `releasing/{publish-commands,deploy-commands}` → `releasing/commands`
- `cli/{completion-commands,doctor-commands}` → `cli/commands`
- `engine/pm/{self-updater-commands,setup-commands}` → `engine/pm/commands`
- `engine/runtime/{runtime-commands,env-commands}` → `engine/runtime/commands`
- `cache/cache-commands` → `cache/commands`
- Fix relative paths in merged test files (pnpmBin, __typings__ references)
- Update jest config to ignore `utils/` dirs at any nesting depth under `test/`
- Fix stale package names in changeset files
|