Commit Graph

80 Commits

Author SHA1 Message Date
Zoltan Kochan
ae2175829a feat(registry-access): extract dist-tag + adduser helpers, dogfood from tests (#11926)
* feat(registry-access): extract setDistTag and dogfood from tests

Add `@pnpm/registry-access.commands#setDistTag` — the low-level PUT to
`/-/package/:pkg/dist-tags/:tag`. The CLI `dist-tag add` handler now
calls it instead of issuing the fetch inline.

Tests in this monorepo now use a thin new package
`@pnpm/testing.registry-mock` (REGISTRY_MOCK_PORT + REGISTRY_MOCK_CREDENTIALS
baked in) that delegates to `setDistTag`, replacing `addDistTag` from
`@pnpm/registry-mock`. That dropped helper relied on
`anonymous-npm-registry-client` and a verdaccio-era
fetch-then-DELETE-then-PUT dance that is no longer needed against
pnpm-registry.

39 test files swapped from `@pnpm/registry-mock` to
`@pnpm/testing.registry-mock`.

* fix: move setDistTag to its own package to break tsconfig project-reference cycle

testing/registry-mock → registry-access.commands → releasing/commands
→ installing/commands → installing/deps-installer → testing/registry-mock.

Extract setDistTag into @pnpm/registry-access.set-dist-tag (only depends
on @pnpm/error, @pnpm/network.fetch, @pnpm/npm-package-arg). Both
@pnpm/registry-access.commands and @pnpm/testing.registry-mock import
from it. Cycle gone.

* feat(registry-access): extract addUser helper, dogfood from login + tests

Add @pnpm/registry-access.add-user — a small helper that PUTs to
/-/user/org.couchdb.user:<name> and returns { token }. The CLI's
classicLogin (pnpm login fallback path) now calls it, and tests
use it via @pnpm/testing.registry-mock instead of the legacy
addUser from @pnpm/registry-mock.

Swapped 3 call sites: globalSetup.js, installing/deps-installer's
auth.ts, and pnpm/test/dlx.ts. AddUserHttpError exposes status +
text + parsed-json-if-applicable + headers so the CLI can still
do its OTP detection. One webauth-OTP login test mock had to be
adjusted to provide its body via `text` (JSON-stringified) rather
than `json` only, since the helper consumes the body via `text()`.

* refactor: consolidate set-dist-tag + add-user helpers into one @pnpm/registry-access.client package

One shared package is better than splitting per endpoint. Future endpoints
(publish, deprecate, etc.) can land here without another wrapper.

No behavioral change — same setDistTag and addUser exports as before,
just under one roof. Callers updated: registry-access.commands,
auth.commands, testing.registry-mock.

* fix(registry-access): sort imports
2026-05-25 14:01:00 +02:00
Zoltan Kochan
f2a4d2caef chore(release): 11.3.0 (#11894) 2026-05-24 02:23:07 +02:00
Zoltan Kochan
11a43b15da chore(release): 11.2.1 (#11777) 2026-05-20 16:51:13 +02:00
Zoltan Kochan
0fb723323f chore(release): 11.2.0 (#11764) 2026-05-20 12:41:09 +02:00
Zoltan Kochan
cd80b2c8ae chore(release): 11.1.3 (#11717) 2026-05-18 15:42:32 +02:00
Zoltan Kochan
fcf95c7faa perf: cache the post-resolution lockfile verification gate (#11691)
Closes #11687.

## What

Cache the result of the post-resolution lockfile verification gate (#11583) so repeat installs against an unchanged lockfile skip the per-package registry round trips entirely. Persisted as JSON Lines at `<cacheDir>/lockfile-verified.jsonl`.

The cache layer is policy-neutral. Today there's one verifier (`minimumReleaseAge`); future resolver-side verifiers (jsr trust, attestation, …) plug in by declaring their own `policy` slot and `canTrustPastCheck` comparator — no install-side changes.

## Why

#11583 re-hits the registry on every install for every locked (name, version) pair. On warm/repeat installs where the lockfile hasn't moved, that's a stack of per-package round trips with nothing to show for them. This change makes the steady-state case effectively free without weakening the protection — the gate still runs in full whenever the lockfile changes, any verifier's policy tightens, or no record exists.

## How

### Cache lookup, in order

The cache is **indexed by content hash** so git worktrees with identical lockfile bytes share a cache entry. A secondary path-keyed index drives the same-machine stat shortcut.

1. **`stat()` shortcut** — when a previous record for this exact `lockfilePath` matches today's `size + mtime + inode`, trust the cached hash without reading anything. Zero I/O beyond the stat. Microseconds.
2. **Content lookup** — hash the in-memory lockfile (not the file bytes — we already have the parsed object) and look up by content hash. Catches worktrees (same content, different path) and CI checkouts (same content, reset stat). On hit, append a refreshed path/stat entry so the next install at this path takes the stat shortcut.
3. **Any active verifier rejects the cached `policy`** — run the full gate.
4. **No record** — run the full gate.

The in-memory object is hashed with `hashObject` from `@pnpm/crypto.object-hasher` (streaming, key-order-stable).

### Record shape

```json
{
  "lockfile": {
    "hash": "<sha256 base64>",
    "path": "/abs/path/to/pnpm-lock.yaml",
    "size": 154,
    "mtimeNs": "1736245123000000000",
    "inode": "12345"
  },
  "verifiedAt": "2026-05-17T...",
  "policy": { "minimumReleaseAge": 1440 }
}
```

`policy` is the union of every active verifier's `policy` contribution. Verifiers checking the same logical policy (e.g. `minimumReleaseAge` honored by multiple registries) name it the same and share the slot — no resolver namespacing.

### File semantics

- **Sync fs throughout** — the cache is consulted once before verification fan-out and recorded once after. No concurrent install work to overlap with; keeping the call sites straight-line.
- **JSONL appends are atomic** on POSIX/NTFS, so parallel pnpm processes (monorepo installs, CI matrices sharing a cache) write without coordination. Latest record per `(path, hash)` tuple wins on read.
- **Bounded file** — capped at ~1000 entries; compaction is triggered by a single `stat()` of the cache file (1.5 MiB byte budget) so we never parse the file on the steady-state path. When triggered, the tail is rewritten via tempfile + rename.
- **No record on rejection** — a failing verification deliberately doesn't write a record; the next install must rerun the gate.
- **Single hash per install** — the in-memory hash is computed lazily and reused: `tryLockfileVerificationCache` returns the precomputed stat+hash to `recordVerification` on a miss, and the stat-shortcut hit forwards the cached record's hash unchanged.

## Plumbing

The verifier contract changed alongside the cache to make this composable without install-side knowledge of each policy:

- **`@pnpm/resolving.resolver-base`** — `ResolutionVerifier` is now `{ verify, policy, canTrustPastCheck }` (was a bare function in #11583). Each resolver-side verifier owns its policy snapshot and the comparator that decides whether a cached policy is still trustworthy.
- **`@pnpm/resolving.npm-resolver`** — `createNpmResolutionVerifier` returns the new shape: `policy: { minimumReleaseAge }`, `canTrustPastCheck` reads `minimumReleaseAge` from the merged cached bag.
- **`@pnpm/resolving.default-resolver`** — `createResolutionVerifier` (singular, returning a combined function) → `createResolutionVerifiers` (plural, returning a `ResolutionVerifier[]`). No combinator; each verifier handles its own protocol short-circuit inside `verify`, so dispatch happens naturally at the install side.
- **`@pnpm/installing.client`** — `Client.verifyResolution?` → `Client.resolutionVerifiers: ResolutionVerifier[]`. Same rename propagates through `@pnpm/store.connection-manager`, `@pnpm/testing.temp-store`, and `StrictInstallOptions`.
- **`@pnpm/installing.deps-installer`** — new `verifyLockfileResolutionsCache.ts` (`tryLockfileVerificationCache` + `recordVerification`). `verifyLockfileResolutions` takes the verifier list plus `cacheDir` + `lockfilePath` as flat options; the cache fires when both are present, otherwise the gate runs without memoization. The dedup key for in-flight candidates includes a serialization of `resolution` so two entries sharing a (name, version) but pinned via different protocols don't collapse.

Breaking but safe — `@pnpm/resolving.npm-resolver` hasn't been released since #11583 introduced the verifier abstraction, so no downstream consumer is on the old shape.

## Tests

- **17 unit tests** in `verifyLockfileResolutionsCache.ts`: cache miss/hit, stat shortcut, size mismatch falling through to hash lookup, hash-fallback on reset stat, content change with matching size, stricter/weaker policy, missing-field policy rejection, multi-verifier policy merge (shared field stored once), worktree case (same content, different path), JSONL append semantics, malformed-line tolerance.
- **12 integration tests** in `verifyLockfileResolutions.ts`: dedup of peer/patch-suffix variants, distinct-resolution dedup at the same (name, version), stable violation ordering, the 20-entry cap, multi-verifier fan-out (first failure wins), cache short-circuit on a passing run, no cache write on a rejecting run, empty-verifier-list passthrough.
- **1 e2e test** in `pnpm/test/install/minimumReleaseAge.ts`: bundled CLI plumbing — install once to seed the lockfile, enable `minimumReleaseAge` + `cacheDir`, install again, assert the cache file lands at `<cacheDir>/lockfile-verified.jsonl` with the documented record shape.
- Existing `minimumReleaseAge` (13) and `frozenLockfile` (12) suites still pass.
2026-05-17 13:07:24 +02:00
Ryo Matsukawa
31538bf8d2 fix: enforce minimumReleaseAge on existing lockfile entries (#11583)
Closes #10438.

## What

Re-verify every entry in `pnpm-lock.yaml` against the policies the resolver chain was configured with — today: `minimumReleaseAge` in strict mode — right after the lockfile is loaded from disk and before any tarball is fetched. A locked version that fails the policy aborts the install with `ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION`; `minimumReleaseAgeExclude` is honored.

## Why

The policy only fires while pnpm is *choosing* a version. Once a version is pinned in the lockfile — e.g. a developer disabled the policy locally and committed a fresh dependency, or a CI cache restored a stale lockfile — every later `pnpm install` (including `--frozen-lockfile` and `pnpm fetch`) installs it without re-checking, which defeats the supply-chain protection the setting is supposed to provide.

The threat model is **a lockfile someone else resolved**, not local resolution: local resolution is already covered by the resolver's own per-version filter. bun fixed the same shape of bug in [oven-sh/bun#30526](https://github.com/oven-sh/bun/pull/30526); this PR is the pnpm side.

## How

The fix introduces a generic `ResolutionVerifier` abstraction in the resolver chain — each resolver factory can ship a sibling verifier factory, exactly the way each resolver ships a `resolve` function. Today there's one verifier (npm); the shape leaves room for future ones (jsr, attestation-based, etc.) without changing the install-side interface.

- **`@pnpm/resolving.resolver-base`** exports the `ResolutionVerifier` / `ResolutionVerification` types — the shared contract.
- **`@pnpm/resolving.npm-resolver`** exports `createNpmResolutionVerifier`. Returns `undefined` when no policy is active, so callers can cheaply decide whether to iterate at all. When active, it inspects each lockfile entry, handles `minimumReleaseAgeExclude`, routes through named-registry prefixes (built-ins like `gh:` merged in), and uses `fetchFullMetadataCached` to fetch full registry metadata — decoupled from the resolver pipeline so neither `peekManifestFromStore` nor abbreviated metadata can hide the publish timestamp.
- **`@pnpm/resolving.default-resolver`** exports `createResolutionVerifier`, a combinator that asks each underlying verifier (today: npm) if it has work and returns `undefined` when none does. Designed so that adding more verifiers later doesn't change the install side.
- **`@pnpm/installing.client`** exposes `verifyResolution` on `Client`, built from the same `fetchFromRegistry` / `getAuthHeader` the resolver chain already uses — **no second fetcher is constructed**.
- **`@pnpm/store.connection-manager`** and **`@pnpm/testing.temp-store`** surface `verifyResolution` alongside the store controller they hand back, so it reaches `mutateModules` through the existing plumbing.
- **`@pnpm/installing.deps-installer`** gains one option on `StrictInstallOptions`: `verifyResolution?: ResolutionVerifier`. `mutateModules` invokes `verifyLockfileResolutions(ctx.wantedLockfile, opts.verifyResolution)` **once**, right after `getContext` returns the on-disk lockfile and before any path branches. When the verifier is `undefined`, the call is a no-op. The iteration is policy-neutral: dedupes by `(name, version)`, applies `pLimit(16)`, sorts violations stably, caps the printed list at 20 with an `…and N more` summary, throws a `PnpmError` carrying the verifier-supplied error code.

The error includes a recovery hint that points at `pnpm clean --lockfile` followed by `pnpm install` — the safe way to throw away a poisoned lockfile and rebuild from fresh resolution.

## Tests

- **9 unit tests** for `verifyLockfileResolutions` against a mock `ResolutionVerifier` — dedup, aggregation, stable ordering, the 20-entry cap, no-op behavior, the verifier-supplied error code surfacing in `PnpmError`.
- **13 integration tests** in `installing/deps-installer/test/install/minimumReleaseAge.ts` via the real `install()` entry — `testDefaults()` wires `verifyResolution` from `createTempStore` → `createClient`, so the npm verifier runs end-to-end at the install boundary. Covers the rejection scenario, `minimumReleaseAgeExclude`, the strict-mode toggle, the existing `minimumReleaseAge` resolver-side suite, and a `pnpm add` scenario where a pre-existing entry would otherwise survive resolution.
- **3 e2e tests** in `pnpm/test/install/minimumReleaseAge.ts` against the bundled CLI: rejection path with the right `ERR_PNPM_*` code and `pnpm clean --lockfile` hint in output, `minimumReleaseAgeExclude` honored, and the strict-off path (which now requires an explicit `minimumReleaseAgeStrict: false` since the config reader auto-enables strict mode when `minimumReleaseAge` is set).
- Existing `frozenLockfile` suite (12 tests) and npm-resolver suite (179 tests) still pass.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-05-16 21:38:06 +02:00
Zoltan Kochan
8a80235c7b chore(release): 11.1.2 2026-05-14 13:31:53 +02:00
Zoltan Kochan
9a327522ce chore(release): 11.1.1 2026-05-12 12:56:32 +02:00
Zoltan Kochan
732312f49e chore(release): 11.1.0 2026-05-11 19:56:10 +02:00
Zoltan Kochan
f2b28f85ff chore(release): 11.0.9 2026-05-09 02:06:35 +02:00
Zoltan Kochan
a516c24ce4 chore(release): 11.0.8 2026-05-07 08:35:07 +02:00
Zoltan Kochan
0c3ef0ec94 chore(release): 11.0.7 2026-05-07 00:21:03 +02:00
Zoltan Kochan
65f9327014 chore(release): 11.0.6 2026-05-05 19:50:32 +02:00
Zoltan Kochan
3a5534d75e chore(release): 11.0.4 2026-05-03 01:24:22 +02:00
Zoltan Kochan
6ef34b7a11 chore(release): 11.0.3 2026-04-30 23:03:46 +02:00
Zoltan Kochan
a53f78b111 chore(release): 11.0.2 2026-04-30 17:16:34 +02:00
Zoltan Kochan
38ffda2a18 chore(release): 11.0.1 2026-04-29 23:00:21 +02:00
Zoltan Kochan
8aeeff4c46 chore(release): 11.0.0 2026-04-28 11:27:43 +02:00
Zoltan Kochan
fd437ded13 chore(release): 11.0.0-rc.4 2026-04-21 15:03:02 +02:00
Zoltan Kochan
aa93759d9b chore(release): drop eslint from lib prepublishOnly (#11320)
Library packages had `prepublishOnly: pn compile`, which expands to
`tsgo --build && pn lint --fix`. During `pn release` that runs eslint
against ~150 packages for no benefit — the code has already been linted
in CI and the release flow's upfront compile has already built dist/.
Switch lib prepublishOnly to a bare `tsgo --build` so the safety-net
compile stays but the per-package eslint cost is gone.
2026-04-21 01:18:03 +02:00
Zoltan Kochan
fcdd50aaa7 chore(release): 11.0.0-rc.3 2026-04-21 00:17:38 +02:00
Zoltan Kochan
9e0833c3cc feat: add minimumReleaseAgeIgnoreMissingTime setting (#11293)
Skips the minimumReleaseAge maturity check when the registry metadata
lacks the "time" field, instead of throwing ERR_PNPM_MISSING_TIME.
Defaults to true, and prints a warning once per affected package.
2026-04-19 00:22:32 +02:00
Zoltan Kochan
96ece9d736 chore(release): 11.0.0-rc.2 2026-04-17 18:21:35 +02:00
Zoltan Kochan
f7c23231a9 chore(release): 11.0.0-rc.1 2026-04-16 01:18:55 +02:00
Zoltan Kochan
06d6c2d405 chore(release): 11.0.0-rc.0 2026-04-10 18:30:33 +02:00
Zoltan Kochan
b103439d9a refactor(test): extract shared DEFAULT_OPTS into @pnpm/testing.command-defaults (#11208)
12 command test suites had near-identical ~50-field DEFAULT_OPTS objects
copy-pasted between them. Extract the common fields into a single shared
package so each suite only declares its overrides.
2026-04-05 21:22:24 +02:00
Zoltan Kochan
45a6cb6b2a refactor(auth): unify auth/SSL into structured configByUri (#11201)
Replaces the dual `authConfig` (raw .npmrc) + `authInfos` (parsed auth) + `sslConfigs` (parsed SSL) pattern with a single structured `configByUri: Record<string, RegistryConfig>` field on Config.

### New types (`@pnpm/types`)
- **`RegistryConfig`** — per-registry config: `{ creds?: Creds, tls?: TlsConfig }`
- **`Creds`** — auth credentials: `{ authToken?, basicAuth?, tokenHelper? }`
- **`TlsConfig`** — TLS config: `{ cert?, key?, ca? }`

### Key changes
- Rewrite `createGetAuthHeaderByURI` to accept `Record<string, RegistryConfig>` instead of raw .npmrc key-value pairs
- Eliminate duplicate auth parsing between `getAuthHeadersFromConfig` and `getNetworkConfigs`
- Remove `authConfig` from the install pipeline (`StrictInstallOptions`, `HeadlessOptions`), replaced by `configByUri`
- Remove `sslConfigs` from Config — SSL fields now live in `configByUri[uri].tls`
- Remove `authConfig['registry']` mutation in `extendInstallOptions` (default registry now passed directly to `createGetAuthHeaderByURI`)
- `authConfig` remains on Config only for raw .npmrc access (config commands, error reporting, config inheritance)

### Security
- tokenHelper in project .npmrc now throws instead of being silently stripped
- tokenHelper execution uses `shell: false` to prevent shell metacharacter injection
- Basic auth uses `Buffer.from().toString('base64')` instead of `btoa()` for Unicode safety
- Dispatcher only creates custom agents when entries actually have TLS fields
2026-04-05 20:15:10 +02:00
Zoltan Kochan
96704a1c58 refactor(config): rename rawConfig to authConfig, add nodeDownloadMirrors, simplify config reader (#11194)
Major cleanup of the config system after migrating settings from `.npmrc` to `pnpm-workspace.yaml`.

### Config reader simplification
- Remove `checkUnknownSetting` (dead code, always `false`)
- Trim `npmConfigTypes` from ~127 to ~67 keys (remove unused npm config keys)
- Replace `rcOptions` iteration over all type keys with direct construction from defaults + auth overlay
- Remove `rcOptionsTypes` parameter from `getConfig()` and its assembly chain

### Rename `rawConfig` to `authConfig`
- `rawConfig` was a confusing mix of auth data and general settings
- Non-auth settings are already on the typed `Config` object — stop duplicating them in `rawConfig`
- Rename `rawConfig` → `authConfig` across the codebase to clarify it only contains auth/registry data from `.npmrc`

### Remove `rawConfig` from non-auth consumers
- **Lifecycle hooks**: replace `rawConfig: object` with `userAgent?: string` — only user-agent was read
- **Fetchers**: remove unused `rawConfig` from git fetcher, binary fetcher, tarball fetcher, prepare-package
- **Update command**: use `opts.production/dev/optional` instead of `rawConfig.*`
- **`pnpm init`**: accept typed init properties instead of parsing `rawConfig`

### Add `nodeDownloadMirrors` setting
- New `nodeDownloadMirrors?: Record<string, string>` on `PnpmSettings` and `Config`
- Replaces the `node-mirror:<channel>` pattern that was stored in `rawConfig`
- Configured in `pnpm-workspace.yaml`:
  ```yaml
  nodeDownloadMirrors:
    release: https://my-mirror.example.com/download/release/
  ```
- Remove unused `rawConfig` from deno-resolver and bun-resolver

### Refactor `pnpm config get/list`
- New `configToRecord()` builds display data from typed Config properties on the fly
- Excludes sensitive internals (`authInfos`, `sslConfigs`, etc.)
- Non-types keys (e.g., `package-extensions`) resolve through `configToRecord` instead of direct property access
- Delete `processConfig.ts` (replaced by `configToRecord.ts`)

### Pre-push hook improvement
- Add `compile-only` (`tsgo --build`) to pre-push hook to catch type errors before push
2026-04-04 20:33:43 +02:00
Zoltan Kochan
6c480a4375 perf: replace node-fetch with undici (#10537)
Replace node-fetch with native undici for HTTP requests throughout pnpm.

Key changes:
- Replace node-fetch with undici's fetch() and dispatcher system
- Replace @pnpm/network.agent with a new dispatcher module in @pnpm/network.fetch
- Cache dispatchers via LRU cache keyed by connection parameters
- Handle proxies via undici ProxyAgent instead of http/https-proxy-agent
- Convert test mocking from nock to undici MockAgent where applicable
- Add minimatch@9 override to fix ESM incompatibility with brace-expansion
2026-03-29 12:44:00 +02:00
Zoltan Kochan
d6b8e281b6 chore: use pn instead of pnpm (#11124) 2026-03-28 11:55:51 +01:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
f47ef4b125 refactor: reorganize monorepo domain structure (#10987)
Reorganize the monorepo's top-level domain directories for clarity:

- pkg-manager/ split into:
  - installing/ (core, headless, client, resolve-dependencies, etc.)
  - installing/linking/ (hoist, direct-dep-linker, modules-cleaner, etc.)
  - bins/ (link-bins, package-bins, remove-bins)
- completion/ merged into cli/
- dedupe/ moved to installing/dedupe/
- env/ renamed to engine/ with subdomains:
  - engine/runtime/ (node.fetcher, node.resolver, plugin-commands-env, etc.)
  - engine/pm/ (plugin-commands-setup, plugin-commands-self-updater)
- env.path moved to shell/
- tools/ and runtime/ dissolved
- reviewing/ and lockfile audit packages moved to deps/:
  - deps/inspection/ (list, outdated, dependencies-hierarchy)
  - deps/compliance/ (audit, licenses, sbom)
- registry/ moved to resolving/registry/
- semver/peer-range moved to deps/
- network/fetching-types moved to fetching/
- packages/ slimmed down, moving packages to proper domains:
  - calc-dep-state, dependency-path -> deps/
  - parse-wanted-dependency -> resolving/
  - git-utils -> network/
  - naming-cases -> text/
  - make-dedicated-lockfile -> lockfile/
  - render-peer-issues -> installing/
  - plugin-commands-doctor -> cli/
  - plugin-commands-init -> workspace/

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 13:45:54 +01:00
Zoltan Kochan
5d5818e44f style: enforce node: protocol for builtin imports (#10951)
Add n/prefer-node-protocol rule and autofix all bare builtin imports
to use the node: prefix. Simplify the simple-import-sort builtins
pattern to just ^node: since all imports now use the prefix.
2026-03-13 07:59:51 +01:00
Zoltan Kochan
1c8c4e49f5 style: add eslint-plugin-simple-import-sort (#10947)
Add eslint-plugin-simple-import-sort to enforce consistent import ordering:
- Node.js builtins first
- External packages second
- Relative imports last
- Named imports sorted alphabetically within each statement
2026-03-13 02:02:38 +01:00
Brandon Cheng
01914345d5 build: enable @typescript-eslint/no-import-type-side-effects (#10630)
* build: enable `@typescript-eslint/no-import-type-side-effects`

* build: disable `@typescript-eslint/consistent-type-imports`

* chore: apply fixes for `no-import-type-side-effects`

pnpm exec eslint "**/src/**/*.ts" "**/test/**/*.ts" --fix
2026-03-08 00:02:48 +01:00
Zoltan Kochan
b7f0f21582 feat: use SQLite for storing package index in the content-addressable store (#10827)
## Summary

Replace individual `.mpk` (MessagePack) files under `$STORE/index/` with a single SQLite database at `$STORE/index.db` using Node.js 22's built-in `node:sqlite` module. This reduces filesystem syscall overhead and improves space efficiency for small metadata entries.

Closes #10826

## Design

### New package: `@pnpm/store.index`

A new `StoreIndex` class wraps a SQLite database with a simple key-value API (`get`, `set`, `delete`, `has`, `entries`). Data is serialized with msgpackr and stored as BLOBs. The table uses `WITHOUT ROWID` for compact storage.

Key design decisions:

- **WAL mode** enables concurrent reads from workers while the main process writes.
- **`busy_timeout=5000`** plus a retry loop with `Atomics.wait`-based `sleepSync` handles `SQLITE_BUSY` errors from concurrent access.
- **Performance PRAGMAs**: `synchronous=NORMAL`, `mmap_size=512MB`, `cache_size=32MB`, `temp_store=MEMORY`, `wal_autocheckpoint=10000`.
- **Write batching**: `queueWrites()` batches pre-packed entries from tarball extraction and flushes them in a single transaction on `process.nextTick`. `setRawMany()` writes immediate batches (e.g. from `addFilesFromDir`).
- **Lifecycle**: `close()` auto-flushes pending writes, runs `PRAGMA optimize`, and closes the DB. A `process.on('exit')` handler ensures cleanup even on unexpected exits.
- **`VACUUM` after `deleteMany`** (used by `pnpm store prune`) to reclaim disk space.

### Key format

Keys are `integrity\tpkgId` (tab-separated). Git-hosted packages use `pkgId\tbuilt` or `pkgId\tnot-built`.

### Shared StoreIndex instance

A single `StoreIndex` instance is threaded through the entire install lifecycle — from `createNewStoreController` through the fetcher chain, package requester, license scanner, SBOM collector, and dependencies hierarchy. This replaces the previous pattern of each component creating its own file-based index access.

### Worker architecture

Index writes are performed in the main process, not in worker threads. Workers send pre-packed `{ key, buffer }` pairs back to the main process via `postMessage`, where they are batched and flushed to SQLite. This avoids SQLite write contention between threads.

### SQLite ExperimentalWarning suppression

`node:sqlite` emits an `ExperimentalWarning` on first load. This is suppressed via a `process.emitWarning` override injected through esbuild's `banner` option, which runs on line 1 of both `dist/pnpm.mjs` and `dist/worker.js` — before any module that loads `node:sqlite`.

### No migration from `.mpk` files

Old `.mpk` index files are not migrated. Packages missing from the new SQLite index are re-fetched on demand (the same behavior as a fresh store).

## Changed packages

121 files changed across these areas:

- **`store/index/`** — New `@pnpm/store.index` package
- **`worker/`** — Write batching moved from worker module into `StoreIndex` class; workers send pre-packed buffers to main process
- **`store/package-store/`** — StoreIndex creation and lifecycle management
- **`store/cafs/`** — Removed `getFilePathInCafs` index-file utilities (no longer needed)
- **`store/pkg-finder/`** — Reads from StoreIndex instead of `.mpk` files
- **`store/plugin-commands-store/`** — `store status` uses StoreIndex
- **`store/plugin-commands-store-inspecting/`** — `cat-index` and `find-hash` use StoreIndex
- **`fetching/tarball-fetcher/`** — Threads StoreIndex through fetchers; git-hosted fetcher flushes before reading
- **`fetching/git-fetcher/`, `binary-fetcher/`, `pick-fetcher/`** — Accept StoreIndex parameter
- **`pkg-manager/`** — `client`, `core`, `headless`, `package-requester` thread StoreIndex
- **`reviewing/`** — `license-scanner`, `sbom`, `dependencies-hierarchy` accept StoreIndex
- **`cache/api/`** — Cache view uses StoreIndex
- **`pnpm/bundle.ts`** — esbuild banner for ExperimentalWarning suppression

## Test plan

- [x] `pnpm --filter @pnpm/store.index test` — Unit tests for StoreIndex CRUD and batching
- [x] `pnpm --filter @pnpm/package-store test` — Store controller lifecycle
- [x] `pnpm --filter @pnpm/package-requester test` — Package requester reads from SQLite index
- [x] `pnpm --filter @pnpm/tarball-fetcher test` — Tarball and git-hosted fetcher writes
- [x] `pnpm --filter @pnpm/headless test` — Headless install
- [x] `pnpm --filter @pnpm/core test` — Core install, side effects, patching
- [x] `pnpm --filter @pnpm/plugin-commands-rebuild test` — Rebuild reads from index
- [x] `pnpm --filter @pnpm/license-scanner test` — License scanning
- [x] e2e tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-06 12:59:04 +01:00
Zoltan Kochan
e18a879d72 feat!: drop Node.js 22.12 support 2026-02-18 14:54:09 +01:00
Zoltan Kochan
1b4df57a01 feat!: drop Node.js 20 and 21 support (#10569) 2026-02-08 19:16:24 +01:00
Zoltan Kochan
a00f9e515c chore: use typescript-go (#10452) 2026-01-14 01:18:13 +01:00
Zoltan Kochan
0bcbaf9994 refactor: move out skip resolution logic from package requester (#10439) 2026-01-12 13:08:50 +01:00
Zoltan Kochan
7e2910e70f chore(release): 11.0.0-alpha.0 2025-11-13 15:44:27 +01:00
Zoltan Kochan
3ce5f82bd7 Merge remote-tracking branch 'origin/main' into v11 2025-10-28 18:40:05 +01:00
Zoltan Kochan
49f03d14ee chore(release): 10.20.0 2025-10-28 17:35:21 +01:00
Zoltan Kochan
dab9abef5c Merge remote-tracking branch 'origin/main' into v11 2025-10-24 14:19:07 +02:00
Zoltan Kochan
0cde1287c8 chore: update repository fields 2025-10-23 11:57:12 +02:00
Zoltan Kochan
43d7b18c2f chore(release): 10.19.0 2025-10-21 15:30:20 +02:00
Zoltan Kochan
1bfc105da0 chore(release): 10.18.3 2025-10-14 11:27:45 +02:00
Zoltan Kochan
a43166624e Merge remote-tracking branch 'origin/main' into v11 2025-10-10 10:01:19 +02:00
Zoltan Kochan
1b15e45ae9 chore(release): 10.18.2 2025-10-09 16:56:04 +02:00