Commit Graph

53 Commits

Author SHA1 Message Date
Zoltan Kochan
96704a1c58 refactor(config): rename rawConfig to authConfig, add nodeDownloadMirrors, simplify config reader (#11194)
Major cleanup of the config system after migrating settings from `.npmrc` to `pnpm-workspace.yaml`.

### Config reader simplification
- Remove `checkUnknownSetting` (dead code, always `false`)
- Trim `npmConfigTypes` from ~127 to ~67 keys (remove unused npm config keys)
- Replace `rcOptions` iteration over all type keys with direct construction from defaults + auth overlay
- Remove `rcOptionsTypes` parameter from `getConfig()` and its assembly chain

### Rename `rawConfig` to `authConfig`
- `rawConfig` was a confusing mix of auth data and general settings
- Non-auth settings are already on the typed `Config` object — stop duplicating them in `rawConfig`
- Rename `rawConfig` → `authConfig` across the codebase to clarify it only contains auth/registry data from `.npmrc`

### Remove `rawConfig` from non-auth consumers
- **Lifecycle hooks**: replace `rawConfig: object` with `userAgent?: string` — only user-agent was read
- **Fetchers**: remove unused `rawConfig` from git fetcher, binary fetcher, tarball fetcher, prepare-package
- **Update command**: use `opts.production/dev/optional` instead of `rawConfig.*`
- **`pnpm init`**: accept typed init properties instead of parsing `rawConfig`

### Add `nodeDownloadMirrors` setting
- New `nodeDownloadMirrors?: Record<string, string>` on `PnpmSettings` and `Config`
- Replaces the `node-mirror:<channel>` pattern that was stored in `rawConfig`
- Configured in `pnpm-workspace.yaml`:
  ```yaml
  nodeDownloadMirrors:
    release: https://my-mirror.example.com/download/release/
  ```
- Remove unused `rawConfig` from deno-resolver and bun-resolver

### Refactor `pnpm config get/list`
- New `configToRecord()` builds display data from typed Config properties on the fly
- Excludes sensitive internals (`authInfos`, `sslConfigs`, etc.)
- Non-types keys (e.g., `package-extensions`) resolve through `configToRecord` instead of direct property access
- Delete `processConfig.ts` (replaced by `configToRecord.ts`)

### Pre-push hook improvement
- Add `compile-only` (`tsgo --build`) to pre-push hook to catch type errors before push
2026-04-04 20:33:43 +02:00
Zoltan Kochan
6c480a4375 perf: replace node-fetch with undici (#10537)
Replace node-fetch with native undici for HTTP requests throughout pnpm.

Key changes:
- Replace node-fetch with undici's fetch() and dispatcher system
- Replace @pnpm/network.agent with a new dispatcher module in @pnpm/network.fetch
- Cache dispatchers via LRU cache keyed by connection parameters
- Handle proxies via undici ProxyAgent instead of http/https-proxy-agent
- Convert test mocking from nock to undici MockAgent where applicable
- Add minimatch@9 override to fix ESM incompatibility with brace-expansion
2026-03-29 12:44:00 +02:00
Zoltan Kochan
d6b8e281b6 chore: use pn instead of pnpm (#11124) 2026-03-28 11:55:51 +01:00
Zoltan Kochan
cd2dc7d481 refactor: prefix internal scripts with . to hide them (#11051)
* fix: ensure PNPM_HOME/bin is in PATH during pnpm setup

When upgrading from old pnpm (global bin = PNPM_HOME) to new pnpm
(global bin = PNPM_HOME/bin), `pnpm setup` would fail because the
spawned `pnpm add -g` checks that the global bin dir is in PATH.
Prepend PNPM_HOME/bin to PATH in the spawned process env so the
check passes during the transition.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update pnpm to v11 beta 2

* chore: update pnpm to v11 beta 2

* chore: update pnpm to v11 beta 2

* chore: update pnpm to v11 beta 2

* fix: lint

* refactor: rename _-prefixed scripts to .-prefixed scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update root package.json to use .test instead of _test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ci: update action-setup

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 14:30:56 +01:00
Zoltan Kochan
1701a65845 chore: reduce noisy warnings in test output (#11022)
* chore: reduce noisy warnings in test output

- Suppress ExperimentalWarning and DEP0169 via --disable-warning in NODE_OPTIONS
- Fix MaxListenersExceededWarning by raising limit in StoreIndex when adding exit listeners
- Update meta-updater to generate the new _test scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: stop streaming pnpm subprocess output during CLI tests

Buffer stdout/stderr from execPnpm instead of writing to the parent
process in real time. Output is still included in the error message on
failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: pipe all subprocess output in CLI tests

Use stdio: 'pipe' for all pnpm/pnpx spawn helpers so subprocess output
is buffered instead of printed. Output is still included in error
messages on failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove duplicate @pnpm/installing.env-installer in pnpm/package.json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: use pipe stdio in dlx and errorHandler tests

Replace stdio: 'inherit' and [null, 'pipe', 'inherit'] with 'pipe' to
prevent subprocess output from leaking into test output.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: skip maxListeners adjustment when set to unlimited (0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 10:43:12 +01:00
Zoltan Kochan
dba4153767 refactor: rename packages and consolidate runtime resolvers (#10999)
* refactor: rename workspace.sort-packages and workspace.pkgs-graph

- workspace.sort-packages -> workspace.projects-sorter
- workspace.pkgs-graph -> workspace.projects-graph

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rename packages/ to core/ and pkg-manifest.read-package-json to reader

- Rename packages/ directory to core/ for clarity
- Rename pkg-manifest/read-package-json to pkg-manifest/reader (@pnpm/pkg-manifest.reader)
- Update all tsconfig, package.json, and lockfile references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: consolidate runtime resolvers under engine/runtime domain

- Remove unused @pnpm/engine.runtime.node.fetcher package
- Rename engine/runtime/node.resolver to node-resolver (dash convention)
- Move resolving/bun-resolver to engine/runtime/bun-resolver
- Move resolving/deno-resolver to engine/runtime/deno-resolver
- Update all package names, tsconfig paths, and lockfile references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update lockfile after removing node.fetcher

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sort tsconfig references and package.json deps alphabetically

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: auto-fix import sorting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update __typings__ paths in tsconfig.lint.json for moved resolvers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove deno-resolver from deps of bun-resolver

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 00:19:58 +01:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
5d5818e44f style: enforce node: protocol for builtin imports (#10951)
Add n/prefer-node-protocol rule and autofix all bare builtin imports
to use the node: prefix. Simplify the simple-import-sort builtins
pattern to just ^node: since all imports now use the prefix.
2026-03-13 07:59:51 +01:00
Zoltan Kochan
1c8c4e49f5 style: add eslint-plugin-simple-import-sort (#10947)
Add eslint-plugin-simple-import-sort to enforce consistent import ordering:
- Node.js builtins first
- External packages second
- Relative imports last
- Named imports sorted alphabetically within each statement
2026-03-13 02:02:38 +01:00
Brandon Cheng
01914345d5 build: enable @typescript-eslint/no-import-type-side-effects (#10630)
* build: enable `@typescript-eslint/no-import-type-side-effects`

* build: disable `@typescript-eslint/consistent-type-imports`

* chore: apply fixes for `no-import-type-side-effects`

pnpm exec eslint "**/src/**/*.ts" "**/test/**/*.ts" --fix
2026-03-08 00:02:48 +01:00
Zoltan Kochan
b7f0f21582 feat: use SQLite for storing package index in the content-addressable store (#10827)
## Summary

Replace individual `.mpk` (MessagePack) files under `$STORE/index/` with a single SQLite database at `$STORE/index.db` using Node.js 22's built-in `node:sqlite` module. This reduces filesystem syscall overhead and improves space efficiency for small metadata entries.

Closes #10826

## Design

### New package: `@pnpm/store.index`

A new `StoreIndex` class wraps a SQLite database with a simple key-value API (`get`, `set`, `delete`, `has`, `entries`). Data is serialized with msgpackr and stored as BLOBs. The table uses `WITHOUT ROWID` for compact storage.

Key design decisions:

- **WAL mode** enables concurrent reads from workers while the main process writes.
- **`busy_timeout=5000`** plus a retry loop with `Atomics.wait`-based `sleepSync` handles `SQLITE_BUSY` errors from concurrent access.
- **Performance PRAGMAs**: `synchronous=NORMAL`, `mmap_size=512MB`, `cache_size=32MB`, `temp_store=MEMORY`, `wal_autocheckpoint=10000`.
- **Write batching**: `queueWrites()` batches pre-packed entries from tarball extraction and flushes them in a single transaction on `process.nextTick`. `setRawMany()` writes immediate batches (e.g. from `addFilesFromDir`).
- **Lifecycle**: `close()` auto-flushes pending writes, runs `PRAGMA optimize`, and closes the DB. A `process.on('exit')` handler ensures cleanup even on unexpected exits.
- **`VACUUM` after `deleteMany`** (used by `pnpm store prune`) to reclaim disk space.

### Key format

Keys are `integrity\tpkgId` (tab-separated). Git-hosted packages use `pkgId\tbuilt` or `pkgId\tnot-built`.

### Shared StoreIndex instance

A single `StoreIndex` instance is threaded through the entire install lifecycle — from `createNewStoreController` through the fetcher chain, package requester, license scanner, SBOM collector, and dependencies hierarchy. This replaces the previous pattern of each component creating its own file-based index access.

### Worker architecture

Index writes are performed in the main process, not in worker threads. Workers send pre-packed `{ key, buffer }` pairs back to the main process via `postMessage`, where they are batched and flushed to SQLite. This avoids SQLite write contention between threads.

### SQLite ExperimentalWarning suppression

`node:sqlite` emits an `ExperimentalWarning` on first load. This is suppressed via a `process.emitWarning` override injected through esbuild's `banner` option, which runs on line 1 of both `dist/pnpm.mjs` and `dist/worker.js` — before any module that loads `node:sqlite`.

### No migration from `.mpk` files

Old `.mpk` index files are not migrated. Packages missing from the new SQLite index are re-fetched on demand (the same behavior as a fresh store).

## Changed packages

121 files changed across these areas:

- **`store/index/`** — New `@pnpm/store.index` package
- **`worker/`** — Write batching moved from worker module into `StoreIndex` class; workers send pre-packed buffers to main process
- **`store/package-store/`** — StoreIndex creation and lifecycle management
- **`store/cafs/`** — Removed `getFilePathInCafs` index-file utilities (no longer needed)
- **`store/pkg-finder/`** — Reads from StoreIndex instead of `.mpk` files
- **`store/plugin-commands-store/`** — `store status` uses StoreIndex
- **`store/plugin-commands-store-inspecting/`** — `cat-index` and `find-hash` use StoreIndex
- **`fetching/tarball-fetcher/`** — Threads StoreIndex through fetchers; git-hosted fetcher flushes before reading
- **`fetching/git-fetcher/`, `binary-fetcher/`, `pick-fetcher/`** — Accept StoreIndex parameter
- **`pkg-manager/`** — `client`, `core`, `headless`, `package-requester` thread StoreIndex
- **`reviewing/`** — `license-scanner`, `sbom`, `dependencies-hierarchy` accept StoreIndex
- **`cache/api/`** — Cache view uses StoreIndex
- **`pnpm/bundle.ts`** — esbuild banner for ExperimentalWarning suppression

## Test plan

- [x] `pnpm --filter @pnpm/store.index test` — Unit tests for StoreIndex CRUD and batching
- [x] `pnpm --filter @pnpm/package-store test` — Store controller lifecycle
- [x] `pnpm --filter @pnpm/package-requester test` — Package requester reads from SQLite index
- [x] `pnpm --filter @pnpm/tarball-fetcher test` — Tarball and git-hosted fetcher writes
- [x] `pnpm --filter @pnpm/headless test` — Headless install
- [x] `pnpm --filter @pnpm/core test` — Core install, side effects, patching
- [x] `pnpm --filter @pnpm/plugin-commands-rebuild test` — Rebuild reads from index
- [x] `pnpm --filter @pnpm/license-scanner test` — License scanning
- [x] e2e tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-06 12:59:04 +01:00
Zoltan Kochan
e18a879d72 feat!: drop Node.js 22.12 support 2026-02-18 14:54:09 +01:00
Zoltan Kochan
1b4df57a01 feat!: drop Node.js 20 and 21 support (#10569) 2026-02-08 19:16:24 +01:00
Zoltan Kochan
a00f9e515c chore: use typescript-go (#10452) 2026-01-14 01:18:13 +01:00
Zoltan Kochan
c5d4d81f56 refactor: rename FilesIndex to FilesMap (#10427) 2026-01-08 18:17:57 +01:00
Brandon Cheng
db72923b5c fix: forward $NODE_OPTIONS when running jest for debug terminals (#10364)
* fix: forward existing `$NODE_OPTIONS` when running jest

* chore: update `package.json` files for meta-updater changes

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-12-27 22:10:52 +01:00
Zoltan Kochan
01760da877 fix(git-resolver): installing git-hosted dependency using annotated tags (#10349)
close #10335
2025-12-22 23:05:40 +01:00
Zoltan Kochan
0048667db4 refactor: use Maps instead of Records (#10312) 2025-12-15 11:48:19 +01:00
Trevor Burnham
38b8e357b5 feat: add custom resolvers and fetchers (#10246) 2025-11-30 14:19:04 +01:00
Zoltan Kochan
7e2910e70f chore(release): 11.0.0-alpha.0 2025-11-13 15:44:27 +01:00
Zoltan Kochan
dab9abef5c Merge remote-tracking branch 'origin/main' into v11 2025-10-24 14:19:07 +02:00
Zoltan Kochan
0cde1287c8 chore: update repository fields 2025-10-23 11:57:12 +02:00
Zoltan Kochan
491a84fb26 feat: use ESM instead of commonjs (#9870) 2025-08-25 10:02:00 +02:00
Zoltan Kochan
27cbc09206 style: fix jest-related linting issues (#9894) 2025-08-22 21:56:49 +02:00
Zoltan Kochan
7d2fd48215 feat!: drop Node.js 18 support (#9858) 2025-08-14 14:06:03 +02:00
Zoltan Kochan
a4d654807c chore(release): 10.14.0 2025-07-31 15:00:26 +02:00
Zoltan Kochan
d1edf732ad feat: support installing Deno runtime (#9791)
* feat: support installing Deno runtime

* refactor: use npm registry to resolve deno version

* feat: wip

* feat: installing deno runtime

* style: fix

* test: fix

* test: deno

* test: fix

* feat: deno

* feat: deno

* feat: create zip fetcher

* style: fix

* refactor: node fetch

* feat: support a new binary fetcher

* test: fix

* feat: rename zip-fetcher to binary-fetcher

* refactor: change naming

* fix: windows

* refactor: rename packages

* refactor: deno resolver

* refactor: runtime resolvers

* refactor: binary fetcher

* refactor: runtime resolvers

* refactor: runtime resolvers

* refactor: create SingleResolution

* refactor: remove not needed change

* refactor: package requester

* docs: add changesets

* refactor: use VariationsResolution and AtomicResolution

* refactor: implement CR suggestions

* docs: add changesets

* fix: address comment in CR

* feat: update formatting of pnpm-lock.yaml
2025-07-30 11:27:07 +02:00
Zoltan Kochan
fb9de7ac3a chore(release): 10.14.0-0 2025-07-23 14:54:13 +02:00
btea
4a8243a00d chore: package.json add type field (#9765)
* chore: `package.json` add type field

* chore: add type field to every package.json

* chore: add type field to every package.json

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2025-07-20 03:21:46 +02:00
Zoltan Kochan
1a07b8fc7c feat: install Node.js runtime as a dependency (#9755) 2025-07-19 15:11:40 +02:00
Zoltan Kochan
89bc2c6637 chore(release): 10.12.0 2025-06-08 15:04:58 +02:00
Khải
6acf819c69 refactor!: stricter Resolution type (#9582)
* refactor!: stricter `Resolution` type

* docs(changeset): don't leave it out

* fix: test

* fix: test

* refactor: replace `as` with `!`
2025-05-30 09:17:12 +02:00
Zoltan Kochan
5e78da2d57 style: order the keys in package.json files 2025-03-01 21:45:13 +01:00
Zoltan Kochan
3fe70e7897 refactor!: all lib versions should start with pnpm major version 2024-11-29 01:04:45 +01:00
Zoltan Kochan
26a910cbf8 chore(release): 10.0.0-alpha.0 2024-10-28 04:16:49 +01:00
Zoltan Kochan
1dd419cbb4 test: fix killing the registry mock server on teardown (#8695) 2024-10-25 21:24:08 +02:00
Khải
9b4f73caaf chore(scripts): typecheck-only (#8395)
* chore(scripts): typecheck-only

* feat: change all configuration

* feat: include pnpm/ and pnpm/test/

* chore(deps): remove unused dependency

* refactor(typescript-only): use find-packages

* refactor(typescript-only): refactor paths

* fix: typescript-only

* fix: update compile-only

* fix: compile pnpm

* fix: windows

* fix: windows

* chore: meta-updater

* refactor(tsconfig): remove explicit composite

* fix: path in windows

* feat: don't depend on cwd

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2024-08-11 08:26:01 +02:00
Brandon Cheng
c7e1b6fae8 chore: configure TypeScript project references for tests (#8128)
* refactor: store link values before converting to references

* fix: use .sort() without localeCompare

https://github.com/pnpm/pnpm/pull/8128#discussion_r1614031566

> Nit, but you probably just want to call sort without a comparison
> function; these are already strings and locale compare is not a good
> comparison for anything but human readable strings since it will
> differ on different people's machines based on their language setting.
> I've hit this too many times before for code gen.

* feat: configure meta-updater to write test/tsconfig.json files

* fix: relative imports for __typings__

* chore: `pnpm run meta-updater`

* fix: explicitly use test/tsconfig.json for ts-jest
2024-05-31 12:48:13 +02:00
Brandon Cheng
5de89d81a6 chore: remove top-level composite property from tsconfig.json files (#8130)
* chore: remove no-op composite property

* chore: remove top-level composite property from existing files
2024-05-28 02:37:40 +02:00
Khải
c17263c62a refactor(fetching): return type annotations (#7903) 2024-04-12 00:32:48 +02:00
Zoltan Kochan
43bd75c5b8 chore(release): libs 2024-04-09 17:14:05 +02:00
Zoltan Kochan
6e3b75a42f chore(release): 9.0.0-alpha.0 2024-01-08 19:12:45 +01:00
Zoltan Kochan
43cdd87c0c feat!: drop Node.js 16 support 2024-01-08 11:57:44 +01:00
Zoltan Kochan
73ebfc94e0 chore(release): 8.8.0 2023-09-26 15:24:32 +03:00
await-ovo
f394cfccda fix(resolve-dependencies): not update git protocol dependency when add unrelated dependency (#7054)
close #7008
2023-09-23 14:11:48 +03:00
Zoltan Kochan
eb5b7d5299 chore(release): 8.0.0-rc.0 2023-03-24 15:52:20 +02:00
Zoltan Kochan
eceaa8b8b7 feat!: drop Node.js 14 support 2023-03-18 17:11:38 +02:00
Zoltan Kochan
a4ce8e1f27 Merge branch 'main' into v8 2023-02-23 13:13:30 +02:00
await-ovo
77c754b10a chore: eslint issues (#6123) 2023-02-23 12:12:32 +02:00
Zoltan Kochan
0058ecfbb9 chore(release): 8.0.0-alpha.0 2023-02-20 23:19:20 +02:00