Commit Graph

32 Commits

Author SHA1 Message Date
Zoltan Kochan
0d88df854f chore: update all dependencies to latest versions (#11032)
* chore: update all dependencies to latest versions

Update all outdated dependencies across the monorepo catalog and fix
breaking changes from major version bumps.

Notable updates:
- ESLint 9 → 10 (fix custom rule API, disable new no-useless-assignment)
- @stylistic/eslint-plugin 4 → 5 (auto-fixed indent changes)
- @cyclonedx/cyclonedx-library 9 → 10 (adapt to removed SPDX API)
- esbuild 0.25 → 0.27
- TypeScript 5.9.2 → 5.9.3
- Various @types packages, test utilities, and build tools

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update unified/remark/mdast imports for v11/v4 API changes

Update imports in get-release-text for the new ESM named exports:
- mdast-util-to-string: default → { toString }
- unified: default → { unified }

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve typecheck errors from dependency updates

- isexe v4: use named import { sync } instead of default export
- remark-parse/remark-stringify v11: add vfile as packageExtension
  dependency so TypeScript can resolve type declarations
- get-release-text: remove unused @ts-expect-error directives

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert runtime dependency major version bumps

Revert major version bumps for runtime dependencies that are bundled
into pnpm to fix test failures where pnpm add silently fails:
- bin-links: keep ^5.0.0 (was ^6.0.0)
- cli-truncate: keep ^4.0.0 (was ^5.2.0)
- delay: keep ^6.0.0 (was ^7.0.0)
- filenamify: keep ^6.0.0 (was ^7.0.1)
- find-up: keep ^7.0.0 (was ^8.0.0)
- isexe: keep 2.0.0 (was 4.0.0)
- normalize-newline: keep 4.1.0 (was 5.0.0)
- p-queue: keep ^8.1.0 (was ^9.1.0)
- ps-list: keep ^8.1.1 (was ^9.0.0)
- string-length: keep ^6.0.0 (was ^7.0.1)
- symlink-dir: keep ^7.0.0 (was ^9.0.0)
- terminal-link: keep ^4.0.0 (was ^5.0.0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: restore runtime dependency major version bumps

Re-apply all runtime dependency major version bumps that were
previously reverted. All packages maintain their default exports
except isexe v4 which needs named imports.

Updated runtime deps:
- bin-links: ^5.0.0 → ^6.0.0
- cli-truncate: ^4.0.0 → ^5.2.0
- delay: ^6.0.0 → ^7.0.0
- filenamify: ^6.0.0 → ^7.0.1
- find-up: ^7.0.0 → ^8.0.0
- isexe: 2.0.0 → 4.0.0 (fix: use named import { sync })
- normalize-newline: 4.1.0 → 5.0.0
- p-queue: ^8.1.0 → ^9.1.0
- ps-list: ^8.1.1 → ^9.0.0
- string-length: ^6.0.0 → ^7.0.1
- symlink-dir: ^7.0.0 → ^9.0.0
- terminal-link: ^4.0.0 → ^5.0.0

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert tempy to 3.0.0 to fix bundle hang

tempy 3.2.0 pulls in temp-dir 3.0.0 which uses async fs.realpath()
inside its module init. When bundled by esbuild into the __esm lazy
init pattern, this causes a deadlock during module initialization,
making the pnpm binary hang silently on startup.

Keeping tempy at 3.0.0 which uses temp-dir 2.x (sync fs.realpathSync).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add comment explaining why tempy cannot be upgraded

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert nock to 13.3.4 for node-fetch compatibility

nock 14 changed its HTTP interception mechanism in a way that doesn't
properly intercept node-fetch requests, causing audit tests to hang
waiting for responses that are never intercepted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add comment explaining why nock cannot be upgraded

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update symlink-dir imports for v10 ESM named exports

symlink-dir v10 removed the default export and switched to named
exports: { symlinkDir, symlinkDirSync }.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert @typescript/native-preview to working version

Newer tsgo dev builds (>= 20260318) have a regression where
@types/node cannot be resolved, breaking all node built-in types.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: vulnerabilities

* fix: align comment indentation in runLifecycleHook

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: pin msgpackr to 1.11.8 for TypeScript 5.9 compatibility

msgpackr 1.11.9 has broken type definitions that use Iterable/Iterator
without required type arguments, causing compile errors with TS 5.9.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 23:28:53 +01:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
5d5818e44f style: enforce node: protocol for builtin imports (#10951)
Add n/prefer-node-protocol rule and autofix all bare builtin imports
to use the node: prefix. Simplify the simple-import-sort builtins
pattern to just ^node: since all imports now use the prefix.
2026-03-13 07:59:51 +01:00
Zoltan Kochan
1c8c4e49f5 style: add eslint-plugin-simple-import-sort (#10947)
Add eslint-plugin-simple-import-sort to enforce consistent import ordering:
- Node.js builtins first
- External packages second
- Relative imports last
- Named imports sorted alphabetically within each statement
2026-03-13 02:02:38 +01:00
Brandon Cheng
01914345d5 build: enable @typescript-eslint/no-import-type-side-effects (#10630)
* build: enable `@typescript-eslint/no-import-type-side-effects`

* build: disable `@typescript-eslint/consistent-type-imports`

* chore: apply fixes for `no-import-type-side-effects`

pnpm exec eslint "**/src/**/*.ts" "**/test/**/*.ts" --fix
2026-03-08 00:02:48 +01:00
Zoltan Kochan
b7f0f21582 feat: use SQLite for storing package index in the content-addressable store (#10827)
## Summary

Replace individual `.mpk` (MessagePack) files under `$STORE/index/` with a single SQLite database at `$STORE/index.db` using Node.js 22's built-in `node:sqlite` module. This reduces filesystem syscall overhead and improves space efficiency for small metadata entries.

Closes #10826

## Design

### New package: `@pnpm/store.index`

A new `StoreIndex` class wraps a SQLite database with a simple key-value API (`get`, `set`, `delete`, `has`, `entries`). Data is serialized with msgpackr and stored as BLOBs. The table uses `WITHOUT ROWID` for compact storage.

Key design decisions:

- **WAL mode** enables concurrent reads from workers while the main process writes.
- **`busy_timeout=5000`** plus a retry loop with `Atomics.wait`-based `sleepSync` handles `SQLITE_BUSY` errors from concurrent access.
- **Performance PRAGMAs**: `synchronous=NORMAL`, `mmap_size=512MB`, `cache_size=32MB`, `temp_store=MEMORY`, `wal_autocheckpoint=10000`.
- **Write batching**: `queueWrites()` batches pre-packed entries from tarball extraction and flushes them in a single transaction on `process.nextTick`. `setRawMany()` writes immediate batches (e.g. from `addFilesFromDir`).
- **Lifecycle**: `close()` auto-flushes pending writes, runs `PRAGMA optimize`, and closes the DB. A `process.on('exit')` handler ensures cleanup even on unexpected exits.
- **`VACUUM` after `deleteMany`** (used by `pnpm store prune`) to reclaim disk space.

### Key format

Keys are `integrity\tpkgId` (tab-separated). Git-hosted packages use `pkgId\tbuilt` or `pkgId\tnot-built`.

### Shared StoreIndex instance

A single `StoreIndex` instance is threaded through the entire install lifecycle — from `createNewStoreController` through the fetcher chain, package requester, license scanner, SBOM collector, and dependencies hierarchy. This replaces the previous pattern of each component creating its own file-based index access.

### Worker architecture

Index writes are performed in the main process, not in worker threads. Workers send pre-packed `{ key, buffer }` pairs back to the main process via `postMessage`, where they are batched and flushed to SQLite. This avoids SQLite write contention between threads.

### SQLite ExperimentalWarning suppression

`node:sqlite` emits an `ExperimentalWarning` on first load. This is suppressed via a `process.emitWarning` override injected through esbuild's `banner` option, which runs on line 1 of both `dist/pnpm.mjs` and `dist/worker.js` — before any module that loads `node:sqlite`.

### No migration from `.mpk` files

Old `.mpk` index files are not migrated. Packages missing from the new SQLite index are re-fetched on demand (the same behavior as a fresh store).

## Changed packages

121 files changed across these areas:

- **`store/index/`** — New `@pnpm/store.index` package
- **`worker/`** — Write batching moved from worker module into `StoreIndex` class; workers send pre-packed buffers to main process
- **`store/package-store/`** — StoreIndex creation and lifecycle management
- **`store/cafs/`** — Removed `getFilePathInCafs` index-file utilities (no longer needed)
- **`store/pkg-finder/`** — Reads from StoreIndex instead of `.mpk` files
- **`store/plugin-commands-store/`** — `store status` uses StoreIndex
- **`store/plugin-commands-store-inspecting/`** — `cat-index` and `find-hash` use StoreIndex
- **`fetching/tarball-fetcher/`** — Threads StoreIndex through fetchers; git-hosted fetcher flushes before reading
- **`fetching/git-fetcher/`, `binary-fetcher/`, `pick-fetcher/`** — Accept StoreIndex parameter
- **`pkg-manager/`** — `client`, `core`, `headless`, `package-requester` thread StoreIndex
- **`reviewing/`** — `license-scanner`, `sbom`, `dependencies-hierarchy` accept StoreIndex
- **`cache/api/`** — Cache view uses StoreIndex
- **`pnpm/bundle.ts`** — esbuild banner for ExperimentalWarning suppression

## Test plan

- [x] `pnpm --filter @pnpm/store.index test` — Unit tests for StoreIndex CRUD and batching
- [x] `pnpm --filter @pnpm/package-store test` — Store controller lifecycle
- [x] `pnpm --filter @pnpm/package-requester test` — Package requester reads from SQLite index
- [x] `pnpm --filter @pnpm/tarball-fetcher test` — Tarball and git-hosted fetcher writes
- [x] `pnpm --filter @pnpm/headless test` — Headless install
- [x] `pnpm --filter @pnpm/core test` — Core install, side effects, patching
- [x] `pnpm --filter @pnpm/plugin-commands-rebuild test` — Rebuild reads from index
- [x] `pnpm --filter @pnpm/license-scanner test` — License scanning
- [x] e2e tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-06 12:59:04 +01:00
Zoltan Kochan
e2e0a321b3 perf: optimize how the integrities of files in the CAFS are stored (#10504) 2026-01-24 21:41:11 +01:00
Zoltan Kochan
40b107efa7 perf: migrate internal cache and index files to MessagePack serialization (#10500) 2026-01-23 01:31:09 +01:00
Zoltan Kochan
e3b35b6f37 style: update eslint to v9 (#10474) 2026-01-17 12:01:23 +01:00
Zoltan Kochan
da112f7cb2 revert: "perf: use v8 serialize/deserialize instead of JSON (#9971)" (#10420)
close #10409
2026-01-13 15:16:33 +01:00
Zoltan Kochan
0048667db4 refactor: use Maps instead of Records (#10312) 2025-12-15 11:48:19 +01:00
Zoltan Kochan
9b344c8982 perf: use v8 serialize/deserialize instead of JSON (#9971)
close #9965
2025-11-06 01:01:06 +01:00
Zoltan Kochan
3df6702bcb fix: update load-json-file, write-json-file, write-pkg 2025-08-28 11:37:38 +02:00
Zoltan Kochan
491a84fb26 feat: use ESM instead of commonjs (#9870) 2025-08-25 10:02:00 +02:00
Zoltan Kochan
facd7656e8 refactor: always use extensions in relative imports (#9878) 2025-08-19 15:25:11 +02:00
Zoltan Kochan
a76da0c53c feat!: remove conversion of lockfile v6 to v9 (#8866) 2024-12-15 14:39:19 +01:00
Zoltan Kochan
298e5dcafd feat!: create a separate directory in the store for index files (#8700) 2024-10-27 21:00:28 +01:00
Zoltan Kochan
d433cb9c9f feat: adding the package name into the index file name (#8510)
close #8204
2024-09-15 18:26:28 +02:00
Zoltan Kochan
db420ab592 refactor(cafs): create a separate function for creating index file path (#8508) 2024-09-08 14:43:09 +02:00
Zoltan Kochan
c5ef9b0643 refactor: rename filter-utils to lockfile.utils 2024-07-28 01:35:40 +02:00
Zoltan Kochan
e9908b3185 refactor: rename lockfile-file to lockfile.fs 2024-07-28 01:15:20 +02:00
Zoltan Kochan
3dd30b1909 refactor: create branded type for project ID (#8164) 2024-06-04 18:30:59 +02:00
Zoltan Kochan
c649954e41 refactor: create a separate type for DepPath (#8094) 2024-05-20 16:11:46 +02:00
Zoltan Kochan
9719a42d0e feat: configurable max length of directories inside node_modules/.pnpm (#7994)
close #7355
2024-04-28 11:07:29 +02:00
Khải
2786b549d9 chore(eslint): explicit-module-boundary-types (#7926) 2024-04-14 13:37:50 +02:00
Khải
757ae8200d refactor(modules-mounter+network): return types (#7894) 2024-04-11 00:59:59 +02:00
Nacho Aldama
7733f3a943 feat: support scoped client certificates (#7626)
close #7427

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2024-02-14 12:11:12 +01:00
Zoltan Kochan
733d612ad1 test: check spelling with cspell (#7229) 2023-10-20 15:58:27 +03:00
Zoltan Kochan
5ce333519d refactor: rename cafs to store.cafs 2023-07-19 01:53:42 +03:00
Zoltan Kochan
bcbfd42c44 fix(mound-modules): read the file mode from the CAFS 2023-05-18 15:09:10 +03:00
Zoltan Kochan
cdcb5d309c fix(mound-modules): read 2023-05-17 15:35:12 +03:00
Zoltan Kochan
b3b5320f66 chore: move mount-modules 2023-05-14 17:26:00 +03:00