Commit Graph

6 Commits

Author SHA1 Message Date
Zoltan Kochan
60fd20536d fix: pin integrity of git-hosted tarballs in lockfile (#11481)
For git-hosted tarballs (`codeload.github.com` / `gitlab.com` / `bitbucket.org`) the fetcher dropped the integrity it computed while downloading, so the lockfile only ever stored the URL. A compromised git host or man-in-the-middle could serve a substituted tarball on subsequent installs and pnpm would install it — the lockfile had no hash to compare against.

This pins the SHA-512 SRI of the raw tarball in the lockfile, in the same `sha512-<base64>` form npm-registry tarballs use. The only difference is the source: for npm we pass through `dist.integrity`, for git we compute it locally from the downloaded buffer. Subsequent installs validate the download against that integrity in the worker (`addTarballToStore` → `parseIntegrity` → hash compare), so a tampered tarball fails with `TarballIntegrityError`.

## Why git-hosted stays on `gitHostedStoreIndexKey`

The lockfile pins integrity for security, but the *store key* for git-hosted resolutions stays on `gitHostedStoreIndexKey(pkgId, { built })` rather than collapsing under the integrity-based key. Reason: git-hosted tarballs are post-processed (`preparePackage` / `packlist`), so the cached file set depends on whether build scripts ran during fetch. The integrity-only key would fold the built and not-built variants into a single slot, letting one overwrite the other and serving the wrong content if `ignoreScripts` was toggled between runs. Keeping git-hosted on the existing key shape preserves that dimension; the integrity is still validated on every fresh download.

## How the routing stays clean

The naive way to express "use gitHostedStoreIndexKey for git-hosted, integrity key for npm" is to call `isGitHostedPkgUrl(resolution.tarball)` everywhere a store key is computed — fragile, scattered, and easy to forget when adding new readers (Copilot caught two of those during review). Instead, a typed annotation: `TarballResolution` gets an optional `gitHosted: boolean` field. The git resolver sets it; the lockfile loader (`convertToLockfileObject`) backfills it for entries written by older pnpm versions; `toLockfileResolution` carries it through on serialize. Every consumer reads `resolution.gitHosted` directly. URL detection lives in exactly two places — the resolver and the loader — instead of seven.

## Changes

### Security fix
- `fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts` — return the `integrity` that the inner remote-tarball fetch already computed (was being silently dropped by the destructure).

### Lockfile schema (additive)
- `@pnpm/lockfile.types` and `@pnpm/resolving.resolver-base` — `TarballResolution` gains optional `gitHosted: boolean`.
- `@pnpm/resolving.git-resolver` — sets `gitHosted: true` on every git-hosted tarball it produces.
- `@pnpm/lockfile.fs` (`convertToLockfileObject`) — backfills the field on load for older lockfiles via inlined URL detection.
- `@pnpm/lockfile.utils` (`toLockfileResolution`, `pkgSnapshotToResolution`) — preserve / read the field.

### Store-key consumers (now one-line typed reads, dropped the URL-sniffing dep)
- `installing/package-requester` (`getFilesIndexFilePath`)
- `store/pkg-finder` (`readPackageFileMap`)
- `modules-mounter/daemon` (`createFuseHandlers`)
- `building/after-install` (side-effects-cache lookup + write)
- `store/commands/storeStatus`
- `installing/deps-installer` (agent-mode store-controller wrapper)

### Fetcher routing
- `fetching/pick-fetcher` — `pickFetcher` prefers `resolution.gitHosted`; URL fallback retained for ad-hoc resolutions.

### Tests
- New integrity-validation test in `tarball-fetcher` (mismatched `integrity` on the resolution must throw `TarballIntegrityError`).
- New git-hosted lookup test in `pkg-finder` asserting routing through `gitHostedStoreIndexKey` even when integrity is present.
- New `toLockfileResolution` test asserting `gitHosted: true` flows through serialization.
- `fromRepo.ts` lockfile snapshot updated for the now-pinned integrity + `gitHosted: true`.
- `git-resolver` tests updated to assert `gitHosted: true` in produced resolutions.
2026-05-06 13:22:25 +02:00
Zoltan Kochan
dba4153767 refactor: rename packages and consolidate runtime resolvers (#10999)
* refactor: rename workspace.sort-packages and workspace.pkgs-graph

- workspace.sort-packages -> workspace.projects-sorter
- workspace.pkgs-graph -> workspace.projects-graph

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rename packages/ to core/ and pkg-manifest.read-package-json to reader

- Rename packages/ directory to core/ for clarity
- Rename pkg-manifest/read-package-json to pkg-manifest/reader (@pnpm/pkg-manifest.reader)
- Update all tsconfig, package.json, and lockfile references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: consolidate runtime resolvers under engine/runtime domain

- Remove unused @pnpm/engine.runtime.node.fetcher package
- Rename engine/runtime/node.resolver to node-resolver (dash convention)
- Move resolving/bun-resolver to engine/runtime/bun-resolver
- Move resolving/deno-resolver to engine/runtime/deno-resolver
- Update all package names, tsconfig paths, and lockfile references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update lockfile after removing node.fetcher

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sort tsconfig references and package.json deps alphabetically

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: auto-fix import sorting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update __typings__ paths in tsconfig.lint.json for moved resolvers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove deno-resolver from deps of bun-resolver

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 00:19:58 +01:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
f47ef4b125 refactor: reorganize monorepo domain structure (#10987)
Reorganize the monorepo's top-level domain directories for clarity:

- pkg-manager/ split into:
  - installing/ (core, headless, client, resolve-dependencies, etc.)
  - installing/linking/ (hoist, direct-dep-linker, modules-cleaner, etc.)
  - bins/ (link-bins, package-bins, remove-bins)
- completion/ merged into cli/
- dedupe/ moved to installing/dedupe/
- env/ renamed to engine/ with subdomains:
  - engine/runtime/ (node.fetcher, node.resolver, plugin-commands-env, etc.)
  - engine/pm/ (plugin-commands-setup, plugin-commands-self-updater)
- env.path moved to shell/
- tools/ and runtime/ dissolved
- reviewing/ and lockfile audit packages moved to deps/:
  - deps/inspection/ (list, outdated, dependencies-hierarchy)
  - deps/compliance/ (audit, licenses, sbom)
- registry/ moved to resolving/registry/
- semver/peer-range moved to deps/
- network/fetching-types moved to fetching/
- packages/ slimmed down, moving packages to proper domains:
  - calc-dep-state, dependency-path -> deps/
  - parse-wanted-dependency -> resolving/
  - git-utils -> network/
  - naming-cases -> text/
  - make-dedicated-lockfile -> lockfile/
  - render-peer-issues -> installing/
  - plugin-commands-doctor -> cli/
  - plugin-commands-init -> workspace/

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 13:45:54 +01:00
Trevor Burnham
38b8e357b5 feat: add custom resolvers and fetchers (#10246) 2025-11-30 14:19:04 +01:00
Zoltan Kochan
c5ef9b0643 refactor: rename filter-utils to lockfile.utils 2024-07-28 01:35:40 +02:00