33 Commits

Author SHA1 Message Date
Zoltan Kochan
39e42665b3 fix(git-resolver): avoid encoded slash in GitLab tarball URL (#11551)
* fix(git-resolver): avoid encoded slash in GitLab tarball URL

hosted-git-info's default GitLab tarball URL routes through
`/api/v4/projects/<user>%2F<project>/...`. The `%2F` survives into the
virtual store directory name (depPathToFilename only escapes raw `/`,
not `%`), and Node refuses to import any module whose path contains an
encoded slash. The same URL is also intermittently rejected by GitLab
with a 406.

Override the GitLab tarballtemplate to the `/-/archive/` URL, which works
for both public and private repos and contains no encoded slashes.

Closes #11533

* test: avoid cspell-flagged words

* test: keep existing gitlab assertions, only add new ones

Restore the skipped tests' original API-URL assertions; they document the
old expected shape and weren't running anyway. Add the new `/-/archive/`
URL to the pick-fetcher fixture as an additional case so both shapes are
exercised.
2026-05-08 22:29:27 +02:00
Zoltan Kochan
60fd20536d fix: pin integrity of git-hosted tarballs in lockfile (#11481)
For git-hosted tarballs (`codeload.github.com` / `gitlab.com` / `bitbucket.org`) the fetcher dropped the integrity it computed while downloading, so the lockfile only ever stored the URL. A compromised git host or man-in-the-middle could serve a substituted tarball on subsequent installs and pnpm would install it — the lockfile had no hash to compare against.

This pins the SHA-512 SRI of the raw tarball in the lockfile, in the same `sha512-<base64>` form npm-registry tarballs use. The only difference is the source: for npm we pass through `dist.integrity`, for git we compute it locally from the downloaded buffer. Subsequent installs validate the download against that integrity in the worker (`addTarballToStore` → `parseIntegrity` → hash compare), so a tampered tarball fails with `TarballIntegrityError`.

## Why git-hosted stays on `gitHostedStoreIndexKey`

The lockfile pins integrity for security, but the *store key* for git-hosted resolutions stays on `gitHostedStoreIndexKey(pkgId, { built })` rather than collapsing under the integrity-based key. Reason: git-hosted tarballs are post-processed (`preparePackage` / `packlist`), so the cached file set depends on whether build scripts ran during fetch. The integrity-only key would fold the built and not-built variants into a single slot, letting one overwrite the other and serving the wrong content if `ignoreScripts` was toggled between runs. Keeping git-hosted on the existing key shape preserves that dimension; the integrity is still validated on every fresh download.

## How the routing stays clean

The naive way to express "use gitHostedStoreIndexKey for git-hosted, integrity key for npm" is to call `isGitHostedPkgUrl(resolution.tarball)` everywhere a store key is computed — fragile, scattered, and easy to forget when adding new readers (Copilot caught two of those during review). Instead, a typed annotation: `TarballResolution` gets an optional `gitHosted: boolean` field. The git resolver sets it; the lockfile loader (`convertToLockfileObject`) backfills it for entries written by older pnpm versions; `toLockfileResolution` carries it through on serialize. Every consumer reads `resolution.gitHosted` directly. URL detection lives in exactly two places — the resolver and the loader — instead of seven.

## Changes

### Security fix
- `fetching/tarball-fetcher/src/gitHostedTarballFetcher.ts` — return the `integrity` that the inner remote-tarball fetch already computed (was being silently dropped by the destructure).

### Lockfile schema (additive)
- `@pnpm/lockfile.types` and `@pnpm/resolving.resolver-base` — `TarballResolution` gains optional `gitHosted: boolean`.
- `@pnpm/resolving.git-resolver` — sets `gitHosted: true` on every git-hosted tarball it produces.
- `@pnpm/lockfile.fs` (`convertToLockfileObject`) — backfills the field on load for older lockfiles via inlined URL detection.
- `@pnpm/lockfile.utils` (`toLockfileResolution`, `pkgSnapshotToResolution`) — preserve / read the field.

### Store-key consumers (now one-line typed reads, dropped the URL-sniffing dep)
- `installing/package-requester` (`getFilesIndexFilePath`)
- `store/pkg-finder` (`readPackageFileMap`)
- `modules-mounter/daemon` (`createFuseHandlers`)
- `building/after-install` (side-effects-cache lookup + write)
- `store/commands/storeStatus`
- `installing/deps-installer` (agent-mode store-controller wrapper)

### Fetcher routing
- `fetching/pick-fetcher` — `pickFetcher` prefers `resolution.gitHosted`; URL fallback retained for ad-hoc resolutions.

### Tests
- New integrity-validation test in `tarball-fetcher` (mismatched `integrity` on the resolution must throw `TarballIntegrityError`).
- New git-hosted lookup test in `pkg-finder` asserting routing through `gitHostedStoreIndexKey` even when integrity is present.
- New `toLockfileResolution` test asserting `gitHosted: true` flows through serialization.
- `fromRepo.ts` lockfile snapshot updated for the now-pinned integrity + `gitHosted: true`.
- `git-resolver` tests updated to assert `gitHosted: true` in produced resolutions.
2026-05-06 13:22:25 +02:00
Zoltan Kochan
187049055f chore: upgrade @typescript/native-preview to 7.0.0-dev.20260421.2 (#11332)
* chore: upgrade @typescript/native-preview to 7.0.0-dev.20260421.2

- Add explicit `types: ["node"]` to the shared tsconfig because tsgo
  20260421 no longer auto-acquires `@types/*` from `node_modules`.
- Refactor test files to explicitly import jest globals (`describe`,
  `it`, `test`, `expect`, `beforeEach`, etc.) from `@jest/globals`
  instead of relying on `@types/jest` ambient declarations. Under the
  new tsgo build, `import { jest } from '@jest/globals'` shadows the
  ambient `jest` namespace, breaking `@types/jest`'s `declare var
  describe: jest.Describe;` globals.
- Add `@jest/globals` to each package's devDependencies where tests
  now import from it, and add `@types/node` to packages that need it
  but were relying on hoisted resolution.
- Replace `fail()` calls with `throw new Error(...)` since `fail` is
  no longer globally available.

* chore: fix remaining tsgo type-strictness errors

- Strip `as <PnpmType>` casts on objects passed to toMatchObject /
  toStrictEqual / toEqual; @jest/globals rejects the typed objects
  (which include AsymmetricMatchers) vs. the repo-specific type.
- Type `jest.fn<...>()` explicitly where the mock's signature matters
  for toHaveBeenCalledWith.
- Replace `beforeEach(() => X)` with `beforeEach(() => { X })` so the
  return value is void, as the stricter jest typing requires.
- Use `expect.objectContaining({...})` in one place where the full
  expected object triggered stricter type resolution.
- Cast `prompt.mock.calls` arg through `as unknown as Record<...>[]`
  for patch.test.ts's nested-array matchers.
- Fix off-by-one `<reference path>` in pnpm/test/getConfig.test.ts
  that only surfaced now.
- Move `@jest/globals` from devDependencies to dependencies in the
  two `__utils__` packages that import it from `src/`.
- Clean up unused imports from the @jest/globals migration.

* chore: address Copilot review on #11332

- Move misplaced `@jest/globals` imports to the top import block in
  checkEngine, run.ts, and workspace/root-finder tests where the
  script dropped them below executable code.
- Replace `try { await x(); throw new Error('should have thrown') } catch`
  in bins/linker, lockfile/fs, and resolving/local-resolver tests with
  `await expect(x()).rejects.toMatchObject({...})`. The old pattern
  swallowed an unrelated `throw` if the under-test call silently
  succeeded, which would fail on the catch-block assertion with a
  misleading message.
2026-04-21 22:50:40 +02:00
Zoltan Kochan
f80de69e84 test(git-resolver): mock network calls in subfolder tests for reliability
The subfolder resolution tests were hitting real GitHub APIs and git
ls-remote, causing flaky failures when HTTP HEAD checks returned non-ok
responses (rate limiting, network issues), which made the resolver take
the private repo path instead of the tarball path.
2026-04-03 15:14:56 +02:00
Zoltan Kochan
6c480a4375 perf: replace node-fetch with undici (#10537)
Replace node-fetch with native undici for HTTP requests throughout pnpm.

Key changes:
- Replace node-fetch with undici's fetch() and dispatcher system
- Replace @pnpm/network.agent with a new dispatcher module in @pnpm/network.fetch
- Cache dispatchers via LRU cache keyed by connection parameters
- Handle proxies via undici ProxyAgent instead of http/https-proxy-agent
- Convert test mocking from nock to undici MockAgent where applicable
- Add minimatch@9 override to fix ESM incompatibility with brace-expansion
2026-03-29 12:44:00 +02:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
5d5818e44f style: enforce node: protocol for builtin imports (#10951)
Add n/prefer-node-protocol rule and autofix all bare builtin imports
to use the node: prefix. Simplify the simple-import-sort builtins
pattern to just ^node: since all imports now use the prefix.
2026-03-13 07:59:51 +01:00
Zoltan Kochan
1c8c4e49f5 style: add eslint-plugin-simple-import-sort (#10947)
Add eslint-plugin-simple-import-sort to enforce consistent import ordering:
- Node.js builtins first
- External packages second
- Relative imports last
- Named imports sorted alphabetically within each statement
2026-03-13 02:02:38 +01:00
Zoltan Kochan
3a5bfaa94f chore: update zkochan packages to latest versions (#10930)
Update all packages from zkochan/packages to their latest major versions
and exclude them from minimumReleaseAge requirement. This includes
updating catalog entries, adapting to breaking API changes (default
exports replaced with named exports, sync functions renamed with Sync
suffix), and updating type declarations.
2026-03-11 13:47:46 +01:00
Zoltan Kochan
ec7c5d7d1a feat: improve git URL detection to recognize plain HTTP/HTTPS URLs
Improve git URL detection to recognize plain HTTP/HTTPS URLs
ending in `.git` and prioritize git resolver over tarball resolver.

close #10468
2026-01-16 19:38:02 +01:00
Zoltan Kochan
0bcbaf9994 refactor: move out skip resolution logic from package requester (#10439) 2026-01-12 13:08:50 +01:00
Zoltan Kochan
01760da877 fix(git-resolver): installing git-hosted dependency using annotated tags (#10349)
close #10335
2025-12-22 23:05:40 +01:00
klassiker
c5fbddee05 fix(git-fetcher): ensure the specified commit is used after checkout (#10310)
* fix(git-fetcher): ensure the specified commit is used after checkout

* fix(git-resolver): always resolve to a full commit

* chore: add changeset heavy-dragons-start

* test: fix related test case

* test: fix some other test that gets stuck

* Update heavy-dragons-start.md with PR reference

Add reference to pull request #10310 for clarity.
2025-12-17 03:26:18 +01:00
Zoltan Kochan
491a84fb26 feat: use ESM instead of commonjs (#9870) 2025-08-25 10:02:00 +02:00
Zoltan Kochan
27cbc09206 style: fix jest-related linting issues (#9894) 2025-08-22 21:56:49 +02:00
Zoltan Kochan
facd7656e8 refactor: always use extensions in relative imports (#9878) 2025-08-19 15:25:11 +02:00
btea
d15c27438d fix: correct url loss pathname (#9548)
close #9545
2025-05-15 18:30:19 +02:00
Zoltan Kochan
8a9f3a4835 refactor: rename pref to bare specifier (#9445) 2025-04-20 22:58:08 +02:00
Zoltan Kochan
5b73df1eb1 refactor: resolvers should calculate the specifiers that are saved into package.json (#9426)
* refactor: resolvers should return specifier templates

* refactor: updating workspace protocol specs in package.json

* refactor: move workspace selector calculation logic to npm-resolver

* refactor: move workspace selector calculation logic to npm-resolver

* refactor: calculating range in npm-resolver

* refactor: rename normalizedPref and specifierTemplate to specifier

* refactor: specifiers creation

* refactor: npm-resolver

* refactor: remove which-version-is-pinned package

* refactor: which version is pinned

* docs: add changesets

* refactor: implement suggestions

* refactor: revert regex usage
2025-04-18 17:48:03 +02:00
mato533
d6a4ff1892 feat(git-resolver): respect proxy settings when resolving git (#9016)
close: #6530
2025-01-28 18:48:37 +01:00
Utsav Shah
b10096228c fix(git-resolver): handle private git repo resolution (#8906)
* fix(git-resolver): handle private git repo resolution

In the case where:

1. No git auth token was specified by the user
2. The package requested to be fetched via https
3. The user does not have SSH access to the repo but has HTTPS access
4. The package was hosted in a private GitHub repo

pnpm would fallback to using SSH since it was a "likely private repo"
and would fail to resolve the package. Now, rather than only checking if
there is an auth token specified, it also checks both:

1. Is the repo private
2. Does the user have access to ls-remote it.

And if these conditions are true, it tries to use https anyway.

This matches the behavior of npm and Yarn berry. Yarn classic also has
this bug, and there's a code comment that alludes to it.
2024-12-27 02:48:59 +01:00
Khải
9b4f73caaf chore(scripts): typecheck-only (#8395)
* chore(scripts): typecheck-only

* feat: change all configuration

* feat: include pnpm/ and pnpm/test/

* chore(deps): remove unused dependency

* refactor(typescript-only): use find-packages

* refactor(typescript-only): refactor paths

* fix: typescript-only

* fix: update compile-only

* fix: compile pnpm

* fix: windows

* fix: windows

* chore: meta-updater

* refactor(tsconfig): remove explicit composite

* fix: path in windows

* feat: don't depend on cwd

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2024-08-11 08:26:01 +02:00
Brandon Cheng
c7e1b6fae8 chore: configure TypeScript project references for tests (#8128)
* refactor: store link values before converting to references

* fix: use .sort() without localeCompare

https://github.com/pnpm/pnpm/pull/8128#discussion_r1614031566

> Nit, but you probably just want to call sort without a comparison
> function; these are already strings and locale compare is not a good
> comparison for anything but human readable strings since it will
> differ on different people's machines based on their language setting.
> I've hit this too many times before for code gen.

* feat: configure meta-updater to write test/tsconfig.json files

* fix: relative imports for __typings__

* chore: `pnpm run meta-updater`

* fix: explicitly use test/tsconfig.json for ts-jest
2024-05-31 12:48:13 +02:00
Zoltan Kochan
c969f374de fix: git-hosted dependencies (#8005)
close #7990
2024-04-24 13:46:09 +02:00
Zoltan Kochan
53594a3787 feat!: package ID should be an exact version spec (#7748) 2024-03-10 14:46:25 +01:00
Rex Zeng
b13d2dc1ae feat: add sub folder support for git url (#7487)
close #4765

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2024-01-24 02:15:02 +01:00
Khải
22bbe92554 feat(git-resolver): fix private git repo (#6832)
fixes #6827
closes #6829
2023-07-19 13:48:05 +03:00
Zoltan Kochan
de9b6c20d4 revert: "fix(git-resolver): wrong scheme to git ls-remote (#6806)"
This reverts commit 6fe0b60e61.

ref #6827
2023-07-18 12:44:51 +03:00
Khải
6fe0b60e61 fix(git-resolver): wrong scheme to git ls-remote (#6806)
close #6805
2023-07-17 11:40:54 +03:00
Zoltan Kochan
4b97f1f07a perf: don't use await inside loops (#6617) 2023-06-05 12:12:47 +03:00
Andrei Neculau
28796377cb fix: handle git+ssh with semver (#6239) 2023-03-21 00:40:26 +02:00
Zoltan Kochan
9cbe293876 refactor: put all non-public projects to directories that start with __ 2022-11-20 02:40:44 +02:00
Zoltan Kochan
4ca53b0b50 refactor: group projects in different subdirectories (#5659) 2022-11-20 01:35:22 +02:00