Commit Graph

9 Commits

Author SHA1 Message Date
Zoltan Kochan
52be454d57 fix: infer missing platform fields of optional dependencies from the package name (#12312)
* fix: infer missing platform fields of optional deps from the package name

Some registries strip the os/cpu/libc fields (or just libc) from the
version objects of the packuments they serve. Resolution then saw every
platform-specific optional dependency as platform-unrestricted, so pnpm
downloaded and installed the binaries of every platform regardless of
supportedArchitectures, and wrote lockfile entries without the platform
fields, which broke installs from that lockfile on every machine.

Platform-specific binary packages encode their platform in the package
name (e.g. @nx/nx-win32-arm64-msvc), so packageIsInstallable now fills
the missing platform fields of an optional dependency from the name's
tokens. Since every install path decides installability through that
check before fetching, foreign-platform binaries are skipped without
even downloading them, in fresh resolution and in headless installs
with both node linkers alike. A package that declares no platform
fields at all is treated as platform-specific only when an operating
system is recognized in its name, so a generic name segment (such as
'arm' on its own) never gets a package skipped.

Fixes https://github.com/pnpm/pnpm/issues/11702
Fixes https://github.com/pnpm/pnpm/issues/9940

* chore: add platform name tokens to the cspell dictionary

* fix(package-is-installable): infer missing platform fields of optional deps from the package name

Port of pnpm commit https://github.com/pnpm/pnpm/commit/34875b2d7c
(PR https://github.com/pnpm/pnpm/pull/12312). Some registries strip
the os/cpu/libc fields (or just libc) from the version objects of the
packuments they serve, and lockfile entries written from such
metadata lack the fields too, so every platform's binaries were
installed regardless of supportedArchitectures.

Platform-specific binary packages encode their platform in the
package name (e.g. @nx/nx-win32-arm64-msvc), so the installability
check now fills the missing platform fields of an optional dependency
from the name's tokens: infer_platform_from_package_name +
inferred_platform in pacquet-package-is-installable, applied inside
package_is_installable (hoisted linker) and in
compute_skipped_snapshots (isolated linker, with the check cache
keyed by the snapshot's optional flag since the verdicts can differ).
The any_installability_constraint fast path now also considers
optional snapshots whose names infer a platform their metadata row
does not declare, so the inference is reachable on lockfiles without
any declared constraint.

Same guard rails as upstream: declared fields always win (each field
is filled only when missing — a missing libc alone is inferred,
disambiguating -gnu vs -musl), and a package declaring no platform
fields at all engages the inference only when an operating-system
token is recognized in its name, so a generic name segment such as
'arm' on its own never gets a package skipped.

Fixes https://github.com/pnpm/pnpm/issues/11702
Fixes https://github.com/pnpm/pnpm/issues/9940

* test: shut the metadata-stripping proxy down cleanly and forward the request method
2026-06-10 21:51:40 +02:00
Marvin Hagemeister
49e6074644 test: replace @pnpm/registry-mock with an in-repo in-process registry (#11927)
Replace the external `@pnpm/registry-mock` (Verdaccio) test dependency with an in-repo, in-process registry that serves package fixtures to **both** the pacquet Rust tests and the pnpm CLI (Jest) tests. No separately managed registry process is needed.

### How it works

- **Fixtures** live at `registry/.fixtures/packages/<name>/<version>/…`, moved verbatim from [`pnpm/registry-mock`](https://github.com/pnpm/registry-mock) (keyed by each `package.json`'s `name`+`version`).
- **`pnpm-registry-fixtures`** builds verdaccio-shaped storage from those fixtures; the in-tree **`pnpm-registry`** crate serves it.
  - Files whose names differ only by case (`@pnpm.e2e/with-same-file-in-different-cases`) and `bundleDependencies` trees are composed **in memory** by the builder, since neither can be committed to the working tree.
- **pacquet**: `pacquet-testing-utils`' `TestRegistry` starts the server lazily (once per process) in proxy mode, serving `@pnpm.e2e` fixtures locally and falling through to the npm uplink for real packages (`is-positive`, `is-negative`, …) — matching how registry-mock behaved.
- **pnpm CLI**: the `with-registry` Jest `globalSetup` builds storage from the fixtures via the new `pnpm-registry-prepare` binary (built from source in the Test CI job) and serves it with `pnpm-registry`. `REGISTRY_MOCK_PORT` / `REGISTRY_MOCK_CREDENTIALS` / `getIntegrity` now come from `@pnpm/testing.registry-mock`.

### Result

`@pnpm/registry-mock` is removed from every manifest, the catalog, and `packageExtensions`; `cargo test` / `cargo nextest run` / `just test` and the pnpm CLI Jest suites all run registry-backed tests without launching Verdaccio.
2026-05-29 14:35:45 +02:00
Ethan Setnik
3cf2b86579 fix: preserve tarball dependency integrity in the lockfile (#12040)
* fix: preserve tarball dependency integrity in the lockfile

URL/tarball resolvers do not return an integrity (it is only known after
the tarball is downloaded). When a remote-tarball dependency was reused
from the lockfile without being re-fetched, the freshly resolved
resolution had no integrity and the existing one was dropped, breaking
subsequent --frozen-lockfile installs under the lockfile-integrity
hardening (ERR_PNPM_MISSING_TARBALL_INTEGRITY). Carry the integrity over
from the current resolution instead.

Closes #12001

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(package-requester): simplify tarball integrity carryover guard

Align the integrity carryover added in the previous commit with its
sibling block in the download path: use `!resolution.type` (the idiom
already used there) and drop the `newIntegrity == null` clause, which is
redundant once `resolution` is the freshly resolved resolution.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(pacquet): cover tarball-dependency integrity preservation (#12001)

pnpm #12040 fixes dropping a remote-tarball dependency's integrity when
an unrelated package is installed afterwards. Pacquet can't reach that
scenario yet: a non-registry https-tarball direct dependency hits the
TarballResolver, which returns no name_ver/integrity, so lockfile build
panics with MissingSuffix. Add the regression test for the target
behavior, gated with allow_known_failure! until external tarball deps
install. Tracked in #12053.

---------

Co-authored-by: Zoltan Kochan <z@kochan.io>
2026-05-29 02:17:58 +00:00
Zoltan Kochan
187049055f chore: upgrade @typescript/native-preview to 7.0.0-dev.20260421.2 (#11332)
* chore: upgrade @typescript/native-preview to 7.0.0-dev.20260421.2

- Add explicit `types: ["node"]` to the shared tsconfig because tsgo
  20260421 no longer auto-acquires `@types/*` from `node_modules`.
- Refactor test files to explicitly import jest globals (`describe`,
  `it`, `test`, `expect`, `beforeEach`, etc.) from `@jest/globals`
  instead of relying on `@types/jest` ambient declarations. Under the
  new tsgo build, `import { jest } from '@jest/globals'` shadows the
  ambient `jest` namespace, breaking `@types/jest`'s `declare var
  describe: jest.Describe;` globals.
- Add `@jest/globals` to each package's devDependencies where tests
  now import from it, and add `@types/node` to packages that need it
  but were relying on hoisted resolution.
- Replace `fail()` calls with `throw new Error(...)` since `fail` is
  no longer globally available.

* chore: fix remaining tsgo type-strictness errors

- Strip `as <PnpmType>` casts on objects passed to toMatchObject /
  toStrictEqual / toEqual; @jest/globals rejects the typed objects
  (which include AsymmetricMatchers) vs. the repo-specific type.
- Type `jest.fn<...>()` explicitly where the mock's signature matters
  for toHaveBeenCalledWith.
- Replace `beforeEach(() => X)` with `beforeEach(() => { X })` so the
  return value is void, as the stricter jest typing requires.
- Use `expect.objectContaining({...})` in one place where the full
  expected object triggered stricter type resolution.
- Cast `prompt.mock.calls` arg through `as unknown as Record<...>[]`
  for patch.test.ts's nested-array matchers.
- Fix off-by-one `<reference path>` in pnpm/test/getConfig.test.ts
  that only surfaced now.
- Move `@jest/globals` from devDependencies to dependencies in the
  two `__utils__` packages that import it from `src/`.
- Clean up unused imports from the @jest/globals migration.

* chore: address Copilot review on #11332

- Move misplaced `@jest/globals` imports to the top import block in
  checkEngine, run.ts, and workspace/root-finder tests where the
  script dropped them below executable code.
- Replace `try { await x(); throw new Error('should have thrown') } catch`
  in bins/linker, lockfile/fs, and resolving/local-resolver tests with
  `await expect(x()).rejects.toMatchObject({...})`. The old pattern
  swallowed an unrelated `throw` if the under-test call silently
  succeeded, which would fail on the catch-block assertion with a
  misleading message.
2026-04-21 22:50:40 +02:00
Zoltan Kochan
45a6cb6b2a refactor(auth): unify auth/SSL into structured configByUri (#11201)
Replaces the dual `authConfig` (raw .npmrc) + `authInfos` (parsed auth) + `sslConfigs` (parsed SSL) pattern with a single structured `configByUri: Record<string, RegistryConfig>` field on Config.

### New types (`@pnpm/types`)
- **`RegistryConfig`** — per-registry config: `{ creds?: Creds, tls?: TlsConfig }`
- **`Creds`** — auth credentials: `{ authToken?, basicAuth?, tokenHelper? }`
- **`TlsConfig`** — TLS config: `{ cert?, key?, ca? }`

### Key changes
- Rewrite `createGetAuthHeaderByURI` to accept `Record<string, RegistryConfig>` instead of raw .npmrc key-value pairs
- Eliminate duplicate auth parsing between `getAuthHeadersFromConfig` and `getNetworkConfigs`
- Remove `authConfig` from the install pipeline (`StrictInstallOptions`, `HeadlessOptions`), replaced by `configByUri`
- Remove `sslConfigs` from Config — SSL fields now live in `configByUri[uri].tls`
- Remove `authConfig['registry']` mutation in `extendInstallOptions` (default registry now passed directly to `createGetAuthHeaderByURI`)
- `authConfig` remains on Config only for raw .npmrc access (config commands, error reporting, config inheritance)

### Security
- tokenHelper in project .npmrc now throws instead of being silently stripped
- tokenHelper execution uses `shell: false` to prevent shell metacharacter injection
- Basic auth uses `Buffer.from().toString('base64')` instead of `btoa()` for Unicode safety
- Dispatcher only creates custom agents when entries actually have TLS fields
2026-04-05 20:15:10 +02:00
Zoltan Kochan
96704a1c58 refactor(config): rename rawConfig to authConfig, add nodeDownloadMirrors, simplify config reader (#11194)
Major cleanup of the config system after migrating settings from `.npmrc` to `pnpm-workspace.yaml`.

### Config reader simplification
- Remove `checkUnknownSetting` (dead code, always `false`)
- Trim `npmConfigTypes` from ~127 to ~67 keys (remove unused npm config keys)
- Replace `rcOptions` iteration over all type keys with direct construction from defaults + auth overlay
- Remove `rcOptionsTypes` parameter from `getConfig()` and its assembly chain

### Rename `rawConfig` to `authConfig`
- `rawConfig` was a confusing mix of auth data and general settings
- Non-auth settings are already on the typed `Config` object — stop duplicating them in `rawConfig`
- Rename `rawConfig` → `authConfig` across the codebase to clarify it only contains auth/registry data from `.npmrc`

### Remove `rawConfig` from non-auth consumers
- **Lifecycle hooks**: replace `rawConfig: object` with `userAgent?: string` — only user-agent was read
- **Fetchers**: remove unused `rawConfig` from git fetcher, binary fetcher, tarball fetcher, prepare-package
- **Update command**: use `opts.production/dev/optional` instead of `rawConfig.*`
- **`pnpm init`**: accept typed init properties instead of parsing `rawConfig`

### Add `nodeDownloadMirrors` setting
- New `nodeDownloadMirrors?: Record<string, string>` on `PnpmSettings` and `Config`
- Replaces the `node-mirror:<channel>` pattern that was stored in `rawConfig`
- Configured in `pnpm-workspace.yaml`:
  ```yaml
  nodeDownloadMirrors:
    release: https://my-mirror.example.com/download/release/
  ```
- Remove unused `rawConfig` from deno-resolver and bun-resolver

### Refactor `pnpm config get/list`
- New `configToRecord()` builds display data from typed Config properties on the fly
- Excludes sensitive internals (`authInfos`, `sslConfigs`, etc.)
- Non-types keys (e.g., `package-extensions`) resolve through `configToRecord` instead of direct property access
- Delete `processConfig.ts` (replaced by `configToRecord.ts`)

### Pre-push hook improvement
- Add `compile-only` (`tsgo --build`) to pre-push hook to catch type errors before push
2026-04-04 20:33:43 +02:00
Zoltan Kochan
6c480a4375 perf: replace node-fetch with undici (#10537)
Replace node-fetch with native undici for HTTP requests throughout pnpm.

Key changes:
- Replace node-fetch with undici's fetch() and dispatcher system
- Replace @pnpm/network.agent with a new dispatcher module in @pnpm/network.fetch
- Cache dispatchers via LRU cache keyed by connection parameters
- Handle proxies via undici ProxyAgent instead of http/https-proxy-agent
- Convert test mocking from nock to undici MockAgent where applicable
- Add minimatch@9 override to fix ESM incompatibility with brace-expansion
2026-03-29 12:44:00 +02:00
Zoltan Kochan
4a36b9a110 refactor: rename internal packages to @pnpm/<domain>.<leaf> convention (#10997)
## Summary

Rename all internal packages so their npm names follow the `@pnpm/<domain>.<leaf>` convention, matching their directory structure. Also rename directories to remove redundancy and improve clarity.

### Bulk rename (94 packages)

All `@pnpm/` packages now derive their name from their directory path using dot-separated segments. Exceptions: `packages/`, `__utils__/`, and `pnpm/artifacts/` keep leaf names only.

### Directory renames (removing redundant prefixes)

- `cli/cli-meta` → `cli/meta`, `cli/cli-utils` → `cli/utils`
- `config/config` → `config/reader`, `config/config-writer` → `config/writer`
- `fetching/fetching-types` → `fetching/types`
- `lockfile/lockfile-to-pnp` → `lockfile/to-pnp`
- `store/store-connection-manager` → `store/connection-manager`
- `store/store-controller-types` → `store/controller-types`
- `store/store-path` → `store/path`

### Targeted renames (clarity improvements)

- `deps/dependency-path` → `deps/path` (`@pnpm/deps.path`)
- `deps/calc-dep-state` → `deps/graph-hasher` (`@pnpm/deps.graph-hasher`)
- `deps/inspection/dependencies-hierarchy` → `deps/inspection/tree-builder` (`@pnpm/deps.inspection.tree-builder`)
- `bins/link-bins` → `bins/linker`, `bins/remove-bins` → `bins/remover`, `bins/package-bins` → `bins/resolver`
- `installing/get-context` → `installing/context`
- `store/package-store` → `store/controller`
- `pkg-manifest/manifest-utils` → `pkg-manifest/utils`

### Manifest reader/writer renames

- `workspace/read-project-manifest` → `workspace/project-manifest-reader` (`@pnpm/workspace.project-manifest-reader`)
- `workspace/write-project-manifest` → `workspace/project-manifest-writer` (`@pnpm/workspace.project-manifest-writer`)
- `workspace/read-manifest` → `workspace/workspace-manifest-reader` (`@pnpm/workspace.workspace-manifest-reader`)
- `workspace/manifest-writer` → `workspace/workspace-manifest-writer` (`@pnpm/workspace.workspace-manifest-writer`)

### Workspace package renames

- `workspace/find-packages` → `workspace/projects-reader`
- `workspace/find-workspace-dir` → `workspace/root-finder`
- `workspace/resolve-workspace-range` → `workspace/range-resolver`
- `workspace/filter-packages-from-dir` merged into `workspace/filter-workspace-packages` → `workspace/projects-filter`

### Domain moves

- `pkg-manifest/read-project-manifest` → `workspace/project-manifest-reader`
- `pkg-manifest/write-project-manifest` → `workspace/project-manifest-writer`
- `pkg-manifest/exportable-manifest` → `releasing/exportable-manifest`

### Scope

- 1206 files changed
- Updated: package.json names/deps, TypeScript imports, tsconfig references, changeset files, renovate.json, test fixtures, import ordering
2026-03-17 21:50:40 +01:00
Zoltan Kochan
f47ef4b125 refactor: reorganize monorepo domain structure (#10987)
Reorganize the monorepo's top-level domain directories for clarity:

- pkg-manager/ split into:
  - installing/ (core, headless, client, resolve-dependencies, etc.)
  - installing/linking/ (hoist, direct-dep-linker, modules-cleaner, etc.)
  - bins/ (link-bins, package-bins, remove-bins)
- completion/ merged into cli/
- dedupe/ moved to installing/dedupe/
- env/ renamed to engine/ with subdomains:
  - engine/runtime/ (node.fetcher, node.resolver, plugin-commands-env, etc.)
  - engine/pm/ (plugin-commands-setup, plugin-commands-self-updater)
- env.path moved to shell/
- tools/ and runtime/ dissolved
- reviewing/ and lockfile audit packages moved to deps/:
  - deps/inspection/ (list, outdated, dependencies-hierarchy)
  - deps/compliance/ (audit, licenses, sbom)
- registry/ moved to resolving/registry/
- semver/peer-range moved to deps/
- network/fetching-types moved to fetching/
- packages/ slimmed down, moving packages to proper domains:
  - calc-dep-state, dependency-path -> deps/
  - parse-wanted-dependency -> resolving/
  - git-utils -> network/
  - naming-cases -> text/
  - make-dedicated-lockfile -> lockfile/
  - render-peer-issues -> installing/
  - plugin-commands-doctor -> cli/
  - plugin-commands-init -> workspace/

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 13:45:54 +01:00