mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-01 20:49:45 -04:00
feat(installing): delegate fetch / import / link to pacquet when configured (#11734)
When `configDependencies` declares pacquet (under either the unscoped `pacquet` or the scoped `@pnpm/pacquet` alias), pnpm delegates the fetch / import / link / build phases of an install to the pacquet Rust binary. Pnpm keeps owning dependency resolution — pacquet's resolver isn't ready yet — and hands pacquet a freshly-written lockfile to materialize. Covered install shapes: - frozen install (`tryFrozenInstall` → pacquet, no resolve needed) - default isolated `nodeLinker` (`installInContext`: lockfileOnly resolve via JS, then pacquet) - hoisted `nodeLinker` (same resolve-then-materialize shape) - workspace partial install (subset of workspace projects mutated) - agent-server install (`@pnpm/agent.client` resolves, pacquet materializes) ```yaml # pnpm-workspace.yaml configDependencies: "@pnpm/pacquet": "^0.2.0" # or unscoped `pacquet` ``` ## How it works - `installing/commands/src/runPacquet.ts` resolves the platform binary via `createRequire(realpath(.pnpm-config/<name>/package.json))` — same algorithm the upstream wrapper uses, but skipping the JS shim's extra Node startup. - Pacquet's NDJSON stderr is forwarded through `@pnpm/logger`'s global `streamParser` so `@pnpm/cli.default-reporter` renders its events the same way it renders pnpm's own. Non-JSON stderr lines pass through verbatim. - A few pnpm-side log emits (`importing_done` placeholder, `pnpm:summary`) are suppressed when pacquet will take over so the reporter doesn't close streams or lock in empty diffs before pacquet's real events arrive. Pacquet's duplicate `pnpm:progress status:resolved` events are filtered on the resolve-then-materialize paths so the reporter doesn't double-count. - `installing/deps-installer/src/install/index.ts` gates the delegation on a `runPacquet?: () => Promise<void>` callback in `StrictInstallOptions`. The CLI layer in `installing/commands/src/installDeps.ts` constructs the callback, threaded through both the single-project and workspace-recursive paths. - The `pacquet` and `@pnpm/pacquet` npm packages ship the same JS shim from `pacquet/npm/pacquet/scripts/generate-packages.mjs`; per-platform binaries stay under the existing `@pacquet/<plat>-<arch>` scope and aren't duplicated.
This commit is contained in:
15
.changeset/pacquet-frozen-install-delegation.md
Normal file
15
.changeset/pacquet-frozen-install-delegation.md
Normal file
@@ -0,0 +1,15 @@
|
||||
---
|
||||
"@pnpm/installing.commands": minor
|
||||
"@pnpm/installing.deps-installer": minor
|
||||
"pnpm": minor
|
||||
---
|
||||
|
||||
Adding [`pacquet`](https://github.com/pnpm/pnpm/tree/main/pacquet) (the Rust port of pnpm) to `configDependencies` in `pnpm-workspace.yaml` now delegates the materialization phase of `pnpm install` to the pacquet binary instead of running the JS installer's headless path. Pacquet emits the same `pnpm:*` NDJSON log events that `@pnpm/cli.default-reporter` already parses, so the install renders identically. Absent the `pacquet` entry, behavior is unchanged.
|
||||
|
||||
```yaml
|
||||
# pnpm-workspace.yaml
|
||||
configDependencies:
|
||||
pacquet: "^0.1.0"
|
||||
```
|
||||
|
||||
Pacquet takes over every place pnpm would otherwise call `headlessInstall`: the frozen-install path, the hoisted-`nodeLinker` install, the workspace partial-install (where pnpm runs a `lockfileOnly` resolve pass first), and the agent-server install. In all cases pnpm still owns dependency resolution; pacquet only fetches and imports from the freshly-written lockfile. This is an opt-in preview of the Rust install engine [#11723](https://github.com/pnpm/pnpm/issues/11723).
|
||||
9
.github/workflows/pacquet-release-to-npm.yml
vendored
9
.github/workflows/pacquet-release-to-npm.yml
vendored
@@ -166,7 +166,7 @@ jobs:
|
||||
run: |
|
||||
node pacquet/npm/pacquet/scripts/generate-packages.mjs
|
||||
cat pacquet/npm/pacquet/package.json
|
||||
for package in pacquet/npm/pacquet*; do cat $package/package.json ; echo ; done
|
||||
for package in pacquet/npm/pacquet* pacquet/npm/pnpm-pacquet; do cat $package/package.json ; echo ; done
|
||||
|
||||
- name: Publish npm packages as latest
|
||||
# Auth is via npm's trusted publishing: `id-token: write` above grants
|
||||
@@ -174,7 +174,12 @@ jobs:
|
||||
# so no NPM_TOKEN is needed. `--provenance` attaches the same OIDC
|
||||
# token to a provenance attestation on each tarball.
|
||||
# The trailing slash on $package/ changes it to publishing the directory.
|
||||
# `pnpm-pacquet` is the `@pnpm/pacquet` scoped alias mirror —
|
||||
# same shim and same `@pacquet/<plat>-<arch>` optional deps as the
|
||||
# unscoped `pacquet`. Published alongside so users can adopt
|
||||
# either name; the per-platform binary sub-packages stay under
|
||||
# the `@pacquet/` scope (no need to mirror those).
|
||||
run: |
|
||||
for package in pacquet/npm/pacquet*; do
|
||||
for package in pacquet/npm/pacquet* pacquet/npm/pnpm-pacquet; do
|
||||
pnpm publish "$package/" --tag latest --access public --provenance --no-git-checks
|
||||
done
|
||||
|
||||
@@ -50,6 +50,7 @@ import {
|
||||
type RecursiveOptions,
|
||||
type UpdateDepsMatcher,
|
||||
} from './recursive.js'
|
||||
import { makeRunPacquet } from './runPacquet.js'
|
||||
import { createWorkspaceSpecs, updateToWorkspacePackagesFromManifest } from './updateWorkspaceDependencies.js'
|
||||
|
||||
const OVERWRITE_UPDATE_OPTIONS = {
|
||||
@@ -195,6 +196,29 @@ export async function installDeps (
|
||||
opts['preserveWorkspaceProtocol'] = !opts.linkWorkspacePackages
|
||||
}
|
||||
const store = await createStoreController(opts)
|
||||
// When `configDependencies` declares pacquet, build the alternative
|
||||
// install engine the deps-installer delegates to. The CLI layer owns
|
||||
// the construction so the installer doesn't need to know about
|
||||
// pacquet's binary path, CLI surface, or any settings that only
|
||||
// pacquet consumes. Threaded through both the workspace recursive
|
||||
// path and the single-project path below. Two declaration names are
|
||||
// accepted: the original unscoped `pacquet` and the official scoped
|
||||
// `@pnpm/pacquet` mirror. Both packages ship the same JS shim and
|
||||
// optional `@pacquet/<plat>-<arch>` binary sub-packages, so the
|
||||
// resolved \`node_modules/.pnpm-config/<name>\` layout pacquet's
|
||||
// wrapper expects is identical either way.
|
||||
const pacquetConfigDepName = opts.configDependencies?.['@pnpm/pacquet'] != null
|
||||
? '@pnpm/pacquet'
|
||||
: opts.configDependencies?.pacquet != null
|
||||
? 'pacquet'
|
||||
: undefined
|
||||
const runPacquet = pacquetConfigDepName != null
|
||||
? makeRunPacquet({
|
||||
lockfileDir: opts.lockfileDir ?? opts.dir,
|
||||
packageName: pacquetConfigDepName,
|
||||
argv: opts.argv.original,
|
||||
})
|
||||
: undefined
|
||||
const includeDirect = opts.includeDirect ?? {
|
||||
dependencies: true,
|
||||
devDependencies: true,
|
||||
@@ -243,6 +267,7 @@ export async function installDeps (
|
||||
selectedProjectsGraph,
|
||||
storeControllerAndDir: store,
|
||||
workspaceDir: opts.workspaceDir,
|
||||
runPacquet,
|
||||
},
|
||||
opts.update ? 'update' : (params.length === 0 ? 'install' : 'add')
|
||||
)
|
||||
@@ -292,6 +317,7 @@ export async function installDeps (
|
||||
workspacePackages,
|
||||
preferredVersions: opts.packageVulnerabilityAudit ? preferNonvulnerablePackageVersions(opts.packageVulnerabilityAudit) : undefined,
|
||||
handleResolutionPolicyViolations: policyHandlers?.handleResolutionPolicyViolations,
|
||||
runPacquet,
|
||||
}
|
||||
|
||||
let updateMatch: UpdateDepsMatcher | null
|
||||
@@ -428,6 +454,7 @@ export async function installDeps (
|
||||
allProjectsGraph: opts.allProjectsGraph!,
|
||||
selectedProjectsGraph,
|
||||
workspaceDir: opts.workspaceDir, // Otherwise TypeScript doesn't understand that is not undefined
|
||||
runPacquet,
|
||||
}, 'install')
|
||||
|
||||
if (opts.ignoreScripts) return
|
||||
|
||||
@@ -119,6 +119,13 @@ export type RecursiveOptions = CreateStoreControllerOptions & Pick<Config,
|
||||
resolutionVerifiers: ResolutionVerifier[]
|
||||
}
|
||||
pnpmfile: string[]
|
||||
/**
|
||||
* Alternative install engine (today: pacquet) the deps-installer
|
||||
* delegates the materialization phase to. Built in `installDeps`
|
||||
* when `configDependencies.pacquet` is declared, threaded through
|
||||
* here so the recursive workspace path picks it up too.
|
||||
*/
|
||||
runPacquet?: () => Promise<void>
|
||||
} & Partial<
|
||||
Pick<Config,
|
||||
| 'ci'
|
||||
|
||||
177
installing/commands/src/runPacquet.ts
Normal file
177
installing/commands/src/runPacquet.ts
Normal file
@@ -0,0 +1,177 @@
|
||||
import { spawn } from 'node:child_process'
|
||||
import fs from 'node:fs'
|
||||
import { createRequire } from 'node:module'
|
||||
import path from 'node:path'
|
||||
import readline from 'node:readline'
|
||||
import type { Writable } from 'node:stream'
|
||||
|
||||
import { PnpmError } from '@pnpm/error'
|
||||
import { logger, streamParser } from '@pnpm/logger'
|
||||
import chalk from 'chalk'
|
||||
|
||||
// The runtime `streamParser` is a `Transform` stream (split2 + JSON.parse).
|
||||
// Its public typing only exposes `on`/`removeListener`, so we narrow to the
|
||||
// writable side here to feed pacquet's NDJSON lines back through the same
|
||||
// parser that `@pnpm/cli.default-reporter` listens on.
|
||||
const streamParserWritable = streamParser as unknown as Writable
|
||||
|
||||
export interface MakeRunPacquetOpts {
|
||||
lockfileDir: string
|
||||
/**
|
||||
* Which `configDependencies` entry installed pacquet: either the
|
||||
* original unscoped `pacquet` or the official scoped
|
||||
* `@pnpm/pacquet` mirror. Drives the directory we look in under
|
||||
* `node_modules/.pnpm-config/<packageName>/`. Both packages ship
|
||||
* the same shim and the same `@pacquet/<plat>-<arch>` binary
|
||||
* sub-packages, so the rest of the lookup is identical.
|
||||
*/
|
||||
packageName: 'pacquet' | '@pnpm/pacquet'
|
||||
/**
|
||||
* The user's original `pnpm` argv (`process.argv.slice(2)`). Not
|
||||
* forwarded to pacquet — we only inspect it to warn about flags
|
||||
* pacquet won't see.
|
||||
*/
|
||||
argv: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the install-engine callback `mutateModules` invokes when
|
||||
* `configDependencies` declares pacquet. Returns `undefined` when no
|
||||
* pacquet binary is on disk — the caller falls back to the JS path in
|
||||
* that case.
|
||||
*
|
||||
* The callback spawns the pacquet binary installed under
|
||||
* `node_modules/.pnpm-config/pacquet` and forwards the user's own
|
||||
* pnpm CLI flags. Pacquet's NDJSON stderr is parsed line-by-line and
|
||||
* the valid JSON records are re-emitted on pnpm's global
|
||||
* `streamParser` so `@pnpm/cli.default-reporter` renders pacquet's
|
||||
* events the same way it renders pnpm's own. Non-JSON stderr lines
|
||||
* (panic backtraces, unexpected diagnostics) are forwarded to the
|
||||
* real stderr verbatim so they reach the user.
|
||||
*/
|
||||
/** Args the deps-installer passes per pacquet invocation. */
|
||||
export interface RunPacquetCallOpts {
|
||||
/**
|
||||
* `true` when pnpm has already run a lockfileOnly resolve pass and
|
||||
* the reporter has already accumulated one `pnpm:progress
|
||||
* status:resolved` per package. Pacquet's own `resolved` events
|
||||
* (emitted for wire-format parity as it walks the lockfile) are
|
||||
* dropped on the way back through the reader so the reporter
|
||||
* doesn't double-count. The frozen-install path passes `false`:
|
||||
* pnpm did no resolution there, so pacquet's events are the only
|
||||
* source.
|
||||
*/
|
||||
filterResolvedProgress?: boolean
|
||||
}
|
||||
|
||||
export function makeRunPacquet (opts: MakeRunPacquetOpts): (callOpts?: RunPacquetCallOpts) => Promise<void> {
|
||||
return async (callOpts) => {
|
||||
const pacquetBin = resolvePacquetBin(opts.lockfileDir, opts.packageName)
|
||||
// Always the same fixed args. We don't forward pnpm's CLI flags
|
||||
// even though pacquet's `install` subcommand mirrors most of them:
|
||||
// pnpm has commands like `add` and `update` that carry flags
|
||||
// pacquet's `install` doesn't recognize (e.g., `--save-dev`,
|
||||
// `--save-peer`), and clap would reject them. The settings users
|
||||
// care about live in `pnpm-workspace.yaml` / `.npmrc`, which
|
||||
// pacquet reads on its own.
|
||||
const args = ['--reporter=ndjson', 'install', '--frozen-lockfile']
|
||||
const droppedFlags = collectDroppedFlags(opts.argv)
|
||||
if (droppedFlags.length > 0) {
|
||||
logger.warn({
|
||||
message: `The following CLI flags are not forwarded to pacquet and may not be honored: ${droppedFlags.join(' ')}. Move the equivalent settings into pnpm-workspace.yaml (or .npmrc for auth/registry) if pacquet needs them.`,
|
||||
prefix: opts.lockfileDir,
|
||||
})
|
||||
}
|
||||
// Banner so users can tell at a glance their install is going
|
||||
// through the Rust engine rather than the JS path. Chalk is the
|
||||
// same dependency the default reporter uses for the "+ pkg
|
||||
// version" summary, so colorization respects the user's TTY
|
||||
// settings consistently.
|
||||
const banner = [
|
||||
chalk.magentaBright('▶ Using pacquet for this install'),
|
||||
chalk.gray(' pacquet is pnpm\'s Rust install engine (preview); declared in configDependencies.'),
|
||||
].join('\n')
|
||||
logger.info({ message: banner, prefix: opts.lockfileDir })
|
||||
const child = spawn(pacquetBin, args, {
|
||||
cwd: opts.lockfileDir,
|
||||
stdio: ['ignore', 'inherit', 'pipe'],
|
||||
})
|
||||
const filterResolved = callOpts?.filterResolvedProgress === true
|
||||
const rl = readline.createInterface({ input: child.stderr!, crlfDelay: Infinity })
|
||||
rl.on('line', (line) => {
|
||||
if (!line) return
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(line)
|
||||
} catch {
|
||||
process.stderr.write(`${line}\n`)
|
||||
return
|
||||
}
|
||||
if (
|
||||
filterResolved &&
|
||||
typeof parsed === 'object' && parsed !== null &&
|
||||
(parsed as { name?: string }).name === 'pnpm:progress' &&
|
||||
(parsed as { status?: string }).status === 'resolved'
|
||||
) {
|
||||
return
|
||||
}
|
||||
streamParserWritable.write(`${line}\n`)
|
||||
})
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
child.once('error', reject)
|
||||
child.once('close', (code) => {
|
||||
rl.close()
|
||||
if (code === 0) {
|
||||
resolve()
|
||||
return
|
||||
}
|
||||
reject(new PnpmError('PACQUET_INSTALL_FAILED', `pacquet exited with code ${code ?? 'null'}`))
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Path of the platform-specific native pacquet binary for the host. The
|
||||
* pacquet npm package ships a Node wrapper at `bin/pacquet` that uses
|
||||
* `require.resolve('@pacquet/<platform>-<arch>/pacquet[.exe]')` to find
|
||||
* the binary — so the platform package lands as a *sibling* of pacquet,
|
||||
* not inside its own `node_modules` (pacquet's own `node_modules` is
|
||||
* empty after configDependencies install). Use Node's resolver rooted
|
||||
* at pacquet's own `package.json` so we follow the same path the
|
||||
* wrapper would have.
|
||||
*
|
||||
* The `realpathSync` is required: `.pnpm-config/pacquet` is a symlink
|
||||
* into the global virtual store, and Node's `createRequire` builds its
|
||||
* search paths from the *literal* ancestors of the path it's given —
|
||||
* it won't follow the symlink up into the store dir where the
|
||||
* `@pacquet/<plat>-<arch>` sibling actually lives.
|
||||
*/
|
||||
function resolvePacquetBin (lockfileDir: string, packageName: 'pacquet' | '@pnpm/pacquet'): string {
|
||||
const ext = process.platform === 'win32' ? '.exe' : ''
|
||||
const pacquetPkg = fs.realpathSync(path.join(lockfileDir, 'node_modules/.pnpm-config', packageName, 'package.json'))
|
||||
return createRequire(pacquetPkg).resolve(`@pacquet/${process.platform}-${process.arch}/pacquet${ext}`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Pull the CLI flags out of pnpm's argv so we can warn about them
|
||||
* before pacquet runs. We don't forward any of them — pacquet always
|
||||
* gets `install --frozen-lockfile --reporter=ndjson` — but most are
|
||||
* handled by pnpm itself before delegation (`--save-dev` rewrites
|
||||
* `package.json`, `--filter` selects projects, etc.) so listing them
|
||||
* to the user makes the "not forwarded" surface concrete.
|
||||
*
|
||||
* Flags we explicitly emit ourselves (`--frozen-lockfile`,
|
||||
* `--reporter=ndjson`) are filtered out: they're honored, so warning
|
||||
* about them would be misleading. `--config.*` is filtered too —
|
||||
* those configure pnpm's runtime and aren't intended for the install
|
||||
* engine.
|
||||
*/
|
||||
function collectDroppedFlags (argv: string[]): string[] {
|
||||
return argv.filter((arg) => {
|
||||
if (!arg.startsWith('-')) return false
|
||||
if (arg === '--frozen-lockfile' || arg === '--reporter=ndjson') return false
|
||||
if (arg.startsWith('--config.')) return false
|
||||
return true
|
||||
})
|
||||
}
|
||||
@@ -215,6 +215,22 @@ export interface StrictInstallOptions {
|
||||
trustPolicyIgnoreAfter?: number
|
||||
packageVulnerabilityAudit?: PackageVulnerabilityAudit
|
||||
blockExoticSubdeps?: boolean
|
||||
/**
|
||||
* Optional alternative install engine. When set, the frozen-install
|
||||
* path invokes this callback instead of `headlessInstall`. The CLI
|
||||
* layer constructs it (today: spawning the pacquet binary installed
|
||||
* via `configDependencies` and forwarding pnpm's own CLI argv); the
|
||||
* installer treats it as an opaque "do the install" hook so it
|
||||
* doesn't need to know about pacquet's binary path, CLI surface, or
|
||||
* any settings that only pacquet consumes.
|
||||
*
|
||||
* `filterResolvedProgress` tells the helper to drop the engine's
|
||||
* own `pnpm:progress status:resolved` events because pnpm already
|
||||
* emitted one per package during a preceding lockfileOnly resolve
|
||||
* pass. The frozen-install path passes `false` (or nothing): no
|
||||
* resolve pass ran, so the engine's events are the only source.
|
||||
*/
|
||||
runPacquet?: (opts?: { filterResolvedProgress?: boolean }) => Promise<void>
|
||||
/**
|
||||
* If true, `mutateModules` does not emit the per-install `summary` log
|
||||
* event. Used by `pnpm add -g` when it runs multiple isolated installs
|
||||
|
||||
@@ -361,25 +361,49 @@ export async function mutateModules (
|
||||
// resolver's own filters already cover fresh resolution. We run this
|
||||
// exactly once, right after the lockfile is loaded from disk, before any
|
||||
// path branches.
|
||||
const cacheActive = opts.cacheDir != null && opts.resolutionVerifiers.length > 0
|
||||
const wantedLockfilePath = cacheActive
|
||||
? path.resolve(ctx.lockfileDir, await getWantedLockfileName({
|
||||
useGitBranchLockfile: opts.useGitBranchLockfile,
|
||||
mergeGitBranchLockfiles: opts.mergeGitBranchLockfiles,
|
||||
}))
|
||||
: undefined
|
||||
try {
|
||||
await verifyLockfileResolutions(ctx.wantedLockfile, opts.resolutionVerifiers, {
|
||||
cacheDir: opts.cacheDir,
|
||||
lockfilePath: wantedLockfilePath,
|
||||
})
|
||||
} catch (err) {
|
||||
// verifyLockfileResolutions is the one throw site in this function
|
||||
// that's part of normal user-facing operation (a rejected lockfile);
|
||||
// other throws here are unexpected. Detach the reporter listener so
|
||||
// long-lived processes don't leak it on every rejected install.
|
||||
detachReporter()
|
||||
throw err
|
||||
//
|
||||
// Skipped when we already know pacquet will run the install: pacquet's
|
||||
// frozen-install path applies the same resolver-policy gate (port of
|
||||
// this function), so re-running here would duplicate the work — and
|
||||
// for `minimumReleaseAge` in strict mode each lockfile entry is an
|
||||
// HTTP probe.
|
||||
//
|
||||
// The predicate mirrors every short-circuit `tryFrozenInstall` checks
|
||||
// before reaching the pacquet branch: anything that would make it
|
||||
// return null, throw, or fall through to the JS path must keep
|
||||
// verification on. The optimistic `preferFrozenLockfile` path decides
|
||||
// whether to delegate later (based on `allProjectsAreUpToDate`), which
|
||||
// isn't known here — so verification still runs in that window, the
|
||||
// duplicate is bounded to it.
|
||||
const willDelegateToPacquet = opts.runPacquet != null &&
|
||||
installsOnly &&
|
||||
!opts.lockfileOnly &&
|
||||
!opts.fixLockfile &&
|
||||
!opts.dedupe &&
|
||||
!ctx.lockfileHadConflicts &&
|
||||
ctx.existsNonEmptyWantedLockfile &&
|
||||
(opts.frozenLockfile === true || opts.frozenLockfileIfExists === true)
|
||||
if (!willDelegateToPacquet) {
|
||||
const cacheActive = opts.cacheDir != null && opts.resolutionVerifiers.length > 0
|
||||
const wantedLockfilePath = cacheActive
|
||||
? path.resolve(ctx.lockfileDir, await getWantedLockfileName({
|
||||
useGitBranchLockfile: opts.useGitBranchLockfile,
|
||||
mergeGitBranchLockfiles: opts.mergeGitBranchLockfiles,
|
||||
}))
|
||||
: undefined
|
||||
try {
|
||||
await verifyLockfileResolutions(ctx.wantedLockfile, opts.resolutionVerifiers, {
|
||||
cacheDir: opts.cacheDir,
|
||||
lockfilePath: wantedLockfilePath,
|
||||
})
|
||||
} catch (err) {
|
||||
// verifyLockfileResolutions is the one throw site in this function
|
||||
// that's part of normal user-facing operation (a rejected lockfile);
|
||||
// other throws here are unexpected. Detach the reporter listener so
|
||||
// long-lived processes don't leak it on every rejected install.
|
||||
detachReporter()
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
if (opts.hooks.preResolution) {
|
||||
@@ -959,6 +983,27 @@ Note that in CI environments, this setting is enabled by default.`,
|
||||
} else {
|
||||
logger.info({ message: 'Lockfile is up to date, resolution step is skipped', prefix: opts.lockfileDir })
|
||||
}
|
||||
if (opts.runPacquet != null) {
|
||||
try {
|
||||
await opts.runPacquet()
|
||||
} catch (err) {
|
||||
// Same reasoning as the verifyLockfileResolutions catch above: this
|
||||
// is the user-facing failure path, so detach the reporter listener
|
||||
// before rethrowing so long-lived processes don't leak it.
|
||||
detachReporter()
|
||||
throw err
|
||||
}
|
||||
return {
|
||||
updatedProjects: projects.map((mutatedProject) => {
|
||||
const project = ctx.projects[mutatedProject.rootDir]
|
||||
return {
|
||||
...project,
|
||||
manifest: project.originalManifest ?? project.manifest,
|
||||
}
|
||||
}),
|
||||
ignoredBuilds: undefined,
|
||||
}
|
||||
}
|
||||
try {
|
||||
const { stats, ignoredBuilds } = await headlessInstall({
|
||||
...ctx,
|
||||
@@ -1737,8 +1782,16 @@ const _installInContext: InstallFunction = async (projects, ctx, opts) => {
|
||||
})
|
||||
}
|
||||
|
||||
if (opts.nodeLinker !== 'hoisted') {
|
||||
// This is only needed because otherwise the reporter will hang
|
||||
if (opts.nodeLinker !== 'hoisted' && opts.runPacquet == null) {
|
||||
// This is only needed because otherwise the reporter will hang.
|
||||
// Skipped when pacquet is about to take over the materialization
|
||||
// phase: the default reporter completes the progress stream for
|
||||
// this prefix on `importing_done`, so emitting it from the
|
||||
// lockfileOnly resolve pass would prematurely close the stream
|
||||
// and pacquet's own `importing_started` / progress events would
|
||||
// render to a stale stream. Pacquet emits its own
|
||||
// `importing_done` after the install, which closes the stream
|
||||
// normally.
|
||||
stageLogger.debug({
|
||||
prefix: opts.lockfileDir,
|
||||
stage: 'importing_done',
|
||||
@@ -1764,7 +1817,15 @@ const _installInContext: InstallFunction = async (projects, ctx, opts) => {
|
||||
rules: opts.peerDependencyRules,
|
||||
})
|
||||
|
||||
if (!opts.omitSummaryLog) {
|
||||
// Skipped when pacquet will take over the materialization. The
|
||||
// default reporter's `reportSummary` `take(1)`s the first summary
|
||||
// event and combines it with whatever `pkgsDiff` it has at that
|
||||
// moment — which is empty here, since pacquet hasn't emitted its
|
||||
// per-direct-dep `pnpm:root` events yet. Letting pnpm fire summary
|
||||
// now would lock in an empty diff. Pacquet emits its own
|
||||
// `pnpm:summary` after the install completes, by which point its
|
||||
// root events have populated the diff.
|
||||
if (!opts.omitSummaryLog && opts.runPacquet == null) {
|
||||
summaryLogger.debug({ prefix: opts.lockfileDir })
|
||||
}
|
||||
|
||||
@@ -1795,6 +1856,37 @@ function allMutationsAreInstalls (projects: MutatedProject[]): boolean {
|
||||
return projects.every((project) => project.mutation === 'install' && !project.update && !project.updateMatching)
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the pacquet binary if it's configured, otherwise run the JS
|
||||
* `headlessInstall`. Callers can hand off any code path that materializes
|
||||
* an already-resolved lockfile (workspace partial install, hoisted
|
||||
* linker, agent-server install, frozen install) without restating the
|
||||
* delegation choice.
|
||||
*
|
||||
* Pacquet reads the wanted lockfile from disk and produces its own
|
||||
* `pnpm:stats` / `pnpm:ignored-scripts` log events that drive the
|
||||
* reporter. The structured stats / ignoredBuilds return values that
|
||||
* `headlessInstall` produces aren't recovered here — pacquet doesn't
|
||||
* surface them through any return path — so callers get `undefined` for
|
||||
* both. `mutateModules` already tolerates that (it falls back to a zero
|
||||
* stats record and a no-op ignoredBuilds iteration).
|
||||
*/
|
||||
async function materializeOrDelegate (
|
||||
opts: { runPacquet?: (opts?: { filterResolvedProgress?: boolean }) => Promise<void> },
|
||||
runHeadlessInstall: () => Promise<{ stats: InstallationResultStats, ignoredBuilds: IgnoredBuilds | undefined }>
|
||||
): Promise<{ stats?: InstallationResultStats, ignoredBuilds?: IgnoredBuilds }> {
|
||||
if (opts.runPacquet != null) {
|
||||
// Reached only from the resolve-then-materialize call sites
|
||||
// (workspace-partial, hoisted-linker, agent install). Each ran a
|
||||
// lockfileOnly resolve pass that emitted one
|
||||
// `pnpm:progress status:resolved` per package, so pacquet's
|
||||
// duplicate `resolved` events would double the reporter's count.
|
||||
await opts.runPacquet({ filterResolvedProgress: true })
|
||||
return {}
|
||||
}
|
||||
return runHeadlessInstall()
|
||||
}
|
||||
|
||||
const installInContext: InstallFunction = async (projects, ctx, opts) => {
|
||||
try {
|
||||
const isPathInsideWorkspace = isSubdir.bind(null, opts.lockfileDir)
|
||||
@@ -1831,7 +1923,7 @@ const installInContext: InstallFunction = async (projects, ctx, opts) => {
|
||||
...opts,
|
||||
lockfileOnly: true,
|
||||
})
|
||||
const { stats, ignoredBuilds } = await headlessInstall({
|
||||
const { stats, ignoredBuilds } = await materializeOrDelegate(opts, () => headlessInstall({
|
||||
...ctx,
|
||||
...opts,
|
||||
currentEngine: {
|
||||
@@ -1845,7 +1937,7 @@ const installInContext: InstallFunction = async (projects, ctx, opts) => {
|
||||
wantedLockfile: result.newLockfile,
|
||||
useLockfile: opts.useLockfile && ctx.wantedLockfileIsModified,
|
||||
hoistWorkspacePackages: opts.hoistWorkspacePackages,
|
||||
})
|
||||
}))
|
||||
return {
|
||||
...result,
|
||||
stats,
|
||||
@@ -1858,7 +1950,7 @@ const installInContext: InstallFunction = async (projects, ctx, opts) => {
|
||||
...opts,
|
||||
lockfileOnly: true,
|
||||
})
|
||||
const { stats, ignoredBuilds } = await headlessInstall({
|
||||
const { stats, ignoredBuilds } = await materializeOrDelegate(opts, () => headlessInstall({
|
||||
...ctx,
|
||||
...opts,
|
||||
currentEngine: {
|
||||
@@ -1872,13 +1964,29 @@ const installInContext: InstallFunction = async (projects, ctx, opts) => {
|
||||
wantedLockfile: result.newLockfile,
|
||||
useLockfile: opts.useLockfile && ctx.wantedLockfileIsModified,
|
||||
hoistWorkspacePackages: opts.hoistWorkspacePackages,
|
||||
})
|
||||
}))
|
||||
return {
|
||||
...result,
|
||||
stats,
|
||||
ignoredBuilds,
|
||||
}
|
||||
}
|
||||
// Isolated `nodeLinker` (the default) with a non-frozen install:
|
||||
// pacquet doesn't ship a resolver yet, so split the install in two —
|
||||
// ask `_installInContext` for a `lockfileOnly` resolve pass (writes
|
||||
// `pnpm-lock.yaml`), then hand the freshly-written lockfile to
|
||||
// pacquet for the fetch / import / link / build phases. The frozen
|
||||
// branch is handled earlier in `tryFrozenInstall`; the hoisted
|
||||
// branch above already runs the same resolve-then-materialize
|
||||
// sequence (it had to even before pacquet existed). When no pacquet
|
||||
// is configured this falls through to the full single-pass install.
|
||||
if (opts.runPacquet != null && !opts.lockfileOnly) {
|
||||
const result = await _installInContext(projects, ctx, { ...opts, lockfileOnly: true })
|
||||
// The resolve pass above emitted a `pnpm:progress status:resolved`
|
||||
// per package; ask pacquet to drop its own duplicates.
|
||||
await opts.runPacquet({ filterResolvedProgress: true })
|
||||
return result
|
||||
}
|
||||
return await _installInContext(projects, ctx, opts)
|
||||
} catch (error: any) { // eslint-disable-line
|
||||
if (
|
||||
@@ -2370,14 +2478,21 @@ async function installFromPnpmRegistry (
|
||||
skipped: new Set<DepPath>(),
|
||||
wantedLockfile: lockfile,
|
||||
}
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const { ignoredBuilds, stats } = await headlessInstall(headlessOpts as any)
|
||||
const { ignoredBuilds, stats } = await materializeOrDelegate(
|
||||
opts,
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
() => headlessInstall(headlessOpts as any)
|
||||
)
|
||||
|
||||
return {
|
||||
updatedCatalogs: undefined,
|
||||
updatedManifest: manifest,
|
||||
ignoredBuilds,
|
||||
stats,
|
||||
// Pacquet doesn't surface a structured stats return; default to
|
||||
// zeros so the agent-path's non-optional `stats` slot is filled.
|
||||
// The reporter still renders accurate counts from pacquet's
|
||||
// `pnpm:stats` log events.
|
||||
stats: stats ?? { added: 0, removed: 0, linkedToRoot: 0 },
|
||||
lockfile,
|
||||
// Server-side resolution (pnpm agent) enforces `minimumReleaseAge`
|
||||
// itself — the agent picks only mature versions and the lockfile
|
||||
|
||||
@@ -5,6 +5,12 @@ import { fileURLToPath } from "node:url";
|
||||
import * as fs from "node:fs";
|
||||
|
||||
const BIN_NAME = "pacquet";
|
||||
// The wrapper package is published under two names: the original
|
||||
// `pacquet` (kept for back-compat) and `@pnpm/pacquet` (the official
|
||||
// pnpm-scoped alias). Both ship the same JS shim and depend on the
|
||||
// same `@pacquet/<platform>-<arch>` binary sub-packages.
|
||||
const SCOPED_ALIAS_NAME = "@pnpm/pacquet";
|
||||
const SCOPED_ALIAS_DIR = "pnpm-pacquet";
|
||||
const PACQUET_ROOT = resolve(fileURLToPath(import.meta.url), "../..");
|
||||
const PACKAGES_ROOT = resolve(PACQUET_ROOT, "..");
|
||||
const REPO_ROOT = resolve(PACKAGES_ROOT, "../..");
|
||||
@@ -83,6 +89,33 @@ function writeManifest() {
|
||||
fs.writeFileSync(manifestPath, content);
|
||||
}
|
||||
|
||||
function generateScopedAliasPackage() {
|
||||
const aliasRoot = resolve(PACKAGES_ROOT, SCOPED_ALIAS_DIR);
|
||||
fs.rmSync(aliasRoot, { recursive: true, force: true });
|
||||
fs.mkdirSync(resolve(aliasRoot, "bin"), { recursive: true });
|
||||
|
||||
// Mirror the JS shim 1:1. Copying instead of symlinking keeps the
|
||||
// tarball self-contained for `pnpm publish`.
|
||||
fs.copyFileSync(
|
||||
resolve(PACQUET_ROOT, "bin", BIN_NAME),
|
||||
resolve(aliasRoot, "bin", BIN_NAME),
|
||||
);
|
||||
|
||||
// The pacquet manifest at this point already carries the version and
|
||||
// optionalDependencies that `writeManifest` patched in. Reuse it and
|
||||
// swap only the package name + repo.directory pointer.
|
||||
const baseManifest = JSON.parse(fs.readFileSync(MANIFEST_PATH, "utf-8"));
|
||||
const aliasManifest = {
|
||||
...baseManifest,
|
||||
name: SCOPED_ALIAS_NAME,
|
||||
repository: { ...baseManifest.repository, directory: `pacquet/npm/${SCOPED_ALIAS_DIR}` },
|
||||
};
|
||||
fs.writeFileSync(
|
||||
resolve(aliasRoot, "package.json"),
|
||||
JSON.stringify(aliasManifest),
|
||||
);
|
||||
}
|
||||
|
||||
const PLATFORMS = ["win32", "darwin", "linux"];
|
||||
const ARCHITECTURES = ["x64", "arm64"];
|
||||
|
||||
@@ -92,4 +125,8 @@ for (const platform of PLATFORMS) {
|
||||
}
|
||||
}
|
||||
|
||||
writeManifest();
|
||||
writeManifest();
|
||||
// Must run after `writeManifest`: the alias mirrors the patched
|
||||
// pacquet manifest (version + optionalDependencies), so reading it
|
||||
// before the patch would copy stale values.
|
||||
generateScopedAliasPackage();
|
||||
146
pnpm/test/install/pacquet.ts
Normal file
146
pnpm/test/install/pacquet.ts
Normal file
@@ -0,0 +1,146 @@
|
||||
import fs from 'node:fs'
|
||||
|
||||
import { expect, test } from '@jest/globals'
|
||||
import { prepare } from '@pnpm/prepare'
|
||||
import { writeYamlFileSync } from 'write-yaml-file'
|
||||
|
||||
import { execPnpm, execPnpmSync } from '../utils/index.js'
|
||||
|
||||
// `pacquet` is fetched from the real npm registry — registry-mock doesn't
|
||||
// carry it (or its platform-specific binary sub-packages). Pinned to a
|
||||
// version known to ship the `configDependencies` integration surface this
|
||||
// PR depends on; tests are gated on the public registry being reachable.
|
||||
const PUBLIC_REGISTRY = '--config.registry=https://registry.npmjs.org/'
|
||||
const PACQUET_VERSION = '0.2.2-9'
|
||||
|
||||
// Each test runs two or three installs against the public registry; raise
|
||||
// the per-test timeout above jest's 5s default to allow for cold caches.
|
||||
const TIMEOUT = 5 * 60 * 1000
|
||||
|
||||
interface PrepareOpts {
|
||||
manifest?: { dependencies?: Record<string, string>, devDependencies?: Record<string, string> }
|
||||
/** Which `configDependencies` slot declares pacquet. Both work. */
|
||||
pacquetConfigDepName?: 'pacquet' | '@pnpm/pacquet'
|
||||
}
|
||||
|
||||
/** Set up a temp project + workspace yaml + initial install. */
|
||||
async function prepareWithPacquet (opts: PrepareOpts = {}): Promise<void> {
|
||||
prepare(opts.manifest ?? {})
|
||||
writeYamlFileSync('pnpm-workspace.yaml', {
|
||||
configDependencies: {
|
||||
[opts.pacquetConfigDepName ?? 'pacquet']: PACQUET_VERSION,
|
||||
},
|
||||
})
|
||||
// Initial install populates pnpm-lock.yaml plus configDependencies
|
||||
// (pacquet + platform binary). This first install goes through the JS
|
||||
// path because `node_modules/.pnpm-config/pacquet` isn't on disk yet
|
||||
// for the delegate to use.
|
||||
await execPnpm([PUBLIC_REGISTRY, 'install'])
|
||||
}
|
||||
|
||||
test('pnpm install --frozen-lockfile delegates to pacquet when declared in configDependencies', async () => {
|
||||
await prepareWithPacquet({ manifest: { dependencies: { 'is-positive': '3.1.0' } } })
|
||||
expect(fs.existsSync('node_modules/.pnpm-config/pacquet/bin/pacquet')).toBe(true)
|
||||
expect(fs.existsSync('node_modules/is-positive/package.json')).toBe(true)
|
||||
|
||||
// Wipe `node_modules` while leaving lockfiles intact — the CI-style
|
||||
// starting state of a checked-out repo with no installed modules.
|
||||
await fs.promises.rm('node_modules', { recursive: true, force: true })
|
||||
|
||||
const { stdout, status } = execPnpmSync(
|
||||
[PUBLIC_REGISTRY, 'install', '--frozen-lockfile'],
|
||||
{ env: { pnpm_config_silent: 'false' }, stdio: 'pipe', expectSuccess: true }
|
||||
)
|
||||
expect(status).toBe(0)
|
||||
expect(stdout.toString()).toContain('Using pacquet for this install')
|
||||
expect(fs.existsSync('node_modules/is-positive/package.json')).toBe(true)
|
||||
}, TIMEOUT)
|
||||
|
||||
test('bare `pnpm install` (no --frozen-lockfile) delegates the materialization to pacquet', async () => {
|
||||
await prepareWithPacquet({ manifest: { dependencies: { 'is-positive': '3.1.0' } } })
|
||||
await fs.promises.rm('node_modules', { recursive: true, force: true })
|
||||
|
||||
// No `--frozen-lockfile` flag. The expected path is: pnpm runs a
|
||||
// lockfileOnly resolve pass (the lockfile is already up-to-date so
|
||||
// it's a no-op write), then hands fetch / import / link off to
|
||||
// pacquet via the default-isolated-linker branch.
|
||||
const { stdout, status } = execPnpmSync(
|
||||
[PUBLIC_REGISTRY, 'install'],
|
||||
{ env: { pnpm_config_silent: 'false' }, stdio: 'pipe', expectSuccess: true }
|
||||
)
|
||||
expect(status).toBe(0)
|
||||
expect(stdout.toString()).toContain('Using pacquet for this install')
|
||||
expect(fs.existsSync('node_modules/is-positive/package.json')).toBe(true)
|
||||
}, TIMEOUT)
|
||||
|
||||
// Skipped until pacquet writes a `.modules.yaml` whose `publicHoistPattern`
|
||||
// matches what pnpm computes on a follow-up command. Today pacquet's
|
||||
// materialization writes a different value, so the second pnpm command
|
||||
// in the same project fails with
|
||||
// `ERR_PNPM_PUBLIC_HOIST_PATTERN_DIFF`. Bare `--frozen-lockfile` /
|
||||
// `install` tests escape this by wiping `node_modules` between
|
||||
// invocations; `pnpm add` and `pnpm update` can't, because they need
|
||||
// the prior install's state to do anything meaningful. Tracked as a
|
||||
// pacquet-side parity gap; re-enable once pacquet's `.modules.yaml`
|
||||
// shape matches pnpm's.
|
||||
test.skip('`pnpm add <pkg>` resolves the new dep with pnpm and materializes with pacquet', async () => {
|
||||
await prepareWithPacquet()
|
||||
|
||||
const { stdout, status } = execPnpmSync(
|
||||
[PUBLIC_REGISTRY, 'add', 'is-positive@3.1.0'],
|
||||
{ env: { pnpm_config_silent: 'false' }, stdio: 'pipe', expectSuccess: true }
|
||||
)
|
||||
expect(status).toBe(0)
|
||||
// Pnpm's resolver handles the new package; pacquet performs the
|
||||
// fetch / import. The delegation log fires on the materialization
|
||||
// pass that follows the resolve.
|
||||
expect(stdout.toString()).toContain('Using pacquet for this install')
|
||||
expect(fs.existsSync('node_modules/is-positive/package.json')).toBe(true)
|
||||
// Package.json must record the new dep so subsequent installs see it.
|
||||
const manifest = JSON.parse(await fs.promises.readFile('package.json', 'utf8'))
|
||||
expect(manifest.dependencies?.['is-positive']).toBeDefined()
|
||||
}, TIMEOUT)
|
||||
|
||||
// Same skip reason as the `pnpm add` test above:
|
||||
// `ERR_PNPM_PUBLIC_HOIST_PATTERN_DIFF` on the second invocation.
|
||||
test.skip('`pnpm update <pkg>` resolves a new version with pnpm and materializes with pacquet', async () => {
|
||||
// Start pinned to an older minor so `update` has something to do.
|
||||
await prepareWithPacquet({ manifest: { dependencies: { 'is-positive': '^3.0.0' } } })
|
||||
const oldVersion = JSON.parse(
|
||||
await fs.promises.readFile('node_modules/is-positive/package.json', 'utf8')
|
||||
).version as string
|
||||
|
||||
const { stdout, status } = execPnpmSync(
|
||||
[PUBLIC_REGISTRY, 'update', 'is-positive', '--latest'],
|
||||
{ env: { pnpm_config_silent: 'false' }, stdio: 'pipe', expectSuccess: true }
|
||||
)
|
||||
expect(status).toBe(0)
|
||||
expect(stdout.toString()).toContain('Using pacquet for this install')
|
||||
const newVersion = JSON.parse(
|
||||
await fs.promises.readFile('node_modules/is-positive/package.json', 'utf8')
|
||||
).version as string
|
||||
// is-positive@4 is the current latest and is a major bump from the 3.x
|
||||
// line; `update --latest` should move past the original `^3.0.0` pin.
|
||||
expect(newVersion).not.toBe(oldVersion)
|
||||
}, TIMEOUT)
|
||||
|
||||
// Skipped until pacquet ships a release built with the updated
|
||||
// `generate-packages.mjs` (this PR's change) so the `@pnpm/pacquet`
|
||||
// scoped alias actually exists on npm. The pinned `0.2.2-9` doesn't
|
||||
// publish that mirror yet. Re-enable when the next pacquet release
|
||||
// ships under both names.
|
||||
test.skip('the `@pnpm/pacquet` scoped alias is recognized in configDependencies', async () => {
|
||||
await prepareWithPacquet({
|
||||
manifest: { dependencies: { 'is-positive': '3.1.0' } },
|
||||
pacquetConfigDepName: '@pnpm/pacquet',
|
||||
})
|
||||
await fs.promises.rm('node_modules', { recursive: true, force: true })
|
||||
|
||||
const { stdout, status } = execPnpmSync(
|
||||
[PUBLIC_REGISTRY, 'install', '--frozen-lockfile'],
|
||||
{ env: { pnpm_config_silent: 'false' }, stdio: 'pipe', expectSuccess: true }
|
||||
)
|
||||
expect(status).toBe(0)
|
||||
expect(stdout.toString()).toContain('Using pacquet for this install')
|
||||
expect(fs.existsSync('node_modules/is-positive/package.json')).toBe(true)
|
||||
}, TIMEOUT)
|
||||
Reference in New Issue
Block a user