mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-01 12:41:16 -04:00
fix: injectWorkspacePackages crashes — lean resolution defense-in-depth + lifecycle re-import (#11662)
Fixes two **independent** crashes hitting `pnpm install --frozen-lockfile` on workspaces with `injectWorkspacePackages: true` (or `dependenciesMeta.*.injected`), surfaced via `turbo prune --docker` pipelines. ## Bug 1 — peer-variant snapshot missing `resolution` (lean, defense-in-depth) A peer-variant injected workspace snapshot (`@scope/pkg@file:packages/pkg(peerA@1)(peerB@2)`) inherits its `resolution` from the base `packages:` entry (`@scope/pkg@file:packages/pkg`). When a tool prunes the lockfile and drops that base entry, readers that deref `pkgSnapshot.resolution` crash with the cryptic: ``` Cannot use 'in' operator to search for 'directory' in undefined ``` **The root cause is upstream of pnpm**: the pruner (e.g. `turbo prune`) emits an internally inconsistent lockfile. Fixed at the source in **vercel/turborepo#12825** (retain the base entry for peer-variant injected deps; minimal repro in **vercel/turborepo#12824**) — empirically verified to produce a correct pruned lockfile for a real multi-service workspace. **pnpm side (this PR): one lean normalization at the read layer** — in `convertToLockfileObject`, where base→variant inheritance already happens via `Object.assign`. When the base entry is absent, reconstruct the directory resolution from the `file:` depPath. This is *reconstruction, not guessing*: for a workspace `file:` dep the directory **is** the depPath suffix — exactly what pnpm's own writer emits. It is **defense-in-depth, not load-bearing**: with a well-formed lockfile (turbo#12825 or any correct input) the branch never fires. Because the normalization sits at the single shared read layer, it also covers the sibling `Cannot use 'in' operator … 'integrity' in undefined` on the `pnpm deploy` path (same `resolution === undefined` root, different deref site). Per review feedback: the earlier per-reader `inheritOrSynthesizeResolution` helper across 5 call sites is **removed**; normalization lives in exactly one place (`convertToLockfileObject`), and the readers are back to `main`. ## Bug 2 — lifecycle re-import wipes `.bin/<tool>` (pure pnpm; the substantive fix) `runLifecycleHooksConcurrently` re-imports an injected workspace package into its targets after `prepare`/`postinstall`. The 2022 `scanDir`-into-`filesMap` workaround (#4299) fed target-internal paths to `importPackage`; once #11088 made `importIndexedDir`'s `makeEmptyDir` fast path the default, that path wipes the target's `node_modules` before copying, so the re-import dies with `ERR_PNPM_ENOENT` on `node_modules/.bin/<tool>`. Fix: drop the `scanDir` workaround and pass `keepModulesDir: true` so `importIndexedDir` skips the destructive fast path and preserves the target's existing `node_modules` (bin symlinks + transitive deps) via its staging/move path. Stays on `storeController.importPackage`, so source files keep their **hardlinks** (no copy-loop regression). Net reduction vs `main`: the `scanDir` helper and the `node:fs` / `FilesMap` imports are removed. ## Tests - The `deps-restorer` regression fixture `peer-variant-missing-resolution` **omits the base `packages:` entry**, so it encodes the actual pruned shape and reproduces the crash on `main`: reverting the `convertToLockfileObject` change yields `resolution: undefined` for the peer-variant (→ the `lockfileToDepGraph` crash); with this PR it is reconstructed as `{ type: 'directory', directory: … }`. - A `lockfile.fs` unit test pins the heuristic boundary: a directory resolution is synthesized for a pruned `file:` peer-variant but **never** for a `file:` tarball. - A `deps-installer` regression test covers the Bug 2 re-import (injected dep with a `prepare` script + a bin-having dependency). ## Validation End-to-end on a real `injectWorkspacePackages` monorepo (`turbo prune --docker` → `pnpm install --frozen-lockfile`), on services that crash on **both** bugs with stock pnpm: - pnpm with both fixes: the crashing services build. - **vercel/turborepo#12825 + pnpm with only Bug 2** (Bug 1 fully reverted): the crashing services still **build** → confirms Bug 1 here is genuine defense-in-depth and turbo#12825 owns the root cause. - Bug 2 reproduces on stock pnpm regardless of turbo (it is purely pnpm's importer fast-path). Pairs with **vercel/turborepo#12825** (Bug 1 root cause; minimal repro **vercel/turborepo#12824**). Tracks #11663. --------- Co-authored-by: Zoltan Kochan <z@kochan.io> Co-authored-by: Eyalm321 <eyal@sunsationsusa.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: UApply Developer <developer@uapply.ai>
This commit is contained in:
10
.changeset/injectworkspacepackages-prune-crashes.md
Normal file
10
.changeset/injectworkspacepackages-prune-crashes.md
Normal file
@@ -0,0 +1,10 @@
|
||||
---
|
||||
"@pnpm/lockfile.fs": patch
|
||||
"@pnpm/exec.lifecycle": patch
|
||||
"pnpm": patch
|
||||
---
|
||||
|
||||
Fix two crashes with `injectWorkspacePackages: true` when the lockfile has been pruned (e.g. by `turbo prune --docker`):
|
||||
|
||||
- `Cannot use 'in' operator to search for 'directory' in undefined`: a peer-dependency-variant injected snapshot inherits its `resolution` from the base `packages:` entry; when a pruner drops that base entry the readers crash. `convertToLockfileObject` now reconstructs the directory resolution from the `file:` depPath at load time — a single normalization point, so every reader sees a fully-formed snapshot.
|
||||
- `ERR_PNPM_ENOENT` on `node_modules/.bin/<tool>`: after `prepare`/`postinstall`, `runLifecycleHooksConcurrently` re-imported each injected workspace package; the `scanDir`-into-`filesMap` workaround fed target-internal paths to the importer, which the `makeEmptyDir` fast path (#11088) then wiped. Drop the workaround and pass `keepModulesDir: true` so the importer preserves the target's existing `node_modules` (bin links + transitive deps) and source files keep their hardlinks.
|
||||
@@ -1,10 +1,8 @@
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
|
||||
import { linkBins } from '@pnpm/bins.linker'
|
||||
import { fetchFromDir } from '@pnpm/fetching.directory-fetcher'
|
||||
import { logger } from '@pnpm/logger'
|
||||
import type { FilesMap } from '@pnpm/store.cafs-types'
|
||||
import type { StoreController } from '@pnpm/store.controller-types'
|
||||
import type { ProjectManifest, ProjectRootDir } from '@pnpm/types'
|
||||
import { runGroups } from 'run-groups'
|
||||
@@ -73,45 +71,30 @@ export async function runLifecycleHooksConcurrently (
|
||||
}
|
||||
}
|
||||
if (targetDirs == null || targetDirs.length === 0 || !isBuilt) return
|
||||
// Re-import only the freshly-built source — fetchFromDir already
|
||||
// excludes the source's node_modules/. `keepModulesDir: true` makes
|
||||
// importIndexedDir skip the destructive makeEmptyDir fast path
|
||||
// (#11088) and preserve the target's existing node_modules (bin
|
||||
// symlinks + transitive deps from the initial install) via its
|
||||
// staging/move path. Replaces the old scanDir-into-filesMap
|
||||
// workaround (#4299) that the fast path then wiped, causing ENOENT
|
||||
// on .bin/<tool>. Stays on storeController.importPackage so source
|
||||
// files keep their hardlinks (no copy-loop).
|
||||
const filesResponse = await fetchFromDir(rootDir, { resolveSymlinks: opts.resolveSymlinksInInjectedDirs })
|
||||
await Promise.all(
|
||||
targetDirs.map(async (targetDir) => {
|
||||
const targetModulesDir = path.join(targetDir, 'node_modules')
|
||||
const newFilesMap: FilesMap = new Map(filesResponse.filesMap)
|
||||
if (fs.existsSync(targetModulesDir)) {
|
||||
// If the target directory contains a node_modules directory
|
||||
// (it may happen when the hoisted node linker is used)
|
||||
// then we need to preserve this node_modules.
|
||||
// So we scan this node_modules directory and pass it as part of the new package.
|
||||
await scanDir('node_modules', targetModulesDir, targetModulesDir, newFilesMap)
|
||||
}
|
||||
return opts.storeController.importPackage(targetDir, {
|
||||
targetDirs.map(async (targetDir) =>
|
||||
opts.storeController.importPackage(targetDir, {
|
||||
filesResponse: {
|
||||
resolvedFrom: 'local-dir',
|
||||
...filesResponse,
|
||||
filesMap: newFilesMap,
|
||||
},
|
||||
force: false,
|
||||
keepModulesDir: true,
|
||||
})
|
||||
})
|
||||
)
|
||||
)
|
||||
}
|
||||
)
|
||||
})
|
||||
await runGroups(childConcurrency, groups)
|
||||
}
|
||||
|
||||
async function scanDir (prefix: string, rootDir: string, currentDir: string, index: FilesMap): Promise<void> {
|
||||
const files = await fs.promises.readdir(currentDir)
|
||||
await Promise.all(files.map(async (file) => {
|
||||
const fullPath = path.join(currentDir, file)
|
||||
const stat = await fs.promises.stat(fullPath)
|
||||
if (stat.isDirectory()) {
|
||||
return scanDir(prefix, rootDir, fullPath, index)
|
||||
}
|
||||
if (stat.isFile()) {
|
||||
const relativePath = path.relative(rootDir, fullPath)
|
||||
index.set(path.join(prefix, relativePath), fullPath)
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -2013,3 +2013,91 @@ test('injected local packages are deduped', async () => {
|
||||
expect(modulesState?.injectedDeps?.['project-1'][0]).toContain(`node_modules${path.sep}.pnpm`)
|
||||
}
|
||||
})
|
||||
|
||||
// Regression test: when an injected workspace package has a prepare script
|
||||
// AND any dep with a bin, the re-import after the script crashed with
|
||||
// `ENOENT: copyfile '...node_modules/.bin/<tool>'`. `runLifecycleHooksConcurrently`
|
||||
// scanned the existing injected node_modules and added absolute paths under
|
||||
// it to the filesMap; the importer's fast path then wiped the target before
|
||||
// reading from those paths. Fixed by keeping storeController.importPackage
|
||||
// but passing keepModulesDir: true, so importIndexedDir skips the
|
||||
// destructive makeEmptyDir fast path (#11088) and preserves the target's
|
||||
// existing node_modules (bin links + transitive deps) instead.
|
||||
test('inject local package with prepare script + bin-having dep does not crash on re-import', async () => {
|
||||
const project1Manifest = {
|
||||
name: 'project-1',
|
||||
version: '1.0.0',
|
||||
dependencies: {
|
||||
// hello-world-js-bin declares "bin": "./index.js", so pnpm creates
|
||||
// a node_modules/.bin/hello-world-js-bin symlink in the injected
|
||||
// copy. Before the fix that symlink triggered the crash.
|
||||
'@pnpm.e2e/hello-world-js-bin': '1.0.0',
|
||||
},
|
||||
scripts: {
|
||||
// `prepare` (not `prepublishOnly`) is the script that pnpm runs during
|
||||
// install — it's the lifecycle stage that triggered the original
|
||||
// re-import crash on bin-having injected deps.
|
||||
prepare: 'node -e "require(\'node:fs\').writeFileSync(\'built.txt\',\'ok\')"',
|
||||
},
|
||||
}
|
||||
const project2Manifest = {
|
||||
name: 'project-2',
|
||||
version: '1.0.0',
|
||||
dependencies: {
|
||||
'project-1': 'workspace:1.0.0',
|
||||
},
|
||||
dependenciesMeta: {
|
||||
'project-1': {
|
||||
injected: true,
|
||||
},
|
||||
},
|
||||
}
|
||||
const projects = preparePackages([
|
||||
{
|
||||
location: 'project-1',
|
||||
package: project1Manifest,
|
||||
},
|
||||
{
|
||||
location: 'project-2',
|
||||
package: project2Manifest,
|
||||
},
|
||||
])
|
||||
|
||||
const importers: MutatedProject[] = [
|
||||
{
|
||||
mutation: 'install',
|
||||
rootDir: path.resolve('project-1') as ProjectRootDir,
|
||||
},
|
||||
{
|
||||
mutation: 'install',
|
||||
rootDir: path.resolve('project-2') as ProjectRootDir,
|
||||
},
|
||||
]
|
||||
const allProjects = [
|
||||
{
|
||||
buildIndex: 0,
|
||||
manifest: project1Manifest,
|
||||
rootDir: path.resolve('project-1') as ProjectRootDir,
|
||||
},
|
||||
{
|
||||
buildIndex: 0,
|
||||
manifest: project2Manifest,
|
||||
rootDir: path.resolve('project-2') as ProjectRootDir,
|
||||
},
|
||||
]
|
||||
|
||||
// The install would throw with ERR_PNPM_ENOENT on `.bin/hello-world-js-bin`
|
||||
// before the fix.
|
||||
await mutateModules(importers, testDefaults({
|
||||
autoInstallPeers: false,
|
||||
allProjects,
|
||||
}))
|
||||
|
||||
// prepare ran in project-1.
|
||||
expect(fs.existsSync(path.resolve('project-1/built.txt'))).toBeTruthy()
|
||||
// The re-import succeeded — project-2's injected copy has both the bin
|
||||
// symlink from the original injection AND the built.txt from prepare.
|
||||
expect(() => projects['project-2'].has('project-1')).not.toThrow()
|
||||
expect(fs.existsSync(path.resolve('project-2/node_modules/project-1/built.txt'))).toBeTruthy()
|
||||
expect(fs.existsSync(path.resolve('project-2/node_modules/project-1/node_modules/.bin/hello-world-js-bin'))).toBeTruthy()
|
||||
})
|
||||
|
||||
8
installing/deps-restorer/test/fixtures/peer-variant-missing-resolution/package.json
vendored
Normal file
8
installing/deps-restorer/test/fixtures/peer-variant-missing-resolution/package.json
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"name": "peer-variant-missing-resolution",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
"pkg-a": "workspace:*"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"name": "peer",
|
||||
"version": "1.0.0",
|
||||
"private": true
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"name": "pkg-a",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"peerDependencies": {
|
||||
"peer": "1.0.0"
|
||||
}
|
||||
}
|
||||
51
installing/deps-restorer/test/fixtures/peer-variant-missing-resolution/pnpm-lock.yaml
generated
vendored
Normal file
51
installing/deps-restorer/test/fixtures/peer-variant-missing-resolution/pnpm-lock.yaml
generated
vendored
Normal file
@@ -0,0 +1,51 @@
|
||||
lockfileVersion: '9.0'
|
||||
|
||||
settings:
|
||||
autoInstallPeers: true
|
||||
excludeLinksFromLockfile: false
|
||||
injectWorkspacePackages: true
|
||||
|
||||
importers:
|
||||
|
||||
.:
|
||||
dependencies:
|
||||
pkg-a:
|
||||
specifier: workspace:*
|
||||
version: 'file:packages/pkg-a(peer@1.0.0)'
|
||||
|
||||
packages/peer: {}
|
||||
|
||||
packages/pkg-a:
|
||||
peerDependencies:
|
||||
peer:
|
||||
specifier: 1.0.0
|
||||
version: 1.0.0
|
||||
|
||||
packages:
|
||||
|
||||
# Peer-variant entry whose base `pkg-a@file:packages/pkg-a` packages entry
|
||||
# was dropped by a pruner (e.g. `turbo prune --docker`). With the base gone
|
||||
# `convertToLockfileObject` cannot inherit `resolution`; pre-fix this
|
||||
# crashed `lockfileToDepGraph` with
|
||||
# `TypeError: Cannot use 'in' operator to search for 'directory' in undefined`.
|
||||
'pkg-a@file:packages/pkg-a(peer@1.0.0)':
|
||||
dependencies:
|
||||
peer: 1.0.0
|
||||
|
||||
# Peer dep resolved to the workspace `packages/peer/` directory. This
|
||||
# entry is required for `filterLockfileByImportersAndEngine` to walk
|
||||
# the graph past the variant entry — otherwise it throws
|
||||
# `LockfileMissingDependencyError` once the graph-builder no longer
|
||||
# crashes early on the missing `resolution`.
|
||||
'peer@1.0.0':
|
||||
resolution: { directory: packages/peer, type: directory }
|
||||
|
||||
snapshots:
|
||||
|
||||
'pkg-a@file:packages/pkg-a': {}
|
||||
|
||||
'pkg-a@file:packages/pkg-a(peer@1.0.0)':
|
||||
dependencies:
|
||||
peer: 1.0.0
|
||||
|
||||
'peer@1.0.0': {}
|
||||
3
installing/deps-restorer/test/fixtures/peer-variant-missing-resolution/pnpm-workspace.yaml
vendored
Normal file
3
installing/deps-restorer/test/fixtures/peer-variant-missing-resolution/pnpm-workspace.yaml
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
packages:
|
||||
- "packages/*"
|
||||
injectWorkspacePackages: true
|
||||
@@ -919,3 +919,25 @@ test('installing a package deeply installs all required dependencies', async ()
|
||||
expect(projectAssertion.requireModule('is-positive')).toBeTruthy()
|
||||
}
|
||||
})
|
||||
|
||||
// Regression test: `lockfileToDepGraph` crashed with
|
||||
// `TypeError: Cannot use 'in' operator to search for 'directory' in undefined`
|
||||
// when a peer-dep variant snapshot in the lockfile omitted its `resolution`
|
||||
// field. The variant intentionally inherits resolution from the base entry,
|
||||
// so this shape is valid pnpm output but the graph builder accessed
|
||||
// `pkgSnapshot.resolution` without guarding for undefined.
|
||||
test('headlessInstall: peer-variant snapshot without `resolution` does not crash', async () => {
|
||||
const workspaceFixture = f.prepare('peer-variant-missing-resolution')
|
||||
const projects = [
|
||||
workspaceFixture,
|
||||
path.join(workspaceFixture, 'packages', 'pkg-a'),
|
||||
path.join(workspaceFixture, 'packages', 'peer'),
|
||||
]
|
||||
|
||||
// The bug surfaces as a rejection from `headlessInstall`, so awaiting it
|
||||
// is enough — jest fails the test on an unhandled rejection.
|
||||
await headlessInstall(await testDefaults({
|
||||
lockfileDir: workspaceFixture,
|
||||
projects,
|
||||
}))
|
||||
})
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { LOCKFILE_VERSION } from '@pnpm/constants'
|
||||
import { refToRelative, removeSuffix } from '@pnpm/deps.path'
|
||||
import { parse, refToRelative, removeSuffix } from '@pnpm/deps.path'
|
||||
import type {
|
||||
LockfileFile,
|
||||
LockfileFileProjectResolvedDependencies,
|
||||
@@ -138,13 +138,31 @@ function pruneTimeInLockfile (time: Record<string, string>, importers: Record<st
|
||||
return pickBy((_, depPath) => rootDepPaths.has(depPath), time)
|
||||
}
|
||||
|
||||
// Mirrors `isFilename` in `resolving/local-resolver/src/parseBareSpecifier.ts`
|
||||
// so the directory-vs-tarball boundary applied at lockfile load time
|
||||
// matches the resolver's at resolve time.
|
||||
const LOCAL_TARBALL_RE = /\.(?:tgz|tar\.gz|tar)$/i
|
||||
|
||||
export function convertToLockfileObject (lockfile: LockfileFile): LockfileObject {
|
||||
const { importers, ...rest } = lockfile
|
||||
|
||||
const packages: PackageSnapshots = {}
|
||||
for (const [depPath, pkg] of Object.entries(lockfile.snapshots ?? {})) {
|
||||
const pkgId = removeSuffix(depPath)
|
||||
packages[depPath as DepPath] = Object.assign(pkg, lockfile.packages?.[pkgId])
|
||||
const snapshot = Object.assign(pkg, lockfile.packages?.[pkgId])
|
||||
// Defense-in-depth for pruned lockfiles (older `turbo prune --docker`,
|
||||
// pre vercel/turborepo#12825): a peer-variant injected workspace
|
||||
// snapshot whose base `packages:` entry was dropped now has a null
|
||||
// `resolution`. Reconstruct it from the `file:` depPath — same value
|
||||
// pnpm's writer emits — so every downstream reader sees a complete
|
||||
// snapshot without per-reader guards.
|
||||
if (snapshot.resolution == null) {
|
||||
const ref = parse(depPath).nonSemverVersion
|
||||
if (ref != null && ref.startsWith('file:') && !LOCAL_TARBALL_RE.test(ref)) {
|
||||
snapshot.resolution = { directory: ref.slice('file:'.length), type: 'directory' }
|
||||
}
|
||||
}
|
||||
packages[depPath as DepPath] = snapshot
|
||||
enrichGitHostedFlag(packages[depPath as DepPath]?.resolution as TarballResolution | undefined)
|
||||
}
|
||||
return {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { expect, test } from '@jest/globals'
|
||||
import type { DepPath } from '@pnpm/types'
|
||||
|
||||
import { convertToLockfileFile, convertToLockfileObject } from '../lib/lockfileFormatConverters.js'
|
||||
|
||||
@@ -93,6 +94,36 @@ test('convertToLockfileFile()', () => {
|
||||
expect(convertToLockfileObject(lockfileV6)).toEqual(lockfileV5)
|
||||
})
|
||||
|
||||
test('convertToLockfileObject() reconstructs a dropped directory resolution for a pruned file: peer-variant, but never for a file: tarball', () => {
|
||||
// Simulates a pruned lockfile (e.g. after `turbo prune --docker`): the
|
||||
// base `pkg@file:...` packages entry that carried `resolution` is gone,
|
||||
// only the peer-variant snapshot remains.
|
||||
const prunedLockfileV6 = {
|
||||
lockfileVersion: '9.0',
|
||||
importers: {},
|
||||
snapshots: {
|
||||
'dir@file:packages/dir(peer@1.0.0)': {},
|
||||
'tar@file:vendor/tar-1.0.0.tgz(peer@1.0.0)': {},
|
||||
// Uppercase tarball extensions must be treated as tarballs too — the
|
||||
// resolver in resolving/local-resolver/src/parseBareSpecifier.ts
|
||||
// matches /\.(?:tgz|tar.gz|tar)$/i, so the boundary applied here at
|
||||
// load time has to be case-insensitive in lockstep.
|
||||
'upper@file:vendor/upper-1.0.0.TGZ(peer@1.0.0)': {},
|
||||
'mixed@file:vendor/mixed-1.0.0.Tar.Gz(peer@1.0.0)': {},
|
||||
},
|
||||
}
|
||||
const lockfile = convertToLockfileObject(prunedLockfileV6)
|
||||
// Local-directory `file:` ref → directory resolution losslessly reconstructed.
|
||||
expect(lockfile.packages?.['dir@file:packages/dir(peer@1.0.0)' as DepPath]?.resolution).toEqual({
|
||||
directory: 'packages/dir',
|
||||
type: 'directory',
|
||||
})
|
||||
// `file:` tarball ref → must NOT be turned into a directory resolution.
|
||||
expect(lockfile.packages?.['tar@file:vendor/tar-1.0.0.tgz(peer@1.0.0)' as DepPath]?.resolution).toBeUndefined()
|
||||
expect(lockfile.packages?.['upper@file:vendor/upper-1.0.0.TGZ(peer@1.0.0)' as DepPath]?.resolution).toBeUndefined()
|
||||
expect(lockfile.packages?.['mixed@file:vendor/mixed-1.0.0.Tar.Gz(peer@1.0.0)' as DepPath]?.resolution).toBeUndefined()
|
||||
})
|
||||
|
||||
test('convertToLockfileFile() with lockfile v6', () => {
|
||||
const lockfileV5 = {
|
||||
lockfileVersion: '9.0',
|
||||
|
||||
@@ -125,4 +125,62 @@ impl Lockfile {
|
||||
&& importer.dependencies.as_ref().is_none_or(HashMap::is_empty)
|
||||
})
|
||||
}
|
||||
|
||||
/// Defense-in-depth for pruned lockfiles (older `turbo prune --docker`,
|
||||
/// pre vercel/turborepo#12825): a peer-variant injected workspace
|
||||
/// snapshot whose base `packages:` entry was dropped is missing its
|
||||
/// inherited `resolution`. Reconstruct a directory resolution from
|
||||
/// the `file:` depPath so [`crate::PackageMetadata::resolution`]
|
||||
/// stays non-optional for downstream callers. Mirrors upstream
|
||||
/// `convertToLockfileObject` in `lockfile/fs/src/lockfileFormatConverters.ts`.
|
||||
pub fn reconstruct_missing_directory_resolutions(&mut self) {
|
||||
let Some(snapshots) = self.snapshots.as_ref() else { return };
|
||||
let to_insert: Vec<(PackageKey, DirectoryResolution)> = snapshots
|
||||
.keys()
|
||||
.filter_map(|snapshot_key| {
|
||||
let metadata_key = snapshot_key.without_peer();
|
||||
let packages = self.packages.as_ref();
|
||||
if packages.is_some_and(|p| p.contains_key(&metadata_key)) {
|
||||
return None;
|
||||
}
|
||||
let VersionPart::File(path) = metadata_key.suffix.version() else { return None };
|
||||
if is_local_tarball_path(path) {
|
||||
return None;
|
||||
}
|
||||
let directory = path.clone();
|
||||
Some((metadata_key, DirectoryResolution { directory }))
|
||||
})
|
||||
.collect();
|
||||
if to_insert.is_empty() {
|
||||
return;
|
||||
}
|
||||
let packages = self.packages.get_or_insert_with(HashMap::new);
|
||||
for (key, directory_resolution) in to_insert {
|
||||
packages.entry(key).or_insert_with(|| PackageMetadata {
|
||||
resolution: LockfileResolution::Directory(directory_resolution),
|
||||
engines: None,
|
||||
cpu: None,
|
||||
os: None,
|
||||
libc: None,
|
||||
deprecated: None,
|
||||
has_bin: None,
|
||||
prepare: None,
|
||||
bundled_dependencies: None,
|
||||
peer_dependencies: None,
|
||||
peer_dependencies_meta: None,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Mirrors `isFilename` (`/\.(?:tgz|tar\.gz|tar)$/i`) in
|
||||
/// `resolving/local-resolver/src/parseBareSpecifier.ts` so the directory-vs-
|
||||
/// tarball boundary applied here matches the resolver's at resolve time.
|
||||
fn is_local_tarball_path(path: &str) -> bool {
|
||||
let lower = path.as_bytes();
|
||||
let ends_with_ci = |suffix: &str| {
|
||||
let bytes = suffix.as_bytes();
|
||||
lower.len() >= bytes.len() && lower[lower.len() - bytes.len()..].eq_ignore_ascii_case(bytes)
|
||||
};
|
||||
ends_with_ci(".tgz") || ends_with_ci(".tar.gz") || ends_with_ci(".tar")
|
||||
}
|
||||
|
||||
@@ -68,7 +68,12 @@ impl Lockfile {
|
||||
if main.trim().is_empty() {
|
||||
return Ok(None);
|
||||
}
|
||||
serde_saphyr::from_str(main).map(Some).map_err(LoadLockfileError::ParseYaml)
|
||||
serde_saphyr::from_str::<Self>(main)
|
||||
.map(|mut lockfile| {
|
||||
lockfile.reconstruct_missing_directory_resolutions();
|
||||
Some(lockfile)
|
||||
})
|
||||
.map_err(LoadLockfileError::ParseYaml)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
use crate::Lockfile;
|
||||
use crate::{DirectoryResolution, Lockfile, LockfileResolution, PackageKey};
|
||||
use pretty_assertions::assert_eq;
|
||||
use tempfile::tempdir;
|
||||
use text_block_macros::text_block;
|
||||
@@ -108,3 +108,51 @@ fn env_only_lockfile_loads_as_none() {
|
||||
.expect("env-only lockfile should not error");
|
||||
assert!(result.is_none(), "expected None for env-only lockfile, got: {result:?}");
|
||||
}
|
||||
|
||||
/// Parity port of the TS heuristic-boundary test
|
||||
/// `lockfile/fs/test/lockfileV6Converters.test.ts::convertToLockfileObject()
|
||||
/// reconstructs a dropped directory resolution for a pruned file:
|
||||
/// peer-variant, but never for a file: tarball`.
|
||||
#[test]
|
||||
fn reconstructs_dropped_directory_resolution_for_pruned_file_peer_variant() {
|
||||
let pruned = text_block! {
|
||||
"lockfileVersion: '9.0'"
|
||||
""
|
||||
"importers: {}"
|
||||
""
|
||||
"snapshots:"
|
||||
""
|
||||
" dir@file:packages/dir(peer@1.0.0): {}"
|
||||
" tar@file:vendor/tar-1.0.0.tgz(peer@1.0.0): {}"
|
||||
" upper@file:vendor/upper-1.0.0.TGZ(peer@1.0.0): {}"
|
||||
" mixed@file:vendor/mixed-1.0.0.Tar.Gz(peer@1.0.0): {}"
|
||||
};
|
||||
let tmp = write_lockfile(pruned);
|
||||
let virtual_store_dir = tmp.path().join("node_modules").join(".pacquet");
|
||||
let lockfile = Lockfile::load_current_from_virtual_store_dir(&virtual_store_dir)
|
||||
.expect("load pruned lockfile")
|
||||
.expect("pruned lockfile should be present");
|
||||
|
||||
let packages = lockfile.packages.as_ref().expect("packages synthesized for dir entry");
|
||||
|
||||
let dir_key: PackageKey = "dir@file:packages/dir".parse().expect("parse dir key");
|
||||
let dir_metadata = packages.get(&dir_key).expect("dir entry synthesized");
|
||||
assert_eq!(
|
||||
dir_metadata.resolution,
|
||||
LockfileResolution::Directory(DirectoryResolution {
|
||||
directory: "packages/dir".to_string()
|
||||
}),
|
||||
);
|
||||
|
||||
for tarball_key in [
|
||||
"tar@file:vendor/tar-1.0.0.tgz",
|
||||
"upper@file:vendor/upper-1.0.0.TGZ",
|
||||
"mixed@file:vendor/mixed-1.0.0.Tar.Gz",
|
||||
] {
|
||||
let key: PackageKey = tarball_key.parse().expect("parse tarball key");
|
||||
assert!(
|
||||
packages.get(&key).is_none(),
|
||||
"tarball `{tarball_key}` must not get a synthesized directory resolution",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,6 +3,25 @@ use node_semver::{SemverError, Version};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::{borrow::Cow, fmt, str::FromStr};
|
||||
|
||||
/// Version slot of a [`PkgVerPeer`]: either a semver, or the raw
|
||||
/// path of an injected workspace `file:<path>` dep. Mirrors pnpm's
|
||||
/// `parseDepPath` `nonSemverVersion` arm in `packages/deps.path/src`.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||
pub enum VersionPart {
|
||||
Semver(Version),
|
||||
/// Path portion of a `file:<path>` dep, scheme stripped.
|
||||
File(String),
|
||||
}
|
||||
|
||||
impl fmt::Display for VersionPart {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
VersionPart::Semver(version) => version.fmt(f),
|
||||
VersionPart::File(path) => write!(f, "file:{path}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Suffix type of [`PkgNameVerPeer`](crate::PkgNameVerPeer) and
|
||||
/// type of [`ResolvedDependencySpec::version`](crate::ResolvedDependencySpec::version).
|
||||
///
|
||||
@@ -25,7 +44,7 @@ pub struct PkgVerPeer {
|
||||
/// the round-trip. [`Prefix::None`] for plain semver — the only
|
||||
/// shape pacquet recognises before pnpm v11.
|
||||
prefix: Prefix,
|
||||
version: Version,
|
||||
version: VersionPart,
|
||||
peer: String,
|
||||
}
|
||||
|
||||
@@ -71,10 +90,20 @@ impl PkgVerPeer {
|
||||
/// Get the version part. The `runtime:` prefix (if any) is
|
||||
/// stripped — callers that need to discriminate runtime
|
||||
/// entries should consult [`PkgVerPeer::prefix`] instead.
|
||||
pub fn version(&self) -> &'_ Version {
|
||||
pub fn version(&self) -> &'_ VersionPart {
|
||||
&self.version
|
||||
}
|
||||
|
||||
/// Semver `Version` if this is a [`VersionPart::Semver`], else
|
||||
/// `None`. Use from call sites that need `major` / `minor` /
|
||||
/// `patch` — Display via [`PkgVerPeer::version`] covers the rest.
|
||||
pub fn version_semver(&self) -> Option<&'_ Version> {
|
||||
match &self.version {
|
||||
VersionPart::Semver(version) => Some(version),
|
||||
VersionPart::File(_) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Get the peer part.
|
||||
pub fn peer(&self) -> &'_ str {
|
||||
self.peer.as_str()
|
||||
@@ -90,7 +119,7 @@ impl PkgVerPeer {
|
||||
/// backward-compatible with pre-runtime consumers. New callers
|
||||
/// that need the prefix should access it via
|
||||
/// [`PkgVerPeer::prefix`] before destructuring.
|
||||
pub fn into_tuple(self) -> (Version, String) {
|
||||
pub fn into_tuple(self) -> (VersionPart, String) {
|
||||
let PkgVerPeer { prefix: _, version, peer } = self;
|
||||
(version, peer)
|
||||
}
|
||||
@@ -103,6 +132,23 @@ pub enum ParsePkgVerPeerError {
|
||||
ParseVersionFailure(#[error(source)] SemverError),
|
||||
#[display("Mismatch parenthesis")]
|
||||
MismatchParenthesis,
|
||||
#[display("Empty path after `file:` scheme")]
|
||||
EmptyFilePath,
|
||||
#[display("`runtime:` and `file:` schemes are mutually exclusive")]
|
||||
ConflictingSchemes,
|
||||
}
|
||||
|
||||
fn parse_version_part(input: &str) -> Result<VersionPart, ParsePkgVerPeerError> {
|
||||
if let Some(path) = input.strip_prefix("file:") {
|
||||
if path.is_empty() {
|
||||
return Err(ParsePkgVerPeerError::EmptyFilePath);
|
||||
}
|
||||
return Ok(VersionPart::File(path.to_string()));
|
||||
}
|
||||
input
|
||||
.parse::<Version>()
|
||||
.map(VersionPart::Semver)
|
||||
.map_err(ParsePkgVerPeerError::ParseVersionFailure)
|
||||
}
|
||||
|
||||
impl FromStr for PkgVerPeer {
|
||||
@@ -119,20 +165,25 @@ impl FromStr for PkgVerPeer {
|
||||
(Prefix::None, value)
|
||||
};
|
||||
|
||||
// Pnpm never writes `runtime:file:...`; rejecting it here keeps
|
||||
// [`PkgVerPeer::to_string`] byte-stable (a silent acceptance would
|
||||
// round-trip back as `file:...` with the prefix dropped).
|
||||
if prefix == Prefix::Runtime && body.starts_with("file:") {
|
||||
return Err(ParsePkgVerPeerError::ConflictingSchemes);
|
||||
}
|
||||
|
||||
if !body.ends_with(')') {
|
||||
if body.find(['(', ')']).is_some() {
|
||||
return Err(ParsePkgVerPeerError::MismatchParenthesis);
|
||||
}
|
||||
|
||||
let version = body.parse().map_err(ParsePkgVerPeerError::ParseVersionFailure)?;
|
||||
let version = parse_version_part(body)?;
|
||||
return Ok(PkgVerPeer { prefix, version, peer: String::new() });
|
||||
}
|
||||
|
||||
let opening_parenthesis =
|
||||
body.find('(').ok_or(ParsePkgVerPeerError::MismatchParenthesis)?;
|
||||
let version = body[..opening_parenthesis]
|
||||
.parse()
|
||||
.map_err(ParsePkgVerPeerError::ParseVersionFailure)?;
|
||||
let version = parse_version_part(&body[..opening_parenthesis])?;
|
||||
let peer = body[opening_parenthesis..].to_string();
|
||||
Ok(PkgVerPeer { prefix, version, peer })
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
use super::{ParsePkgVerPeerError, PkgVerPeer, Prefix};
|
||||
use super::{ParsePkgVerPeerError, PkgVerPeer, Prefix, VersionPart};
|
||||
use node_semver::Version;
|
||||
use pretty_assertions::assert_eq;
|
||||
|
||||
@@ -8,7 +8,7 @@ where
|
||||
Peer: Into<String>,
|
||||
{
|
||||
dbg!(&received);
|
||||
let expected_version = expected_version.into();
|
||||
let expected_version = VersionPart::Semver(expected_version.into());
|
||||
let expected_peer = expected_peer.into();
|
||||
assert_eq!((received.version(), received.peer()), (&expected_version, expected_peer.as_str()));
|
||||
assert_eq!(received.into_tuple(), (expected_version, expected_peer));
|
||||
@@ -66,6 +66,8 @@ fn parse_err() {
|
||||
case!("1.21.3(" => "Mismatch parenthesis", ParsePkgVerPeerError::MismatchParenthesis);
|
||||
case!("1.21.3)" => "Mismatch parenthesis", ParsePkgVerPeerError::MismatchParenthesis);
|
||||
case!("a.b.c" => "Failed to parse the version part: Failed to parse version.", ParsePkgVerPeerError::ParseVersionFailure(_));
|
||||
case!("file:" => "Empty path after `file:` scheme", ParsePkgVerPeerError::EmptyFilePath);
|
||||
case!("runtime:file:packages/pkg" => "`runtime:` and `file:` schemes are mutually exclusive", ParsePkgVerPeerError::ConflictingSchemes);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -132,7 +134,7 @@ fn parse_runtime_prefix_round_trips() {
|
||||
let parsed: PkgVerPeer = "runtime:22.0.0".parse().expect("parse runtime version");
|
||||
dbg!(&parsed);
|
||||
assert_eq!(parsed.prefix(), Prefix::Runtime);
|
||||
assert_eq!(parsed.version(), &"22.0.0".parse::<Version>().unwrap());
|
||||
assert_eq!(parsed.version_semver(), Some(&"22.0.0".parse::<Version>().unwrap()));
|
||||
assert_eq!(parsed.peer(), "");
|
||||
assert_eq!(parsed.to_string(), "runtime:22.0.0");
|
||||
}
|
||||
@@ -149,7 +151,7 @@ fn parse_runtime_prefix_with_peer_suffix() {
|
||||
"runtime:22.0.0(node@22.0.0)".parse().expect("parse runtime with peer");
|
||||
dbg!(&parsed);
|
||||
assert_eq!(parsed.prefix(), Prefix::Runtime);
|
||||
assert_eq!(parsed.version(), &"22.0.0".parse::<Version>().unwrap());
|
||||
assert_eq!(parsed.version_semver(), Some(&"22.0.0".parse::<Version>().unwrap()));
|
||||
assert_eq!(parsed.peer(), "(node@22.0.0)");
|
||||
assert_eq!(parsed.to_string(), "runtime:22.0.0(node@22.0.0)");
|
||||
}
|
||||
@@ -173,7 +175,7 @@ fn parse_runtime_substring_in_version_is_not_a_prefix() {
|
||||
// `1.21.3-runtime` is a valid semver pre-release tag.
|
||||
let parsed: PkgVerPeer = "1.21.3-runtime".parse().expect("parse semver pre-release");
|
||||
assert_eq!(parsed.prefix(), Prefix::None);
|
||||
assert_eq!(parsed.version(), &"1.21.3-runtime".parse::<Version>().unwrap());
|
||||
assert_eq!(parsed.version_semver(), Some(&"1.21.3-runtime".parse::<Version>().unwrap()));
|
||||
}
|
||||
|
||||
/// Serde round-trip on a runtime version — pacquet stores
|
||||
@@ -185,7 +187,38 @@ fn parse_runtime_substring_in_version_is_not_a_prefix() {
|
||||
fn serde_round_trip_runtime_prefix() {
|
||||
let parsed: PkgVerPeer = serde_saphyr::from_str("runtime:22.0.0").expect("deserialize runtime");
|
||||
assert_eq!(parsed.prefix(), Prefix::Runtime);
|
||||
assert_eq!(parsed.version(), &"22.0.0".parse::<Version>().unwrap());
|
||||
assert_eq!(parsed.version_semver(), Some(&"22.0.0".parse::<Version>().unwrap()));
|
||||
let serialized = serde_saphyr::to_string(&parsed).expect("serialize").trim().to_string();
|
||||
assert_eq!(serialized, "runtime:22.0.0");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_file_prefix_round_trips() {
|
||||
let parsed: PkgVerPeer = "file:packages/pkg".parse().expect("parse file version");
|
||||
dbg!(&parsed);
|
||||
assert_eq!(parsed.prefix(), Prefix::None);
|
||||
assert_eq!(parsed.version(), &VersionPart::File("packages/pkg".to_string()));
|
||||
assert_eq!(parsed.version_semver(), None);
|
||||
assert_eq!(parsed.peer(), "");
|
||||
assert_eq!(parsed.to_string(), "file:packages/pkg");
|
||||
}
|
||||
|
||||
/// Pruned-lockfile peer-variant snapshot keys take this shape, so the
|
||||
/// `file:` body must compose with the parenthesised peer suffix.
|
||||
#[test]
|
||||
fn parse_file_prefix_with_peer_suffix() {
|
||||
let parsed: PkgVerPeer = "file:packages/pkg(peer@1.0.0)".parse().expect("parse file with peer");
|
||||
assert_eq!(parsed.prefix(), Prefix::None);
|
||||
assert_eq!(parsed.version(), &VersionPart::File("packages/pkg".to_string()));
|
||||
assert_eq!(parsed.peer(), "(peer@1.0.0)");
|
||||
assert_eq!(parsed.to_string(), "file:packages/pkg(peer@1.0.0)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serde_round_trip_file_prefix() {
|
||||
let parsed: PkgVerPeer =
|
||||
serde_saphyr::from_str("file:packages/pkg(peer@1.0.0)").expect("deserialize file");
|
||||
assert_eq!(parsed.version(), &VersionPart::File("packages/pkg".to_string()));
|
||||
let serialized = serde_saphyr::to_string(&parsed).expect("serialize").trim().to_string();
|
||||
assert_eq!(serialized, "file:packages/pkg(peer@1.0.0)");
|
||||
}
|
||||
|
||||
@@ -1300,7 +1300,7 @@ fn find_runtime_node_major(snapshots: Option<&HashMap<PackageKey, SnapshotEntry>
|
||||
// `engine_name` argument is `u32`, matching upstream's
|
||||
// `process.version.split('.')[0].substring(1)`-derived
|
||||
// integer.
|
||||
let major = key.suffix.version().major;
|
||||
let major = key.suffix.version_semver()?.major;
|
||||
return Some(major as u32);
|
||||
}
|
||||
None
|
||||
@@ -1340,7 +1340,7 @@ pub(crate) fn find_own_runtime_node_major(snapshot: &SnapshotEntry) -> Option<u3
|
||||
}
|
||||
// Same cast as `find_runtime_node_major` above; see the
|
||||
// comment there for why `u64 → u32` is lossless in practice.
|
||||
return Some(ver_peer.version().major as u32);
|
||||
return Some(ver_peer.version_semver()?.major as u32);
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user