mirror of
https://github.com/pnpm/pnpm.git
synced 2026-06-29 18:35:18 -04:00
* fix(pacquet): materialize cached side-effects on warm reinstall When `sideEffectsCache` is on (the default), pacquet's build phase skips a rebuild whenever the store holds a side-effects-cache entry for the package's engine cache key (the `is_built` gate). But the warm-link phase only materialized the pristine tarball files — it never applied the cached `added` / `deleted` overlay. So a package approved via `allowBuilds` whose lifecycle script produces files outside its tarball (e.g. `bun`'s postinstall downloading its binary) was left in its pre-build state on the second frozen install: the first install seeded the cache, and the next one skipped the rebuild without re-linking the build output. pnpm doesn't have this bug — `getFlatMap` applies the side-effects diff to the file map at import time, so the built output is linked in one pass. Mirror that: at the `is_built` gate, re-import the package directory from the cached overlay (`base - deleted + added`, already resolved to CAS paths) via `import_indexed_dir(force, keep_modules_dir)`, preserving the slot's nested `node_modules/`. Guarded to project-local layouts — under the global virtual store the slot persists in the store with its output already on disk and is read-only under `frozen_store`. Adds an end-to-end regression (install -> frozen -> wipe node_modules -> frozen, asserting the postinstall output survives) and strengthens the `using_side_effects_cache_skips_rebuild` unit test to assert the overlay is materialized. Fixes the report at https://github.com/pnpm/pnpm/issues/12042#issuecomment-4682732058 * fix(pacquet): fall back to rebuild when cached side-effects can't be materialized A side-effects cache hit now degrades to a cache miss when the overlay can't be imported, instead of aborting the install. Side-effects `added` blobs aren't re-verified, so a CAS blob deleted out from under the store surfaces as an import error at the `is_built` gate; the gate runs before the optional-dependency swallow, so a corrupt entry on an optional dep would otherwise fail the whole install. On failure we now log a warning and fall through to the normal build path, which re-runs the script over the pristine files and re-seeds the cache. Addresses review feedback on the PR. * fix(pacquet): reject unsafe paths in side-effects cache overlay The side-effects overlay is joined onto the package directory and written during the warm-reinstall import, so an `added` key from a poisoned or corrupted store index (store integrity is explicitly not a tamper boundary) could escape the slot via a `..` or absolute path. Validate every `added` filename in `build_side_effects_maps` and drop the whole cache-key entry on any unsafe path — absolute, containing `..`, or using a `\` separator — so the importer falls back to a rebuild, matching the existing malformed-digest handling and the tarball extractor's path-traversal guard. Addresses review feedback on the PR. * fix(pacquet): use a raw string for the backslash overlay-path test case Satisfies the dylint `perfectionist::prefer-raw-string` lint that the pre-push hook and CI enforce. * fix(pacquet): skip redundant side-effects re-import when slot already matches On a side-effects cache hit the gate force-reimported the overlay into the package slot every time, re-staging every built package on full reinstalls whose slots are already materialized (the warm link short-circuits on an intact slot, so a previous install's build output is still present). Skip the stage-and-swap when the slot already matches the overlay exactly — every overlay file present and no extra files outside `node_modules/`. A freshly linked, pristine-only slot is missing the `added` files, so the match fails and the full re-import still runs; checking both directions keeps the skip sound across overlay changes that drop a file. Also simplifies the e2e test's frozen-install helper to reuse the outer workspace and process-global mock registry instead of spinning up an unused second `CommandTempCwd`/registry per call. Addresses review feedback on the PR. * fix(pacquet): always re-import side-effects overlay on a cache hit Drops the slot-match fast path: skipping the re-import by filename set is unsound (a slot left from a different cache key can carry the same filenames with stale bytes, leaving the package in the wrong built state), and a content-equality check would read every file — as expensive as the hardlink-based re-import it would avoid. So the import always runs on a cache hit (non-GVS), which is the sound behavior. The redundant re-import this had aimed to avoid is bounded: a true no-op `--frozen-lockfile` exits via the optimistic repeat-install fast path before the build phase, and the import is hardlink-based. A cheap *and* sound skip needs a link-phase "this slot was re-linked pristine-only this install" signal, left as a follow-up. Addresses review feedback on the PR. * fix(pacquet): don't finish install with a slot left broken by a failed materialization The materialize-failure fall-through assumed the pristine slot was still intact. That holds for the common failure (a missing CAS blob fails while staging, before the existing dir is touched), but a stage-and-swap that fails mid-replace can leave the slot without its manifest. Rebuilding against that runs scripts on an incomplete dir — or skips them when the manifest is gone — and the install finishes with a broken package. Now the fall-through to rebuild only happens when the slot's manifest survived the failed materialization. When it didn't, an optional dependency is skipped (as for any optional build failure) and a non-optional one surfaces a hard error instead of completing silently. Addresses review feedback on the PR.
86 lines
3.8 KiB
Rust
86 lines
3.8 KiB
Rust
#![cfg(unix)]
|
|
|
|
use assert_cmd::prelude::*;
|
|
use command_extra::CommandExtra;
|
|
use pacquet_testing_utils::bin::{AddMockedRegistry, CommandTempCwd};
|
|
use std::{fs, path::Path, process::Command};
|
|
|
|
/// Regression for <https://github.com/pnpm/pnpm/issues/12042#issuecomment-4682732058>:
|
|
/// a package approved via `allowBuilds` whose lifecycle script produces
|
|
/// files not in its tarball (e.g. `bun`'s postinstall downloading a
|
|
/// binary) loses that output on a warm frozen reinstall.
|
|
///
|
|
/// `sideEffectsCache` is on by default, so the first build seeds the
|
|
/// cache. On the second frozen install the `is_built` gate skips the
|
|
/// rebuild — the cached build output must still be materialized into the
|
|
/// freshly linked slot, mirroring pnpm's `getFlatMap` applying the
|
|
/// side-effects diff at import time. Without that, the slot is left with
|
|
/// only the pristine tarball files and the package is broken at runtime.
|
|
#[test]
|
|
fn side_effects_materialized_on_warm_frozen_reinstall() {
|
|
let CommandTempCwd { pacquet, root, workspace, npmrc_info, .. } =
|
|
CommandTempCwd::init().add_mocked_registry();
|
|
let AddMockedRegistry { mock_instance, .. } = npmrc_info;
|
|
|
|
// `allowBuilds` in `pnpm-workspace.yaml`, exactly like the report.
|
|
let yaml_path = workspace.join("pnpm-workspace.yaml");
|
|
let mut yaml = fs::read_to_string(&yaml_path).expect("read pnpm-workspace.yaml");
|
|
if !yaml.ends_with('\n') {
|
|
yaml.push('\n');
|
|
}
|
|
yaml.push_str("allowBuilds:\n '@pnpm.e2e/pre-and-postinstall-scripts-example': true\n");
|
|
fs::write(&yaml_path, yaml).expect("write pnpm-workspace.yaml");
|
|
|
|
let manifest_path = workspace.join("package.json");
|
|
let package_json = serde_json::json!({
|
|
"dependencies": {
|
|
"@pnpm.e2e/pre-and-postinstall-scripts-example": "1.0.0",
|
|
},
|
|
});
|
|
fs::write(&manifest_path, package_json.to_string()).expect("write package.json");
|
|
|
|
// `generated-by-postinstall.js` is written by the package's
|
|
// postinstall and is not part of its tarball, so it only exists if
|
|
// the build ran or its cached output was materialized.
|
|
let postinstall_artifact = workspace.join(
|
|
"node_modules/.pnpm/@pnpm.e2e+pre-and-postinstall-scripts-example@1.0.0\
|
|
/node_modules/@pnpm.e2e/pre-and-postinstall-scripts-example/generated-by-postinstall.js",
|
|
);
|
|
|
|
eprintln!("First install (non-frozen, writes lockfile + populates store)...");
|
|
pacquet.with_arg("install").assert().success();
|
|
|
|
eprintln!("Wiping node_modules before the first frozen install...");
|
|
fs::remove_dir_all(workspace.join("node_modules")).expect("remove node_modules");
|
|
|
|
eprintln!("Frozen install (builds, writes the side-effects cache)...");
|
|
run_frozen_install(&workspace);
|
|
assert!(postinstall_artifact.exists(), "postinstall must run on the first frozen install");
|
|
|
|
eprintln!("Wiping node_modules (keep store + lockfile, like a fresh CI checkout)...");
|
|
fs::remove_dir_all(workspace.join("node_modules")).expect("remove node_modules");
|
|
|
|
eprintln!("Frozen reinstall (warm store, hits the is_built gate)...");
|
|
run_frozen_install(&workspace);
|
|
assert!(
|
|
postinstall_artifact.exists(),
|
|
"the cached postinstall output must be materialized after a warm frozen reinstall",
|
|
);
|
|
|
|
drop((root, mock_instance));
|
|
}
|
|
|
|
/// A fresh `pacquet install --frozen-lockfile` against an existing
|
|
/// workspace. The registry config lives in the workspace's `.npmrc` /
|
|
/// `pnpm-workspace.yaml` and the mock registry is a process-global
|
|
/// singleton kept alive by the caller, so this only needs its own
|
|
/// command — no extra `CommandTempCwd` / registry.
|
|
fn run_frozen_install(workspace: &Path) {
|
|
Command::cargo_bin("pacquet")
|
|
.expect("find the pacquet binary")
|
|
.with_current_dir(workspace)
|
|
.with_args(["install", "--frozen-lockfile"])
|
|
.assert()
|
|
.success();
|
|
}
|