Files
pnpm/pacquet/crates/cli/tests/side_effects_cache.rs
Zoltan Kochan 214bd1a0e9 fix(pacquet): materialize cached side-effects on warm reinstall (#12428)
* fix(pacquet): materialize cached side-effects on warm reinstall

When `sideEffectsCache` is on (the default), pacquet's build phase skips
a rebuild whenever the store holds a side-effects-cache entry for the
package's engine cache key (the `is_built` gate). But the warm-link phase
only materialized the pristine tarball files — it never applied the
cached `added` / `deleted` overlay. So a package approved via
`allowBuilds` whose lifecycle script produces files outside its tarball
(e.g. `bun`'s postinstall downloading its binary) was left in its
pre-build state on the second frozen install: the first install seeded
the cache, and the next one skipped the rebuild without re-linking the
build output.

pnpm doesn't have this bug — `getFlatMap` applies the side-effects diff
to the file map at import time, so the built output is linked in one
pass. Mirror that: at the `is_built` gate, re-import the package
directory from the cached overlay (`base - deleted + added`, already
resolved to CAS paths) via `import_indexed_dir(force, keep_modules_dir)`,
preserving the slot's nested `node_modules/`. Guarded to project-local
layouts — under the global virtual store the slot persists in the store
with its output already on disk and is read-only under `frozen_store`.

Adds an end-to-end regression (install -> frozen -> wipe node_modules ->
frozen, asserting the postinstall output survives) and strengthens the
`using_side_effects_cache_skips_rebuild` unit test to assert the overlay
is materialized.

Fixes the report at
https://github.com/pnpm/pnpm/issues/12042#issuecomment-4682732058

* fix(pacquet): fall back to rebuild when cached side-effects can't be materialized

A side-effects cache hit now degrades to a cache miss when the overlay
can't be imported, instead of aborting the install. Side-effects `added`
blobs aren't re-verified, so a CAS blob deleted out from under the store
surfaces as an import error at the `is_built` gate; the gate runs before
the optional-dependency swallow, so a corrupt entry on an optional dep
would otherwise fail the whole install. On failure we now log a warning
and fall through to the normal build path, which re-runs the script over
the pristine files and re-seeds the cache.

Addresses review feedback on the PR.

* fix(pacquet): reject unsafe paths in side-effects cache overlay

The side-effects overlay is joined onto the package directory and written
during the warm-reinstall import, so an `added` key from a poisoned or
corrupted store index (store integrity is explicitly not a tamper
boundary) could escape the slot via a `..` or absolute path. Validate
every `added` filename in `build_side_effects_maps` and drop the whole
cache-key entry on any unsafe path — absolute, containing `..`, or using
a `\` separator — so the importer falls back to a rebuild, matching the
existing malformed-digest handling and the tarball extractor's
path-traversal guard.

Addresses review feedback on the PR.

* fix(pacquet): use a raw string for the backslash overlay-path test case

Satisfies the dylint `perfectionist::prefer-raw-string` lint that the
pre-push hook and CI enforce.

* fix(pacquet): skip redundant side-effects re-import when slot already matches

On a side-effects cache hit the gate force-reimported the overlay into the
package slot every time, re-staging every built package on full reinstalls
whose slots are already materialized (the warm link short-circuits on an
intact slot, so a previous install's build output is still present). Skip
the stage-and-swap when the slot already matches the overlay exactly —
every overlay file present and no extra files outside `node_modules/`. A
freshly linked, pristine-only slot is missing the `added` files, so the
match fails and the full re-import still runs; checking both directions
keeps the skip sound across overlay changes that drop a file.

Also simplifies the e2e test's frozen-install helper to reuse the outer
workspace and process-global mock registry instead of spinning up an
unused second `CommandTempCwd`/registry per call.

Addresses review feedback on the PR.

* fix(pacquet): always re-import side-effects overlay on a cache hit

Drops the slot-match fast path: skipping the re-import by filename set is
unsound (a slot left from a different cache key can carry the same
filenames with stale bytes, leaving the package in the wrong built state),
and a content-equality check would read every file — as expensive as the
hardlink-based re-import it would avoid. So the import always runs on a
cache hit (non-GVS), which is the sound behavior.

The redundant re-import this had aimed to avoid is bounded: a true no-op
`--frozen-lockfile` exits via the optimistic repeat-install fast path
before the build phase, and the import is hardlink-based. A cheap *and*
sound skip needs a link-phase "this slot was re-linked pristine-only this
install" signal, left as a follow-up.

Addresses review feedback on the PR.

* fix(pacquet): don't finish install with a slot left broken by a failed materialization

The materialize-failure fall-through assumed the pristine slot was still
intact. That holds for the common failure (a missing CAS blob fails while
staging, before the existing dir is touched), but a stage-and-swap that
fails mid-replace can leave the slot without its manifest. Rebuilding
against that runs scripts on an incomplete dir — or skips them when the
manifest is gone — and the install finishes with a broken package.

Now the fall-through to rebuild only happens when the slot's manifest
survived the failed materialization. When it didn't, an optional
dependency is skipped (as for any optional build failure) and a
non-optional one surfaces a hard error instead of completing silently.

Addresses review feedback on the PR.
2026-06-15 23:53:12 +02:00

86 lines
3.8 KiB
Rust

#![cfg(unix)]
use assert_cmd::prelude::*;
use command_extra::CommandExtra;
use pacquet_testing_utils::bin::{AddMockedRegistry, CommandTempCwd};
use std::{fs, path::Path, process::Command};
/// Regression for <https://github.com/pnpm/pnpm/issues/12042#issuecomment-4682732058>:
/// a package approved via `allowBuilds` whose lifecycle script produces
/// files not in its tarball (e.g. `bun`'s postinstall downloading a
/// binary) loses that output on a warm frozen reinstall.
///
/// `sideEffectsCache` is on by default, so the first build seeds the
/// cache. On the second frozen install the `is_built` gate skips the
/// rebuild — the cached build output must still be materialized into the
/// freshly linked slot, mirroring pnpm's `getFlatMap` applying the
/// side-effects diff at import time. Without that, the slot is left with
/// only the pristine tarball files and the package is broken at runtime.
#[test]
fn side_effects_materialized_on_warm_frozen_reinstall() {
let CommandTempCwd { pacquet, root, workspace, npmrc_info, .. } =
CommandTempCwd::init().add_mocked_registry();
let AddMockedRegistry { mock_instance, .. } = npmrc_info;
// `allowBuilds` in `pnpm-workspace.yaml`, exactly like the report.
let yaml_path = workspace.join("pnpm-workspace.yaml");
let mut yaml = fs::read_to_string(&yaml_path).expect("read pnpm-workspace.yaml");
if !yaml.ends_with('\n') {
yaml.push('\n');
}
yaml.push_str("allowBuilds:\n '@pnpm.e2e/pre-and-postinstall-scripts-example': true\n");
fs::write(&yaml_path, yaml).expect("write pnpm-workspace.yaml");
let manifest_path = workspace.join("package.json");
let package_json = serde_json::json!({
"dependencies": {
"@pnpm.e2e/pre-and-postinstall-scripts-example": "1.0.0",
},
});
fs::write(&manifest_path, package_json.to_string()).expect("write package.json");
// `generated-by-postinstall.js` is written by the package's
// postinstall and is not part of its tarball, so it only exists if
// the build ran or its cached output was materialized.
let postinstall_artifact = workspace.join(
"node_modules/.pnpm/@pnpm.e2e+pre-and-postinstall-scripts-example@1.0.0\
/node_modules/@pnpm.e2e/pre-and-postinstall-scripts-example/generated-by-postinstall.js",
);
eprintln!("First install (non-frozen, writes lockfile + populates store)...");
pacquet.with_arg("install").assert().success();
eprintln!("Wiping node_modules before the first frozen install...");
fs::remove_dir_all(workspace.join("node_modules")).expect("remove node_modules");
eprintln!("Frozen install (builds, writes the side-effects cache)...");
run_frozen_install(&workspace);
assert!(postinstall_artifact.exists(), "postinstall must run on the first frozen install");
eprintln!("Wiping node_modules (keep store + lockfile, like a fresh CI checkout)...");
fs::remove_dir_all(workspace.join("node_modules")).expect("remove node_modules");
eprintln!("Frozen reinstall (warm store, hits the is_built gate)...");
run_frozen_install(&workspace);
assert!(
postinstall_artifact.exists(),
"the cached postinstall output must be materialized after a warm frozen reinstall",
);
drop((root, mock_instance));
}
/// A fresh `pacquet install --frozen-lockfile` against an existing
/// workspace. The registry config lives in the workspace's `.npmrc` /
/// `pnpm-workspace.yaml` and the mock registry is a process-global
/// singleton kept alive by the caller, so this only needs its own
/// command — no extra `CommandTempCwd` / registry.
fn run_frozen_install(workspace: &Path) {
Command::cargo_bin("pacquet")
.expect("find the pacquet binary")
.with_current_dir(workspace)
.with_args(["install", "--frozen-lockfile"])
.assert()
.success();
}