fd564a43b9
Unify the two install-benchmark stacks (`benchmarks/bench.sh` and `pacquet/tasks/integrated-benchmark/`) into one shared Rust orchestrator. Scenario, fixture, workspace-manifest, install-script, and report generation live in one place so changes propagate to both stacks. Scenario sets per workflow are unchanged: `benchmark.yml` keeps measuring the same 6 scenarios; `pacquet-integrated-benchmark.yml` keeps measuring the same 2. Both verdaccio and live-npm registry modes are preserved; neither is removed in favor of the other. All six scenarios accept both `pacquet@<rev>` and `pnpm@<rev>` targets.
pnpm Benchmarks
Compares pnpm install performance between the current branch (HEAD) and
main, across the six scenarios listed below.
This wrapper builds both pnpm revisions and runs hyperfine through the
shared Rust orchestrator at
pacquet/tasks/integrated-benchmark,
so scenario / fixture / workspace / install-script / report generation
stay consistent with the pacquet benchmark.
Prerequisites
cargo(install Rust via rustup if you don't have it).hyperfine,pnpm,node,giton$PATH.
Usage
./benchmarks/bench.sh
The script:
- Builds the
integrated-benchmarkbinary in release mode. - Clones the current repo into the temp work-env once per revision
(
HEADandmain) and runspnpm install && pnpm run compile-onlyin each to producepnpm/dist/pnpm.mjs.compile-onlyskips theupdate-manifestspass that the rootcompilescript does — it would rewrite tracked files and trigger a second install per revision, neither of which the bench needs. - Runs hyperfine on each scenario with
--registry=npm(hitsregistry.npmjs.orgdirectly, no proxy — same as before). - Writes a per-scenario
BENCHMARK_REPORT.md/.jsonand a consolidatedresults.mdinto the temp work-env. The path is printed at the end of the run.
Scenarios
| # | Name (orchestrator) | Lockfile | Store + Cache | Description |
|---|---|---|---|---|
| 1 | frozen-lockfile-hot-cache |
✔ frozen | warm | Headless install (repeat install with warm store) |
| 2 | peek |
✔ + add dep | warm | Re-resolution: add a new dep to an existing lockfile |
| 3 | full-resolution |
✗ | warm | Resolve everything from scratch with warm cache |
| 4 | frozen-lockfile |
✔ frozen | cold | Typical CI install — fetch all packages with lockfile |
| 5 | clean-install |
✗ | cold | True cold start — nothing cached |
| 6 | gvs-warm |
✔ frozen | warm + GVS | GVS warm reinstall (frozen lockfile, warm global virtual store) |
All scenarios use --ignore-scripts and isolated store/cache directories per revision.
Fixture
The fixture lives at fixture/ — a synthetic
package.json with ~80 typical front-end dependencies, plus a committed
pnpm-lock.yaml (generated once with pnpm install --lockfile-only).
The lockfile is checked in so every CI run starts from the same
resolution graph regardless of registry drift.
Configuration
Environment variables read by bench.sh:
WARMUP— number of warmup runs before timing (default: 1)RUNS— number of timed runs per benchmark (default: 10)