Files
pnpm/pnpm11/benchmarks
Zoltan Kochan f361560ab4 perf(pnpr): skip the server exchange when there is nothing to resolve (#13905)
With a configured pnprServer, every install paid a full server exchange
even when the answer was already on disk, making pnpr strictly slower
than a direct install on up-to-date projects. Close that gap in three
layers, each deciding locally that the server has nothing to add:

- Let the pre-runtime "Already up to date" fast path run with a pnpr
  server configured. The check decides purely locally that nothing
  changed since the last install; asking the server cannot change that
  answer. The trust/policy settings guard pnpm already records in the
  workspace state (trustPolicy, trustPolicyExclude,
  trustPolicyIgnoreAfter, minimumReleaseAgeStrict,
  minimumReleaseAgeExclude) is now recorded and compared by pacquet
  too, so a policy change still defeats the fast path.

- Gate the resolve exchange on a local satisfaction check: an
  unfiltered install whose on-disk lockfile still satisfies every
  manifest goes straight to the frozen materialization, exactly like
  the non-pnpr preferFrozenLockfile dispatch.

- Consult the local lockfile-verified.jsonl cache before delegating
  input-lockfile verification to the server, and record server-verified
  and server-resolved lockfiles into that cache (pnpm's
  writeWantedLockfileAndRecordVerified), so repeat verifications of an
  unchanged lockfile stay local.

Add repeat-install scenarios (populated node_modules, hot and cold
cache) to the integrated benchmark, pre-warmed before hyperfine runs
and wired into the pnpr-not-slower-than-direct gate with an absolute
slack for tens-of-milliseconds runs.

Closes pnpm/pnpm#13904
2026-08-14 15:25:16 +02:00
..

pnpm Benchmarks

Compares pnpm install performance between the current branch (HEAD) and main, across the scenarios listed below.

This wrapper builds both pnpm revisions and runs hyperfine through the shared Rust orchestrator at pnpm/tasks/integrated-benchmark, so scenario / fixture / workspace / install-script / report generation stay consistent with the pacquet benchmark.

Prerequisites

  • cargo (install Rust via rustup if you don't have it).
  • hyperfine, pnpm, node, git on $PATH.

Usage

./pnpm11/benchmarks/bench.sh

The script:

  1. Builds the integrated-benchmark binary in release mode.
  2. Clones the current repo into the temp work-env once per revision (HEAD and main) and runs pnpm install && pnpm run compile-only in each to produce pnpm11/pnpm/dist/pnpm.mjs. compile-only skips the update-manifests pass that the root compile script does — it would rewrite tracked files and trigger a second install per revision, neither of which the bench needs.
  3. Runs hyperfine on each scenario with --registry=npm (hits registry.npmjs.org directly, no proxy — same as before).
  4. Writes a per-scenario BENCHMARK_REPORT.md / .json and a consolidated results.md into the temp work-env. The path is printed at the end of the run.
  5. Emits bencher-results.json — a hyperfine-shaped file with one result per scenario (the @HEAD revision only, command renamed to the scenario name) that the Benchmarks GitHub Actions workflow uploads to Bencher for continuous tracking.

Scenarios

Slugs follow <linker>.<action>.<cache state>.<store state> so the leading segment groups runs by linker mode. Today there are two groups (isolated-linker.* and gvs-linker.*); future scenarios will add hoisted-linker.* and pnp-linker.*.

A "fresh" scenario starts with node_modules wiped. (For fresh-resolve the wipe also keeps the install's up-to-date short-circuit from skipping the measured resolution.) A "repeat-install" scenario starts with node_modules populated and up to date, measuring exactly that short-circuit.

# Slug Lockfile Cache Store Description
0 isolated-linker.repeat-install.hot-cache.hot-store ✔ hot hot Repeat pnpm install in a current tree — the up-to-date short-circuit
1 isolated-linker.fresh-restore.hot-cache.hot-store ✔ frozen hot hot Restore from lockfile with both directories hot (repeat-headless shape)
2 isolated-linker.fresh-add-dep.hot-cache.hot-store ✔ + add dep hot hot pnpm add <dep> against an existing lockfile
3 isolated-linker.fresh-install.hot-cache.hot-store ✗ hot hot Resolve from scratch with both directories hot
4 isolated-linker.fresh-restore.cold-cache.cold-store ✔ frozen cold cold Restore from lockfile with cold disks (typical CI shape)
5 isolated-linker.fresh-install.cold-cache.cold-store ✗ cold cold True cold start — no lockfile, nothing cached
6 isolated-linker.fresh-resolve.hot-cache.offline ✗ (removed per run) hot n/a install --offline --lockfile-only: full re-resolution from the warm on-disk packument mirror — no network, no linking; guards offline/prefer-offline resolution (an online pre-warm pass primes the mirror first)
7 gvs-linker.fresh-restore.hot-cache.hot-store ✔ frozen hot hot + GVS Frozen-lockfile restore with enableGlobalVirtualStore: true, pre-warmed GVS

All scenarios use --ignore-scripts and isolated store/cache directories per revision.

Fixture

The fixture lives at fixture/ — a synthetic package.json with ~80 typical front-end dependencies, plus a committed pnpm-lock.yaml (generated once with pnpm install --lockfile-only). The lockfile is checked in so every CI run starts from the same resolution graph regardless of registry drift.

Configuration

Environment variables read by bench.sh:

  • WARMUP — number of warmup runs before timing (default: 1)
  • RUNS — number of timed runs per benchmark (default: 10)