mirror of
https://github.com/pnpm/pnpm.git
synced 2026-07-20 12:42:38 -04:00
569bdfd158
Resolves #289. ## Summary A cold install against an empty store on macOS dies with `Too many open files (os error 24)` because the spawned process's `RLIMIT_NOFILE` (10240 soft on macOS by default) doesn't survive pacquet's concurrent fan-out: - `post_download_semaphore` allows `num_cpus * 2` in-flight tarballs. - Each tarball's rayon entry-loop opens fds for write + occasional `verify_or_rewrite` byte-compare reads. - Inter-tarball content overlap (LICENSE / empty `.npmignore` / shared assets) sends multiple workers down the `AlreadyExists` → `file_equals_bytes` path simultaneously, each holding a read fd. `ulimit -n` reporting `unlimited` is a zsh display quirk on macOS — `getrlimit(RLIMIT_NOFILE)` for the spawned process is still the 10240 soft default. So a user with no shell tweaks just sees the install fail. ## What this PR does Match pnpm's `graceful-fs` shape: a `retry_on_fd_pressure` helper that wraps the three open sites in `crates/fs/src/ensure_file.rs`: 1. `ensure_file`'s EAFP `O_CREAT|O_EXCL` open 2. `file_equals_bytes`'s `File::open` (the one that surfaced the bug) 3. `write_atomic`'s temp-file `O_CREAT|O_EXCL` open Helper catches `EMFILE` / `ENFILE` (POSIX errno 24 / 23, hardcoded under `cfg(unix)` rather than pulling in `libc` for two stable integers), sleeps with exponential backoff (2 ms doubling, capped at 200 ms), retries up to 32 times before a final attempt propagates the error. Roughly a 5–6 s budget. Steady-state happy-path cost is one branch. Windows: pass-through. Different errno namespace, different fd model, not where users are hitting this. ## Why retries instead of a bounded-concurrency semaphore A semaphore is the more "principled" fix — proactive vs reactive — but threading one through every fs call site is a real structural change. Retry-on-EMFILE matches pnpm's production behaviour, the scope is one file, and it unblocks the immediate failure. A future PR can layer a semaphore on top if convoy-effect sleeps show up in profiling under sustained pressure.