mirror/pnpm
1
0
Fork 0
mirror of https://github.com/pnpm/pnpm.git synced 2026-01-23 14:28:50 -05:00
Code Issues Packages Projects Releases Wiki Activity

fix(cafs): avoid race on extracting files from tarballs (#8703)

Browse Source 
* fix(cafs): avoid race on extracting files from tarballs

On running addFilesFromTarball with many workers in the pool there were sometimes
race conditions on moving files having same content.
It resulted in ERR_PNPM_ENOENT and ERR_PNPM_TARBALL_EXTRACT exceptions during install.
The issue was introduced in v8.7.0 and generated many defect reports.

close #7676, potentially #7607, #7697 and others

* refactor: add comment, release notes and decrease length of temp file

---------

Co-authored-by: VladimirNT1 <vladimir.nikolaich@gmail.com>
Co-authored-by: Zoltan Kochan <z@kochan.io>
This commit is contained in:
Vladimir Turov
2024-10-28 05:04:49 +03:00
committed by GitHub
parent 5e98817eaf
commit a1f4df297d
2 changed files with 19 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.changeset/soft-kids-build.md Normal file
View File

@@ -0,0 +1,6 @@
---
"@pnpm/store.cafs": patch
"pnpm": patch
---
Fixed a race condition in temporary file creation in the store by including worker thread ID in filename. Previously, multiple worker threads could attempt to use the same temporary file. Temporary files now include both process ID and thread ID for uniqueness [#8703](https://github.com/pnpm/pnpm/pull/8703).

17
store/cafs/src/writeBufferToCafs.ts
View File

@@ -1,5 +1,6 @@
import fs from 'fs'
import path from 'path'
import workerThreads from 'worker_threads'
import util from 'util'
import renameOverwrite from 'rename-overwrite'
import type ssri from 'ssri'
@@ -75,13 +76,21 @@ export function optimisticRenameOverwrite (temp: string, fileDest: string): void
}
/**
* The process ID is appended to the file name to create a temporary file.
* If the process fails, on rerun the new temp file may get a filename the got left over.
* That is fine, the file will be overridden.
* Creates a unique temporary file path by appending both process ID and worker thread ID
* to the original filename.
*
* The process ID prevents conflicts between different processes, while the worker thread ID
* prevents race conditions between threads in the same process.
*
* If a process fails, its temporary file may remain. When the process is rerun, it will
* safely overwrite any existing temporary file with the same name.
*
* @param file - The original file path
* @returns A temporary file path in the format: {basename}{pid}{threadId}
*/
export function pathTemp (file: string): string {
const basename = removeSuffix(path.basename(file))
return path.join(path.dirname(file), `${basename}${process.pid}`)
return path.join(path.dirname(file), `${basename}${process.pid}${workerThreads.threadId}`)
}
function removeSuffix (filePath: string): string {