mirror of
https://github.com/pnpm/pnpm.git
synced 2026-07-18 19:52:31 -04:00
feat: bin (#333)
Resolves #330.
Mirrors pnpm v11's `@pnpm/bins.resolver`, `@pnpm/bins.linker`, and `@zkochan/cmd-shim` (resolved at pnpm `4750fd370c` and cmd-shim `0d79ca9534`):
- New `pacquet-cmd-shim` crate parses the `bin` field of a package manifest into commands. Both `bin: string | object` and the `directories.bin` glob fallback are supported, with the same path-traversal and URL-safe-name guards as upstream. Conflict resolution follows `pkgOwnsBin` + lexical compare.
- Shim generator writes all three flavors per bin: Unix `/bin/sh` (byte-compatible with pnpm's `generateShShim`), Windows `.cmd` (CRLF, `%~dp0` paths), and PowerShell `.ps1` (`$basedir`/`$exe` header with Windows/POSIX-pwsh detection). Mirrors `generateCmdShim` and `generatePwshShim` minus the unused `nodePath`/`prependToPath`/`progArgs` branches.
- All filesystem IO behind per-capability DI traits — `FsReadHead`, `FsReadFile`, `FsReadString`, `FsReadDir`, `FsWalkFiles`, `FsCreateDirAll`, `FsWrite`, `FsSetExecutable`, `FsEnsureExecutableBits` — following the pattern documented in [#332](https://github.com/pnpm/pacquet/pull/332#issuecomment-4345054524). `FsWrite` is a thin shim over `std::fs::write` and is **not atomic**; a future atomic-write capability can build on it without renaming. Production callers turbofish the unit-struct provider `::<RealApi>`; tests inject unit-struct fakes to cover IO error paths real fs can't trigger portably. Generic param is named `Api` (not `Fs`) so the same provider can grow non-fs capabilities without a rename.
- New `LinkVirtualStoreBins` step walks each virtual-store slot and links its child packages' bins into the slot's own `node_modules/.bin`, matching `linkBinsOfDependencies` in pnpm's `building/during-install`. Slot iteration, per-slot child reads, and shim writes parallelised on rayon.
- `SymlinkDirectDependencies` and `InstallWithoutLockfile` call the bin linker after the symlink layout is in place. Direct-dep linking and the symlink loop are exposed as free `link_direct_dep_bins` / `symlink_direct_deps_into_node_modules` functions so they're unit-testable without the mock-registry scaffold.
Tests port the relevant `bins/resolver` and `bins/linker` cases from pnpm (`directories.bin` discovery, scoped bin names, dangerous locations, idempotent shim skip, conflict resolution) and add fakes-injected coverage for the IO error variants.
**Not in this PR**: hoisted-bin precedence and lifecycle-script-created bins. Both depend on subsystems pacquet hasn't built yet — hoisting (`hoistPattern` / `publicHoistPattern` resolution + the lift step) and lifecycle scripts (`exec/lifecycle/` runner + `allowBuilds` plumbing). The bin-linking side already does the right thing for both (shims are paths, so they pick up files generated post-extract; the precedence rule is a small addition once hoisting tags candidates as hoisted). Tracked separately in with full porting coordinates.
## Performance
When the bin-linking step landed it added ~7% to the warm-cache Frozen Lockfile install ([benchmark](https://github.com/pnpm/pacquet/pull/333#issuecomment-4344568317)). Closing that gap took a sequence of changes that mirror how pnpm v11 avoids the same costs in its own `linkBinsOfDependencies` ([4750fd370c](https://github.com/pnpm/pnpm/blob/4750fd370c/building/during-install/src/index.ts#L258-L309)):
- **Bundled manifests in `index.db`.** `extract_tarball_entries` captures the parsed `package.json`, picks the subset pnpm caches via a port of `normalizeBundledManifest` (`name`, `version`, `bin`, `directories`, `dependencies`, lifecycle `scripts`, …), and stuffs it into `PackageFilesIndex.manifest`. `prefetch_cas_paths` surfaces it back as a `HashMap<cache_key, Arc<Value>>` so the bin linker doesn't have to re-read `package.json` per child off disk.
- **msgpackr-records encoder learns `serde_json::Value`.** The encoder previously errored with `ManifestNotSupported` if anything ever set `manifest`; now it records-encodes nested JSON objects (slot-allocated per distinct key set) so `useRecords: true` readers see them as JS `Object`s rather than `Map`s — necessary for pnpm's `manifest.bin` / `manifest.directories?.bin` property accesses to resolve.
- **`Arc<Value>` for `PackageBinSource.manifest`.** The lockfile-driven bin-link path produces `slots × children` clones of the parsed manifest (~6k on the integrated-benchmark fixture). Sharing via `Arc` turns each push into a refcount bump.
- **`hasBin` filter via lockfile.** `LinkVirtualStoreBins` now takes the lockfile `packages:` section, pre-computes an `Option<HashSet<PackageKey>>` of `hasBin: true` keys at install start, and skips snapshot children that aren't in the set *before* any path-building or manifest lookup. ~95% of a real lockfile's packages don't declare a bin, so this short-circuits the bulk of the per-slot work. `None` is reserved for the pathological case where the lockfile lacks a `packages:` section (fall back to processing every child); `Some(empty)` is authoritative — lockfile says no package has a bin, every slot short-circuits. Mirrors pnpm's filter at `building/during-install/src/index.ts:283`.
- **Lockfile-driven slot iteration.** `run_lockfile_driven` walks `snapshots` directly and builds slot paths lexically. The previous `run_with_readdir` path enumerated every slot via `Api::read_dir(virtual_store_dir)` and probed each slot's own package directory with a wasted `read_dir` (~1267 wasted `open(O_DIRECTORY) + close` per warm install). The readdir-driven path survives as a fallback for `install_without_lockfile` and retains the existence probe there (small N, no virtual-store invariant from `create_virtual_dir_by_snapshot` to lean on).
- **Parallel prefetch decode.** `StoreIndex::get_many_raw` returns undecoded row bytes; the SQLite mutex releases before `prefetch_cas_paths` fans the msgpackr decode + integrity check across rayon.
Latest CI benchmark on this branch ([job 75462998251](https://github.com/pnpm/pacquet/actions/runs/25701679172/job/75462998251)):
| Scenario | pacquet@HEAD | pacquet@main | Relative |
|---|---|---|---|
| Frozen Lockfile | 2.074s ± 102ms | 2.026s ± 70ms | +2.4% (within noise) |
| Frozen Lockfile (Hot Cache) | **479.6ms ± 38ms** | **535.8ms ± 37ms** | **~10% faster than main** |
The follow-up #417 splits manifests into a dedicated `package_manifests` SQLite table so the unfiltered `package_index` prefetch payload stays at its original size and the manifest fetch becomes a targeted SELECT against just the `hasBin: true` subset.
---------
Co-authored-by: Zoltan Kochan <z@kochan.io>
This commit is contained in:
@@ -7,6 +7,7 @@ extend-exclude = []
|
||||
|
||||
[default.extend-words]
|
||||
cafs = "cafs"
|
||||
pn = "pn"
|
||||
|
||||
[default.extend-identifiers]
|
||||
PnP = "PnP"
|
||||
|
||||
15
pacquet/Cargo.lock
generated
15
pacquet/Cargo.lock
generated
@@ -1880,6 +1880,19 @@ dependencies = [
|
||||
"walkdir",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pacquet-cmd-shim"
|
||||
version = "0.0.1"
|
||||
dependencies = [
|
||||
"derive_more",
|
||||
"miette 7.6.0",
|
||||
"pipe-trait",
|
||||
"rayon",
|
||||
"serde_json",
|
||||
"tempfile",
|
||||
"walkdir",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pacquet-diagnostics"
|
||||
version = "0.0.1"
|
||||
@@ -2027,6 +2040,7 @@ dependencies = [
|
||||
"insta",
|
||||
"miette 7.6.0",
|
||||
"node-semver",
|
||||
"pacquet-cmd-shim",
|
||||
"pacquet-executor",
|
||||
"pacquet-fs",
|
||||
"pacquet-lockfile",
|
||||
@@ -2154,6 +2168,7 @@ dependencies = [
|
||||
"pacquet-store-dir",
|
||||
"pipe-trait",
|
||||
"pretty_assertions",
|
||||
"rayon",
|
||||
"reqwest",
|
||||
"serde",
|
||||
"serde_json",
|
||||
|
||||
@@ -14,6 +14,7 @@ repository = "https://github.com/pnpm/pacquet"
|
||||
[workspace.dependencies]
|
||||
# Crates
|
||||
pacquet-cli = { path = "crates/cli" }
|
||||
pacquet-cmd-shim = { path = "crates/cmd-shim" }
|
||||
pacquet-fs = { path = "crates/fs" }
|
||||
pacquet-registry = { path = "crates/registry" }
|
||||
pacquet-tarball = { path = "crates/tarball" }
|
||||
|
||||
@@ -5,12 +5,15 @@ expression: get_all_folders(&workspace)
|
||||
---
|
||||
[
|
||||
"node_modules",
|
||||
"node_modules/.bin",
|
||||
"node_modules/.pnpm",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent/node_modules/.bin",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0/node_modules/@pnpm.e2e",
|
||||
|
||||
@@ -5,12 +5,15 @@ expression: get_all_folders(&workspace)
|
||||
---
|
||||
[
|
||||
"node_modules",
|
||||
"node_modules/.bin",
|
||||
"node_modules/.pnpm",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent/node_modules/.bin",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0/node_modules/@pnpm.e2e",
|
||||
|
||||
@@ -5,12 +5,15 @@ expression: "(workspace_folders, store_files)"
|
||||
(
|
||||
[
|
||||
"node_modules",
|
||||
"node_modules/.bin",
|
||||
"node_modules/.pnpm",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin-parent@1.0.0/node_modules/@pnpm.e2e/hello-world-js-bin-parent/node_modules/.bin",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0/node_modules",
|
||||
"node_modules/.pnpm/@pnpm.e2e+hello-world-js-bin@1.0.0/node_modules/@pnpm.e2e",
|
||||
|
||||
22
pacquet/crates/cmd-shim/Cargo.toml
Normal file
22
pacquet/crates/cmd-shim/Cargo.toml
Normal file
@@ -0,0 +1,22 @@
|
||||
[package]
|
||||
name = "pacquet-cmd-shim"
|
||||
description = "Bin field parsing and command-shim generation for pacquet, mirroring pnpm's @pnpm/bins.resolver, @pnpm/bins.linker, and @zkochan/cmd-shim"
|
||||
version = "0.0.1"
|
||||
publish = false
|
||||
authors.workspace = true
|
||||
edition.workspace = true
|
||||
homepage.workspace = true
|
||||
keywords.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
|
||||
[dependencies]
|
||||
derive_more = { workspace = true }
|
||||
miette = { workspace = true }
|
||||
pipe-trait = { workspace = true }
|
||||
rayon = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
walkdir = { workspace = true }
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = { workspace = true }
|
||||
198
pacquet/crates/cmd-shim/src/bin_resolver.rs
Normal file
198
pacquet/crates/cmd-shim/src/bin_resolver.rs
Normal file
@@ -0,0 +1,198 @@
|
||||
use crate::{capabilities::FsWalkFiles, path_util::lexical_normalize};
|
||||
use serde_json::Value;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// One bin entry resolved from a package's `package.json`.
|
||||
///
|
||||
/// `name` is the command name as it should appear under `node_modules/.bin/`.
|
||||
/// `path` is the absolute path to the script the shim invokes.
|
||||
///
|
||||
/// Mirrors `Command` in pnpm v11's `@pnpm/bins.resolver`:
|
||||
/// <https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/src/index.ts>.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Command {
|
||||
pub name: String,
|
||||
pub path: PathBuf,
|
||||
}
|
||||
|
||||
/// Bin names that legitimately ship inside a different package than their own
|
||||
/// name. Mirrors `BIN_OWNER_OVERRIDES` in
|
||||
/// <https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/src/index.ts>.
|
||||
///
|
||||
/// Used by [`pkg_owns_bin`] for conflict resolution between two packages
|
||||
/// declaring the same bin name.
|
||||
const BIN_OWNER_OVERRIDES: &[(&str, &[&str])] = &[
|
||||
("npx", &["npm"]),
|
||||
("pn", &["pnpm", "@pnpm/exe"]),
|
||||
("pnpm", &["@pnpm/exe"]),
|
||||
("pnpx", &["pnpm", "@pnpm/exe"]),
|
||||
("pnx", &["pnpm", "@pnpm/exe"]),
|
||||
];
|
||||
|
||||
/// Whether `pkg_name` is a legitimate owner of the given `bin_name`. The
|
||||
/// default rule is "the package named `X` owns the `X` bin"; overrides cover
|
||||
/// cases like `npx` shipping inside `npm`. Mirrors `pkgOwnsBin`.
|
||||
pub fn pkg_owns_bin(bin_name: &str, pkg_name: &str) -> bool {
|
||||
if bin_name == pkg_name {
|
||||
return true;
|
||||
}
|
||||
BIN_OWNER_OVERRIDES
|
||||
.iter()
|
||||
.find(|(name, _)| *name == bin_name)
|
||||
.is_some_and(|(_, owners)| owners.contains(&pkg_name))
|
||||
}
|
||||
|
||||
/// Read every bin declared by `manifest` and return them as [`Command`]s
|
||||
/// rooted at `pkg_path`.
|
||||
///
|
||||
/// Handles the three cases pnpm supports, in order:
|
||||
///
|
||||
/// 1. `bin` as a string. The bin name is the package's own `name` (with any
|
||||
/// `@scope/` prefix stripped). Empty / missing `name` skips the entry, in
|
||||
/// parity with pnpm's `INVALID_PACKAGE_NAME` guard.
|
||||
/// 2. `bin` as an object. Each `(commandName, relativePath)` becomes a
|
||||
/// command, with `@scope/` stripped from the key.
|
||||
/// 3. Fallback: `directories.bin`. Every regular file under the directory
|
||||
/// becomes a command, with the file basename as the bin name. The
|
||||
/// directory itself must resolve under `pkg_path`; a `directories.bin`
|
||||
/// that escapes via `..` returns an empty list.
|
||||
///
|
||||
/// Validation, exactly mirroring pnpm:
|
||||
///
|
||||
/// - Bin name must be URL-safe (`name == encodeURIComponent(name)`) or be the
|
||||
/// single-character `$`. This is the path-traversal guard.
|
||||
/// - Bin path must resolve under `pkg_path`. Prevents a malicious manifest
|
||||
/// from writing shims that exec a sibling package.
|
||||
pub fn get_bins_from_package_manifest<Api: FsWalkFiles>(
|
||||
manifest: &Value,
|
||||
pkg_path: &Path,
|
||||
) -> Vec<Command> {
|
||||
let pkg_name = manifest.get("name").and_then(Value::as_str);
|
||||
if let Some(bin) = manifest.get("bin") {
|
||||
return commands_from_bin(bin, pkg_name, pkg_path);
|
||||
}
|
||||
if let Some(bin_dir_rel) =
|
||||
manifest.get("directories").and_then(|d| d.get("bin")).and_then(Value::as_str)
|
||||
{
|
||||
return commands_from_directories_bin::<Api>(bin_dir_rel, pkg_path);
|
||||
}
|
||||
Vec::new()
|
||||
}
|
||||
|
||||
/// Walk every regular file under `<pkg_path>/<bin_dir_rel>` and emit one
|
||||
/// [`Command`] per file. Mirrors pnpm's `findFiles` + the `directories.bin`
|
||||
/// branch in `getBinsFromPackageManifest`:
|
||||
/// <https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/src/index.ts>.
|
||||
///
|
||||
/// Symlinks are not followed; pnpm uses `tinyglobby` with
|
||||
/// `followSymbolicLinks: false`. Missing directory degrades to an empty
|
||||
/// list (pnpm's `ENOENT` short-circuit).
|
||||
fn commands_from_directories_bin<Api: FsWalkFiles>(
|
||||
bin_dir_rel: &str,
|
||||
pkg_path: &Path,
|
||||
) -> Vec<Command> {
|
||||
let bin_dir = pkg_path.join(bin_dir_rel);
|
||||
if !is_subdir(pkg_path, &bin_dir) {
|
||||
return Vec::new();
|
||||
}
|
||||
// Treat a top-level walk error as "no bins". This matches pnpm's
|
||||
// tinyglobby ENOENT short-circuit. The trait's production impl
|
||||
// already drops per-entry errors inside its iterator, so an `Err`
|
||||
// here only fires when the walker can't even open `bin_dir`.
|
||||
let Ok(paths) = Api::walk_files(&bin_dir) else {
|
||||
return Vec::new();
|
||||
};
|
||||
let mut commands = Vec::new();
|
||||
for path in paths {
|
||||
let Some(name) = path.file_name().and_then(|s| s.to_str()) else {
|
||||
continue;
|
||||
};
|
||||
// Same URL-safe-name guard as the keyed-bin path.
|
||||
if !is_safe_bin_name(name) {
|
||||
continue;
|
||||
}
|
||||
commands.push(Command { name: name.to_string(), path });
|
||||
}
|
||||
commands
|
||||
}
|
||||
|
||||
fn commands_from_bin(bin: &Value, pkg_name: Option<&str>, pkg_path: &Path) -> Vec<Command> {
|
||||
let mut entries: Vec<(String, String)> = Vec::new();
|
||||
match bin {
|
||||
Value::String(rel_path) => {
|
||||
let Some(name) = pkg_name else {
|
||||
return Vec::new();
|
||||
};
|
||||
entries.push((name.to_string(), rel_path.clone()));
|
||||
}
|
||||
Value::Object(map) => {
|
||||
for (key, value) in map {
|
||||
let Some(rel_path) = value.as_str() else {
|
||||
continue;
|
||||
};
|
||||
entries.push((key.clone(), rel_path.to_string()));
|
||||
}
|
||||
}
|
||||
_ => return Vec::new(),
|
||||
}
|
||||
|
||||
let mut commands = Vec::with_capacity(entries.len());
|
||||
for (command_name, bin_relative_path) in entries {
|
||||
// Strip any `@scope/` prefix. Mirrors `commandsFromBin`'s
|
||||
// `commandName[0] === '@'` branch.
|
||||
let bin_name = if command_name.starts_with('@') {
|
||||
match command_name.find('/') {
|
||||
Some(slash) => command_name[slash + 1..].to_string(),
|
||||
None => command_name,
|
||||
}
|
||||
} else {
|
||||
command_name
|
||||
};
|
||||
|
||||
if !is_safe_bin_name(&bin_name) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let bin_path = pkg_path.join(&bin_relative_path);
|
||||
if !is_subdir(pkg_path, &bin_path) {
|
||||
continue;
|
||||
}
|
||||
|
||||
commands.push(Command { name: bin_name, path: bin_path });
|
||||
}
|
||||
commands
|
||||
}
|
||||
|
||||
/// Whether `name` matches the URL-safe character set allowed by JavaScript's
|
||||
/// `encodeURIComponent`, or is the single-character escape hatch `$` pnpm
|
||||
/// permits for awkward but legitimate bin names. Together these are the only
|
||||
/// names pnpm allows the linker to write to disk.
|
||||
///
|
||||
/// `encodeURIComponent` leaves the following bytes unescaped:
|
||||
/// `A-Z a-z 0-9 - _ . ! ~ * ' ( )`.
|
||||
fn is_safe_bin_name(name: &str) -> bool {
|
||||
if name == "$" {
|
||||
return true;
|
||||
}
|
||||
if name.is_empty() {
|
||||
return false;
|
||||
}
|
||||
name.bytes().all(|b| {
|
||||
b.is_ascii_alphanumeric()
|
||||
|| matches!(b, b'-' | b'_' | b'.' | b'!' | b'~' | b'*' | b'\'' | b'(' | b')')
|
||||
})
|
||||
}
|
||||
|
||||
/// Whether `child` resolves to a path under `parent`, after lexically
|
||||
/// normalising `..` segments. Mirrors `isSubdir(pkgPath, binPath)` from pnpm's
|
||||
/// `is-subdir`. We deliberately do not canonicalize via the filesystem.
|
||||
/// The guard runs before the bin file exists at its final location, and
|
||||
/// pnpm's implementation is purely lexical too.
|
||||
fn is_subdir(parent: &Path, child: &Path) -> bool {
|
||||
let parent_norm = lexical_normalize(parent);
|
||||
let child_norm = lexical_normalize(child);
|
||||
child_norm.starts_with(&parent_norm)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
503
pacquet/crates/cmd-shim/src/bin_resolver/tests.rs
Normal file
503
pacquet/crates/cmd-shim/src/bin_resolver/tests.rs
Normal file
@@ -0,0 +1,503 @@
|
||||
use super::{get_bins_from_package_manifest, pkg_owns_bin};
|
||||
use crate::{capabilities::RealApi, path_util::lexical_normalize};
|
||||
use pipe_trait::Pipe;
|
||||
use serde_json::json;
|
||||
use std::{
|
||||
fs::{create_dir_all, write as write_file},
|
||||
path::{Path, PathBuf},
|
||||
};
|
||||
use tempfile::tempdir;
|
||||
|
||||
#[test]
|
||||
fn bin_as_string_uses_package_name() {
|
||||
let manifest = json!({"name": "foo", "bin": "cli.js"});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/pkg/foo"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "foo");
|
||||
assert_eq!(commands[0].path, Path::new("/pkg/foo/cli.js"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bin_as_string_strips_scope() {
|
||||
let manifest = json!({"name": "@scope/foo", "bin": "cli.js"});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/pkg/foo"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "foo");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bin_as_object_keeps_keys_and_strips_scope() {
|
||||
let manifest = json!({
|
||||
"name": "tool",
|
||||
"bin": {
|
||||
"tool": "bin/tool.js",
|
||||
"@scope/extra": "bin/extra.js",
|
||||
},
|
||||
});
|
||||
let mut commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"));
|
||||
commands.sort_by(|a, b| a.name.cmp(&b.name));
|
||||
assert_eq!(commands.len(), 2);
|
||||
assert_eq!(commands[0].name, "extra");
|
||||
assert_eq!(commands[1].name, "tool");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_unsafe_bin_names() {
|
||||
let manifest = json!({
|
||||
"name": "x",
|
||||
"bin": {
|
||||
"good-name": "ok.js",
|
||||
"../bad": "evil.js",
|
||||
"with space": "no.js",
|
||||
"$": "dollar.js",
|
||||
},
|
||||
});
|
||||
let mut names: Vec<_> = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"))
|
||||
.into_iter()
|
||||
.map(|c| c.name)
|
||||
.collect();
|
||||
names.sort();
|
||||
assert_eq!(names, vec!["$".to_string(), "good-name".to_string()]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_path_traversal_outside_package_root() {
|
||||
let manifest = json!({
|
||||
"name": "x",
|
||||
"bin": {"x": "../../../etc/passwd"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/pkg/x"));
|
||||
assert!(commands.is_empty(), "must reject `..`-escapes from pkg root");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_bin_field_returns_empty() {
|
||||
let manifest = json!({"name": "x"});
|
||||
assert!(get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p")).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pkg_owns_bin_default_rule() {
|
||||
assert!(pkg_owns_bin("foo", "foo"));
|
||||
assert!(!pkg_owns_bin("foo", "bar"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pkg_owns_bin_overrides() {
|
||||
assert!(pkg_owns_bin("npx", "npm"));
|
||||
assert!(pkg_owns_bin("pnpx", "pnpm"));
|
||||
assert!(pkg_owns_bin("pnpx", "@pnpm/exe"));
|
||||
assert!(!pkg_owns_bin("npx", "anything-else"));
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's "should allow $ as command name" test
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/index.ts#L21-L36>).
|
||||
/// `$` is the documented escape hatch for awkward bin names; it must
|
||||
/// survive the URL-safe-name guard.
|
||||
#[test]
|
||||
fn dollar_is_allowed_as_command_name() {
|
||||
let manifest = json!({
|
||||
"name": "undollar",
|
||||
"version": "1.0.0",
|
||||
"bin": {"$": "./undollar.js"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "$");
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's "skip dangerous bin names" test
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/index.ts#L74-L94>).
|
||||
/// Path-traversal characters in the *key* must be filtered, not just the
|
||||
/// value.
|
||||
#[test]
|
||||
fn skip_dangerous_bin_names() {
|
||||
let manifest = json!({
|
||||
"name": "foo",
|
||||
"version": "1.0.0",
|
||||
"bin": {
|
||||
"../bad": "./bad",
|
||||
"..\\bad": "./bad",
|
||||
"good": "./good",
|
||||
"~/bad": "./bad",
|
||||
},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "good");
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's "skip dangerous bin locations" test
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/index.ts#L96-L112>).
|
||||
/// `../bad` in the *value* must be filtered by the `is_subdir` check.
|
||||
#[test]
|
||||
fn skip_dangerous_bin_locations() {
|
||||
let manifest = json!({
|
||||
"name": "foo",
|
||||
"version": "1.0.0",
|
||||
"bin": {
|
||||
"bad": "../bad",
|
||||
"good": "./good",
|
||||
},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/pkg/foo"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "good");
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's "get bin from scoped bin name" test
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/index.ts#L114-L130>).
|
||||
/// A scoped key like `@foo/a` collapses to `a` before validation.
|
||||
#[test]
|
||||
fn scoped_bin_name_strips_scope_prefix() {
|
||||
let manifest = json!({
|
||||
"name": "@foo/a",
|
||||
"version": "1.0.0",
|
||||
"bin": {"@foo/a": "./a"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "a");
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's "skip scoped bin names with path traversal" test
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/index.ts#L132-L148>).
|
||||
/// After the scope strip, the resulting bare name still has to pass the
|
||||
/// URL-safe guard. A `@scope/../etc/passwd` collapses to `../etc/passwd`
|
||||
/// which must be rejected.
|
||||
#[test]
|
||||
fn skip_scoped_bin_names_with_path_traversal() {
|
||||
let manifest = json!({
|
||||
"name": "malicious",
|
||||
"version": "1.0.0",
|
||||
"bin": {
|
||||
"@scope/../../.npmrc": "./malicious.js",
|
||||
"@scope/../etc/passwd": "./evil.js",
|
||||
"@scope/legit": "./good.js",
|
||||
},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "legit");
|
||||
}
|
||||
|
||||
/// `bin` as a non-string non-object (number, array, null) is malformed.
|
||||
/// Pacquet's port must return an empty list rather than panic, mirroring
|
||||
/// pnpm's silent fall-through to the empty default.
|
||||
#[test]
|
||||
fn malformed_bin_type_returns_empty() {
|
||||
for shape in [json!(42), json!(["a", "b"]), json!(null), json!(true)] {
|
||||
let manifest = json!({"name": "x", "version": "1.0.0", "bin": shape});
|
||||
assert!(
|
||||
get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p")).is_empty(),
|
||||
"malformed bin shape must be tolerated",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// `bin` as a string requires the manifest to declare `name` (mirrors
|
||||
/// pnpm's `INVALID_PACKAGE_NAME` guard). Pacquet returns an empty list
|
||||
/// rather than throwing because the install pipeline would have already
|
||||
/// surfaced a missing-name failure upstream.
|
||||
#[test]
|
||||
fn bin_string_with_missing_package_name_returns_empty() {
|
||||
let manifest = json!({"bin": "cli.js"});
|
||||
assert!(get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p")).is_empty());
|
||||
}
|
||||
|
||||
/// Object-form bin entries whose values aren't strings (number, null, etc.)
|
||||
/// are silently skipped. Same defensive shape pnpm has for malformed
|
||||
/// manifests.
|
||||
#[test]
|
||||
fn bin_object_with_non_string_value_is_skipped() {
|
||||
let manifest = json!({
|
||||
"name": "tool",
|
||||
"version": "1.0.0",
|
||||
"bin": {
|
||||
"good": "ok.js",
|
||||
"bad-num": 42,
|
||||
"bad-null": null,
|
||||
},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "good");
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's "find all the bin files from a bin directory"
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/index.ts#L37-L57>).
|
||||
/// Every regular file under `directories.bin`, including files in
|
||||
/// subdirectories, becomes a command.
|
||||
#[test]
|
||||
fn directories_bin_walks_files_recursively() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
let bin_dir = pkg.join("bin-dir");
|
||||
create_dir_all(bin_dir.join("subdir")).unwrap();
|
||||
write_file(bin_dir.join("rootBin.js"), "").unwrap();
|
||||
write_file(bin_dir.join("subdir/subBin.js"), "").unwrap();
|
||||
|
||||
let manifest = json!({
|
||||
"name": "bin-dir",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "bin-dir"},
|
||||
});
|
||||
let mut commands = get_bins_from_package_manifest::<RealApi>(&manifest, &pkg);
|
||||
commands.sort_by(|a, b| a.name.cmp(&b.name));
|
||||
assert_eq!(commands.len(), 2);
|
||||
assert_eq!(commands[0].name, "rootBin.js");
|
||||
assert_eq!(commands[0].path, bin_dir.join("rootBin.js"));
|
||||
assert_eq!(commands[1].name, "subBin.js");
|
||||
assert_eq!(commands[1].path, bin_dir.join("subdir/subBin.js"));
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's "skip directories.bin with path traversal"
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/index.ts#L150-L170>).
|
||||
/// `directories.bin: '../sibling'` must be rejected by the `is_subdir`
|
||||
/// guard. The sibling directory is populated with a real file so a
|
||||
/// regression that disables `is_subdir` would observably emit that file
|
||||
/// as a command. Without the file the test would pass for the wrong
|
||||
/// reason (empty dir, hence empty commands).
|
||||
#[test]
|
||||
fn directories_bin_rejects_path_traversal() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
|
||||
// Sibling dir reachable via `../siblings` from the package root,
|
||||
// populated with a "smoking gun" file the resolver would emit if
|
||||
// it failed to reject the traversal.
|
||||
let siblings = tmp.path().join("siblings");
|
||||
create_dir_all(&siblings).unwrap();
|
||||
write_file(siblings.join("smoking-gun"), "").unwrap();
|
||||
|
||||
let manifest = json!({
|
||||
"name": "malicious",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "../siblings"},
|
||||
});
|
||||
assert!(
|
||||
get_bins_from_package_manifest::<RealApi>(&manifest, &pkg).is_empty(),
|
||||
"is_subdir guard must reject `..`-escapes from the pkg root, even \
|
||||
when the resolved directory exists and has files",
|
||||
);
|
||||
}
|
||||
|
||||
/// Mirrors pnpm's `path-traversal.test.ts`
|
||||
/// (<https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/test/path-traversal.test.ts>).
|
||||
/// A `directories.bin` value that resolves outside the package root via
|
||||
/// `..` must yield no commands, even though the target dir exists and
|
||||
/// has files in it.
|
||||
#[test]
|
||||
fn directories_bin_rejects_real_path_traversal() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let secret_dir = tmp.path().join("secret");
|
||||
create_dir_all(&secret_dir).unwrap();
|
||||
write_file(secret_dir.join("secret.sh"), "echo secret").unwrap();
|
||||
|
||||
let pkg = tmp.path().join("pkg");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
|
||||
// From `<tmp>/pkg`, `../secret` reaches `<tmp>/secret`. The
|
||||
// `is_subdir` guard must reject this even though the resolved path
|
||||
// exists and contains files.
|
||||
let manifest = json!({
|
||||
"name": "malicious",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "../secret"},
|
||||
});
|
||||
assert!(get_bins_from_package_manifest::<RealApi>(&manifest, &pkg).is_empty());
|
||||
}
|
||||
|
||||
/// `directories.bin` pointing at a non-existent subdirectory must
|
||||
/// degrade to an empty list (pnpm's `ENOENT` swallowing in `findFiles`).
|
||||
#[test]
|
||||
fn directories_bin_missing_directory_returns_empty() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
let manifest = json!({
|
||||
"name": "x",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "missing-dir"},
|
||||
});
|
||||
assert!(get_bins_from_package_manifest::<RealApi>(&manifest, &pkg).is_empty());
|
||||
}
|
||||
|
||||
/// `directories.bin` filters out files whose basename fails the
|
||||
/// URL-safe-name guard. Pin via a `..` filename: once the
|
||||
/// path-traversal guard already passed (the dir was a real subdir),
|
||||
/// a *file* inside it with an unsafe name still gets dropped.
|
||||
#[test]
|
||||
fn directories_bin_filters_unsafe_file_names() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
let bin_dir = pkg.join("bin");
|
||||
create_dir_all(&bin_dir).unwrap();
|
||||
write_file(bin_dir.join("good"), "").unwrap();
|
||||
write_file(bin_dir.join("bad space"), "").unwrap();
|
||||
|
||||
let manifest = json!({
|
||||
"name": "tool",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "bin"},
|
||||
});
|
||||
let mut commands = get_bins_from_package_manifest::<RealApi>(&manifest, &pkg);
|
||||
commands.sort_by(|a, b| a.name.cmp(&b.name));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "good");
|
||||
}
|
||||
|
||||
/// Empty bin name returns false via the `is_empty` guard inside
|
||||
/// `is_safe_bin_name`. Exercised via a `bin` object with an empty key.
|
||||
#[test]
|
||||
fn empty_bin_key_is_rejected() {
|
||||
let manifest = json!({
|
||||
"name": "x",
|
||||
"version": "1.0.0",
|
||||
"bin": {"": "ok.js", "good": "ok.js"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, Path::new("/p"));
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "good");
|
||||
}
|
||||
|
||||
/// [`lexical_normalize`] drops `.` (CurDir) segments. This is a direct
|
||||
/// test on the helper. The integration-style test below covers the same
|
||||
/// arm via `directories.bin`, but a direct assertion makes the CurDir
|
||||
/// branch visible to coverage tooling that can't see through inlined
|
||||
/// call chains.
|
||||
#[test]
|
||||
fn lexical_normalize_drops_curdir_segments_directly() {
|
||||
assert_eq!(lexical_normalize(Path::new("a/./b")), PathBuf::from("a/b"));
|
||||
assert_eq!(lexical_normalize(Path::new("./a/b")), PathBuf::from("a/b"));
|
||||
assert_eq!(lexical_normalize(Path::new("a/b/.")), PathBuf::from("a/b"));
|
||||
assert_eq!(lexical_normalize(Path::new("./.")), PathBuf::new());
|
||||
}
|
||||
|
||||
/// On an absolute path, a `..` that would escape the root is dropped
|
||||
/// instead of being materialised as a literal `..` segment. Mirrors
|
||||
/// Node's `path.resolve` (and therefore pnpm's `is-subdir`), where
|
||||
/// `path.resolve("/a/../../a/bin.js") === "/a/bin.js"`.
|
||||
///
|
||||
/// Without this guard, `is_subdir("/a", "/a/../../a/bin.js")` would
|
||||
/// reject the path as outside `/a` even though it resolves back
|
||||
/// inside: `lexical_normalize` would have produced `/../a/bin.js`,
|
||||
/// which fails `starts_with("/a")`.
|
||||
#[test]
|
||||
fn lexical_normalize_drops_excess_parent_dirs_on_absolute_paths() {
|
||||
assert_eq!(lexical_normalize(Path::new("/a/../../a/bin.js")), PathBuf::from("/a/bin.js"));
|
||||
assert_eq!(lexical_normalize(Path::new("/..")), PathBuf::from("/"));
|
||||
assert_eq!(lexical_normalize(Path::new("/../..")), PathBuf::from("/"));
|
||||
}
|
||||
|
||||
/// `is_subdir` accepts a path that goes outside the package root and
|
||||
/// comes back inside via `..`. Regression coverage for the lexical
|
||||
/// normalisation bug where `/<pkg>/x/../../<pkg>/bin.js` was rejected
|
||||
/// because the `..` past the root was materialised as a literal
|
||||
/// `/..` segment. Mirrors pnpm's `path.resolve`-based `is-subdir`.
|
||||
#[test]
|
||||
fn directories_bin_accepts_excess_parent_dirs_that_resolve_inside_pkg() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
let bin_dir = pkg.join("bin-dir");
|
||||
create_dir_all(&bin_dir).unwrap();
|
||||
// `x` must exist on disk so the walker can follow the literal
|
||||
// `<pkg>/x/../../pkg/bin-dir` path the resolver builds.
|
||||
create_dir_all(pkg.join("x")).unwrap();
|
||||
write_file(bin_dir.join("cli"), "").unwrap();
|
||||
|
||||
// `<pkg>/x/../../<pkg-name>/bin-dir` resolves back inside `<pkg>`.
|
||||
let manifest = json!({
|
||||
"name": "tool",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "x/../../pkg/bin-dir"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, &pkg);
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "cli");
|
||||
}
|
||||
|
||||
/// [`lexical_normalize`] `CurDir` branch drops `.` segments. Visible
|
||||
/// via [`super::is_subdir`] accepting a target with embedded `./`
|
||||
/// that resolves inside the package root.
|
||||
#[test]
|
||||
fn directories_bin_handles_curdir_in_relative_path() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
let bin_dir = pkg.join("bin-dir");
|
||||
create_dir_all(&bin_dir).unwrap();
|
||||
write_file(bin_dir.join("cli"), "").unwrap();
|
||||
|
||||
let manifest = json!({
|
||||
"name": "tool",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "./bin-dir"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, &pkg);
|
||||
assert_eq!(commands.len(), 1);
|
||||
assert_eq!(commands[0].name, "cli");
|
||||
}
|
||||
|
||||
/// `commands_from_directories_bin` skips entries whose path doesn't
|
||||
/// yield a usable file name. That covers the `path.file_name() == None`
|
||||
/// and `to_str() == None` branches, both of which the real fs hardly
|
||||
/// ever reaches (file_name() returns None only for paths ending in
|
||||
/// `..`, and to_str() fails only on non-UTF-8 bytes which are rare on
|
||||
/// Unix and impossible on Windows). A fake [`FsWalkFiles`] hands back one
|
||||
/// such path so the `continue` arm gets exercised directly. The
|
||||
/// regular `cli` entry alongside it confirms that the well-formed
|
||||
/// path still flows through and emits a [`Command`](super::Command).
|
||||
#[test]
|
||||
fn directories_bin_skips_path_without_usable_file_name() {
|
||||
use crate::capabilities::FsWalkFiles;
|
||||
use std::io;
|
||||
|
||||
struct EvilWalker;
|
||||
impl FsWalkFiles for EvilWalker {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
[
|
||||
// `file_name()` returns None for a path ending in `..`,
|
||||
// hitting the `let-else continue` branch.
|
||||
PathBuf::from("/pkg/bin/.."),
|
||||
// Well-formed sibling so we can assert the loop's
|
||||
// happy path still runs after the skip.
|
||||
PathBuf::from("/pkg/bin/cli"),
|
||||
]
|
||||
.into_iter()
|
||||
.pipe(Ok)
|
||||
}
|
||||
}
|
||||
|
||||
let manifest = json!({
|
||||
"name": "tool",
|
||||
"version": "1.0.0",
|
||||
"directories": {"bin": "bin"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<EvilWalker>(&manifest, Path::new("/pkg"));
|
||||
assert_eq!(commands.len(), 1, "the `..` entry must be skipped, not crashed on");
|
||||
assert_eq!(commands[0].name, "cli");
|
||||
}
|
||||
|
||||
/// `bin` field takes precedence over `directories.bin` when both are
|
||||
/// present. Mirrors upstream's order-of-checks in
|
||||
/// `getBinsFromPackageManifest`.
|
||||
#[test]
|
||||
fn bin_field_takes_precedence_over_directories_bin() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
let bin_dir = pkg.join("legacy-bin");
|
||||
create_dir_all(&bin_dir).unwrap();
|
||||
write_file(bin_dir.join("ignored.js"), "").unwrap();
|
||||
write_file(pkg.join("primary.js"), "").unwrap();
|
||||
|
||||
let manifest = json!({
|
||||
"name": "tool",
|
||||
"version": "1.0.0",
|
||||
"bin": "primary.js",
|
||||
"directories": {"bin": "legacy-bin"},
|
||||
});
|
||||
let commands = get_bins_from_package_manifest::<RealApi>(&manifest, &pkg);
|
||||
assert_eq!(commands.len(), 1, "bin field wins, directories.bin is ignored");
|
||||
assert_eq!(commands[0].name, "tool");
|
||||
}
|
||||
241
pacquet/crates/cmd-shim/src/capabilities.rs
Normal file
241
pacquet/crates/cmd-shim/src/capabilities.rs
Normal file
@@ -0,0 +1,241 @@
|
||||
//! Per-capability dependency-injection traits and the production
|
||||
//! [`RealApi`] provider. Mirrors the pattern documented at
|
||||
//! <https://github.com/pnpm/pacquet/pull/332#issuecomment-4345054524>:
|
||||
//!
|
||||
//! 1. One trait per capability.
|
||||
//! 2. Functions bind only what they consume (compose bounds).
|
||||
//! 3. No `&self` on capability methods.
|
||||
//! 4. Production callers turbofish the real impl explicitly.
|
||||
//!
|
||||
//! Tests inject unit-struct fakes to exercise IO error paths that the
|
||||
//! real filesystem can't reach portably (e.g. permission denied,
|
||||
//! ENOSPC).
|
||||
|
||||
use pipe_trait::Pipe;
|
||||
use std::{
|
||||
io,
|
||||
path::{Path, PathBuf},
|
||||
};
|
||||
|
||||
/// Read up to `buf.len()` bytes of `path` starting at byte `offset`.
|
||||
///
|
||||
/// The returned `usize` is the number of bytes actually written into
|
||||
/// `buf`. Like `std::io::Read::read`, an impl is allowed to return
|
||||
/// fewer bytes than requested (a "short read") even when more data is
|
||||
/// available, so callers that need a fully-filled buffer must loop.
|
||||
/// [`crate::read_head_filled`] supplies that loop while staying
|
||||
/// generic over this trait, so test fakes do not have to grow.
|
||||
///
|
||||
/// The trait makes no claim about how many syscalls a particular
|
||||
/// impl will use — the production `RealApi` impl opens the file,
|
||||
/// seeks to `offset` (if non-zero), and reads, which is more than
|
||||
/// one. What it does promise is the semantic contract: read up to
|
||||
/// `buf.len()` bytes starting at `offset` into `buf`.
|
||||
///
|
||||
/// Used by [`crate::search_script_runtime`] (via [`crate::read_head_filled`])
|
||||
/// to detect the script runtime via the shebang at the head of a bin
|
||||
/// file.
|
||||
pub trait FsReadHead {
|
||||
fn read_head(path: &Path, offset: u64, buf: &mut [u8]) -> io::Result<usize>;
|
||||
}
|
||||
|
||||
/// Read the entire contents of a file into a `Vec<u8>`. Used to read
|
||||
/// `package.json` files when collecting bin sources.
|
||||
pub trait FsReadFile {
|
||||
fn read_file(path: &Path) -> io::Result<Vec<u8>>;
|
||||
}
|
||||
|
||||
/// Read the entire contents of a file into a `String`. Used by
|
||||
/// [`crate::link_bins_of_packages`] to short-circuit on warm reinstalls
|
||||
/// where the existing shim already targets the same bin file.
|
||||
pub trait FsReadString {
|
||||
fn read_to_string(path: &Path) -> io::Result<String>;
|
||||
}
|
||||
|
||||
/// List the entries of a directory.
|
||||
///
|
||||
/// Returns an `impl Iterator<Item = PathBuf>` rather than a
|
||||
/// `Vec<PathBuf>`, so the production impl can stream entries straight
|
||||
/// out of `fs::ReadDir` without materialising the whole list. The
|
||||
/// associated-type-free shape also frees fakes from declaring an
|
||||
/// `Iter` type per impl. Each fake just returns whatever concrete
|
||||
/// iterator it wants.
|
||||
///
|
||||
/// We deliberately do not expose `fs::ReadDir` directly: its iterator
|
||||
/// type is platform-specific and yields `io::Result<DirEntry>`,
|
||||
/// which would force every fake to fabricate a `DirEntry` (and tie
|
||||
/// the trait to libstd's filesystem types). Yielding plain
|
||||
/// `PathBuf` keeps fakes trivial.
|
||||
pub trait FsReadDir {
|
||||
fn read_dir(path: &Path) -> io::Result<impl Iterator<Item = PathBuf>>;
|
||||
}
|
||||
|
||||
/// Recursively walk `path` and yield every regular file found beneath
|
||||
/// it (depth-first, no symlink follow). Used by
|
||||
/// [`crate::get_bins_from_package_manifest`] to enumerate
|
||||
/// `directories.bin` entries.
|
||||
///
|
||||
/// Returns an `impl Iterator<Item = PathBuf>` rather than a
|
||||
/// `Vec<PathBuf>`, so the production walker streams entries straight
|
||||
/// out of `walkdir` instead of materialising the whole list up front.
|
||||
/// `directories.bin` trees are usually tiny in practice, but the
|
||||
/// abstraction should not bake in an allocation the real
|
||||
/// implementation does not need. Fakes return whatever concrete
|
||||
/// iterator they want. [`std::iter::empty`] fits the unreachable-walk
|
||||
/// case, and [`Vec::into_iter`] fits the case that feeds a fixed list
|
||||
/// of paths.
|
||||
///
|
||||
/// `walkdir`'s builder exposes many knobs (`follow_links`, `min_depth`,
|
||||
/// `max_depth`, `sort_by`, and so on); pacquet uses just one
|
||||
/// (`follow_links = false`). Mirroring the full builder through the
|
||||
/// trait would be over-engineering for the single call site, so the
|
||||
/// trait keeps its surface dead-simple and the impl bakes the option
|
||||
/// in. If a future caller needs different walk options, add a new
|
||||
/// capability rather than parameterise this one.
|
||||
pub trait FsWalkFiles {
|
||||
fn walk_files(path: &Path) -> io::Result<impl Iterator<Item = PathBuf>>;
|
||||
}
|
||||
|
||||
/// Create a directory and any missing ancestors. Used to prepare
|
||||
/// `<modules_dir>/.bin` and per-slot `node_modules/.bin` directories.
|
||||
pub trait FsCreateDirAll {
|
||||
fn create_dir_all(path: &Path) -> io::Result<()>;
|
||||
}
|
||||
|
||||
/// Write `bytes` to `path`, replacing the file's contents if it
|
||||
/// exists. Used to write the three shim flavors (`.sh`, `.cmd`,
|
||||
/// `.ps1`).
|
||||
///
|
||||
/// **Not atomic.** This trait is the moral equivalent of
|
||||
/// `std::fs::write`: it opens (or creates and truncates) the file,
|
||||
/// writes `bytes`, and closes. No tempfile + rename guard, no
|
||||
/// `fsync`. A SIGINT or crash mid-write can leave a truncated file
|
||||
/// on disk. Number of syscalls is up to the impl — `std::fs::write`
|
||||
/// itself is open/(truncate)/write/close, and a fake might loop.
|
||||
/// If a future caller needs atomic write semantics, build it on top
|
||||
/// of this trait by writing to a sibling tempfile and then
|
||||
/// renaming. Hiding that algorithm inside the capability would
|
||||
/// obscure what each callsite inherits; keeping the trait minimal
|
||||
/// lets every callsite see exactly what guarantees it gets.
|
||||
pub trait FsWrite {
|
||||
fn write(path: &Path, bytes: &[u8]) -> io::Result<()>;
|
||||
}
|
||||
|
||||
/// Replace the permission bits at `path` with `0o755`. Used to chmod
|
||||
/// the freshly written shim file so it is executable.
|
||||
///
|
||||
/// The method is always present so callers don't have to
|
||||
/// `#[cfg(unix)]` every chmod call site. On Windows the production
|
||||
/// impl is a no-op (Windows has no equivalent permission concept).
|
||||
pub trait FsSetExecutable {
|
||||
fn set_executable(path: &Path) -> io::Result<()>;
|
||||
}
|
||||
|
||||
/// Read the existing permission bits at `path`, OR in `0o111`, and
|
||||
/// write them back. Used to add the executable bits to the underlying
|
||||
/// target binary (mirrors pnpm's `fixBin`) without clobbering the
|
||||
/// existing read/write bits the way [`FsSetExecutable`] would.
|
||||
///
|
||||
/// The method is always present for the same reason as
|
||||
/// [`FsSetExecutable::set_executable`]; the production impl is a
|
||||
/// no-op on Windows.
|
||||
pub trait FsEnsureExecutableBits {
|
||||
fn ensure_executable_bits(path: &Path) -> io::Result<()>;
|
||||
}
|
||||
|
||||
/// The production filesystem provider. Every method delegates straight
|
||||
/// to `std::fs`.
|
||||
pub struct RealApi;
|
||||
|
||||
impl FsReadHead for RealApi {
|
||||
fn read_head(path: &Path, offset: u64, buf: &mut [u8]) -> io::Result<usize> {
|
||||
use std::io::{Read, Seek, SeekFrom};
|
||||
let mut file = std::fs::File::open(path)?;
|
||||
if offset > 0 {
|
||||
file.seek(SeekFrom::Start(offset))?;
|
||||
}
|
||||
file.read(buf)
|
||||
}
|
||||
}
|
||||
|
||||
impl FsReadFile for RealApi {
|
||||
fn read_file(path: &Path) -> io::Result<Vec<u8>> {
|
||||
std::fs::read(path)
|
||||
}
|
||||
}
|
||||
|
||||
impl FsReadString for RealApi {
|
||||
fn read_to_string(path: &Path) -> io::Result<String> {
|
||||
std::fs::read_to_string(path)
|
||||
}
|
||||
}
|
||||
|
||||
impl FsReadDir for RealApi {
|
||||
fn read_dir(path: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
// `flatten()` silently drops per-entry errors. This matches the
|
||||
// prior collect-then-flatten shape and the `tinyglobby`-style
|
||||
// ENOENT-on-subtree behaviour pacquet's callers expect.
|
||||
std::fs::read_dir(path)?.flatten().map(|entry| entry.path()).pipe(Ok)
|
||||
}
|
||||
}
|
||||
|
||||
impl FsWalkFiles for RealApi {
|
||||
fn walk_files(path: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
// `flatten()` silently drops per-entry errors and matches
|
||||
// pnpm's `tinyglobby` ENOENT-on-subtree behaviour. The
|
||||
// top-level missing-dir case also flows through here as a
|
||||
// single dropped `Err`, so a missing `bin_dir` produces an
|
||||
// empty stream rather than an error.
|
||||
path.pipe(walkdir::WalkDir::new)
|
||||
.follow_links(false)
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.filter(|entry| entry.file_type().is_file())
|
||||
.map(|entry| entry.path().to_path_buf())
|
||||
.pipe(Ok)
|
||||
}
|
||||
}
|
||||
|
||||
impl FsCreateDirAll for RealApi {
|
||||
fn create_dir_all(path: &Path) -> io::Result<()> {
|
||||
std::fs::create_dir_all(path)
|
||||
}
|
||||
}
|
||||
|
||||
impl FsWrite for RealApi {
|
||||
fn write(path: &Path, bytes: &[u8]) -> io::Result<()> {
|
||||
std::fs::write(path, bytes)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
impl FsSetExecutable for RealApi {
|
||||
fn set_executable(path: &Path) -> io::Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o755))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
impl FsSetExecutable for RealApi {
|
||||
fn set_executable(_path: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
impl FsEnsureExecutableBits for RealApi {
|
||||
fn ensure_executable_bits(path: &Path) -> io::Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
let metadata = std::fs::metadata(path)?;
|
||||
let mode = metadata.permissions().mode() | 0o111;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(mode))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
impl FsEnsureExecutableBits for RealApi {
|
||||
fn ensure_executable_bits(_path: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
29
pacquet/crates/cmd-shim/src/lib.rs
Normal file
29
pacquet/crates/cmd-shim/src/lib.rs
Normal file
@@ -0,0 +1,29 @@
|
||||
//! Bin field parsing and command-shim generation.
|
||||
//!
|
||||
//! Mirrors three pnpm v11 packages:
|
||||
//!
|
||||
//! - `@pnpm/bins.resolver` parses the `bin` (and `directories.bin`) field of a
|
||||
//! package's `package.json` into a list of `(name, path)` commands. See
|
||||
//! <https://github.com/pnpm/pnpm/blob/4750fd370c/bins/resolver/src/index.ts>.
|
||||
//! - `@pnpm/bins.linker` orchestrates the per-`node_modules` linking pass and
|
||||
//! the conflict resolution between bins of the same name. See
|
||||
//! <https://github.com/pnpm/pnpm/blob/4750fd370c/bins/linker/src/index.ts>.
|
||||
//! - `@zkochan/cmd-shim` generates the actual shim file contents. See
|
||||
//! <https://github.com/pnpm/cmd-shim/blob/0d79ca9534/src/index.ts>.
|
||||
//!
|
||||
//! Pacquet's first iteration covers the direct-dependency path (root project's
|
||||
//! `node_modules/.bin`) and the per-virtual-store path
|
||||
//! (`node_modules/.pacquet/<pkg>@<ver>/node_modules/<pkg>/node_modules/.bin`).
|
||||
//! Hoisted-bin precedence and lifecycle-script-created bins are deferred per
|
||||
//! `plans/TEST_PORTING.md`.
|
||||
|
||||
mod bin_resolver;
|
||||
mod capabilities;
|
||||
mod link_bins;
|
||||
mod path_util;
|
||||
mod shim;
|
||||
|
||||
pub use bin_resolver::*;
|
||||
pub use capabilities::*;
|
||||
pub use link_bins::*;
|
||||
pub use shim::*;
|
||||
396
pacquet/crates/cmd-shim/src/link_bins.rs
Normal file
396
pacquet/crates/cmd-shim/src/link_bins.rs
Normal file
@@ -0,0 +1,396 @@
|
||||
use crate::{
|
||||
bin_resolver::{Command, get_bins_from_package_manifest, pkg_owns_bin},
|
||||
capabilities::{
|
||||
FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadString,
|
||||
FsSetExecutable, FsWalkFiles, FsWrite,
|
||||
},
|
||||
shim::{
|
||||
generate_cmd_shim, generate_pwsh_shim, generate_sh_shim, is_shim_pointing_at,
|
||||
search_script_runtime,
|
||||
},
|
||||
};
|
||||
use derive_more::{Display, Error};
|
||||
use miette::Diagnostic;
|
||||
use rayon::prelude::*;
|
||||
use serde_json::Value;
|
||||
use std::{
|
||||
collections::HashMap,
|
||||
io,
|
||||
path::{Path, PathBuf},
|
||||
sync::Arc,
|
||||
};
|
||||
|
||||
/// One package known to be installed at `location`, with its parsed
|
||||
/// `package.json`. Mirrors the per-package input shape of pnpm's
|
||||
/// `linkBinsOfPackages`.
|
||||
///
|
||||
/// The manifest is shared via `Arc` rather than owned by value: the
|
||||
/// lockfile-driven bin-link path looks up the same parsed manifest
|
||||
/// from a process-wide map, so packing it into a [`PackageBinSource`]
|
||||
/// is a refcount bump (cheap) rather than a deep clone of the JSON
|
||||
/// tree (which would have been the bulk of the per-slot CPU work,
|
||||
/// since the per-install clone count is `slots × children` =
|
||||
/// thousands of times).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PackageBinSource {
|
||||
pub location: PathBuf,
|
||||
pub manifest: Arc<Value>,
|
||||
}
|
||||
|
||||
/// Error type for [`link_bins_of_packages`].
|
||||
#[derive(Debug, Display, Error, Diagnostic)]
|
||||
pub enum LinkBinsError {
|
||||
#[display("Failed to create bin directory at {dir:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_cmd_shim::create_bin_dir))]
|
||||
CreateBinDir {
|
||||
dir: PathBuf,
|
||||
#[error(source)]
|
||||
error: io::Error,
|
||||
},
|
||||
|
||||
#[display("Failed to read modules directory at {dir:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_cmd_shim::read_modules_dir))]
|
||||
ReadModulesDir {
|
||||
dir: PathBuf,
|
||||
#[error(source)]
|
||||
error: io::Error,
|
||||
},
|
||||
|
||||
#[display("Failed to read package manifest at {path:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_cmd_shim::read_manifest))]
|
||||
ReadManifest {
|
||||
path: PathBuf,
|
||||
#[error(source)]
|
||||
error: io::Error,
|
||||
},
|
||||
|
||||
#[display("Failed to parse package manifest at {path:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_cmd_shim::parse_manifest))]
|
||||
ParseManifest {
|
||||
path: PathBuf,
|
||||
#[error(source)]
|
||||
error: serde_json::Error,
|
||||
},
|
||||
|
||||
#[display("Failed to read shim source {path:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_cmd_shim::probe_shim_source))]
|
||||
ProbeShimSource {
|
||||
path: PathBuf,
|
||||
#[error(source)]
|
||||
error: io::Error,
|
||||
},
|
||||
|
||||
#[display("Failed to write shim file at {path:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_cmd_shim::write_shim))]
|
||||
WriteShim {
|
||||
path: PathBuf,
|
||||
#[error(source)]
|
||||
error: io::Error,
|
||||
},
|
||||
|
||||
#[display("Failed to chmod {path:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_cmd_shim::chmod))]
|
||||
Chmod {
|
||||
path: PathBuf,
|
||||
#[error(source)]
|
||||
error: io::Error,
|
||||
},
|
||||
}
|
||||
|
||||
/// Read `<location>/package.json` for each entry under `modules_dir` and link
|
||||
/// its bins into `bins_dir`. Mirrors pnpm v11's `linkBins(modulesDir, binsDir)`
|
||||
/// at <https://github.com/pnpm/pnpm/blob/4750fd370c/bins/linker/src/index.ts>.
|
||||
///
|
||||
/// Skips:
|
||||
/// - The `.bin` and `.pacquet` directories themselves (and any other
|
||||
/// dot-prefixed entry, matching pnpm).
|
||||
/// - Entries whose `package.json` cannot be read (legitimate when a directory
|
||||
/// under `node_modules` happens to not be a package, e.g. an empty scope
|
||||
/// directory).
|
||||
///
|
||||
/// Scoped packages are recursed: `node_modules/@scope/foo` becomes one
|
||||
/// candidate. This mirrors `binNamesAndPaths` in upstream `linkBins`.
|
||||
pub fn link_bins<Api>(modules_dir: &Path, bins_dir: &Path) -> Result<(), LinkBinsError>
|
||||
where
|
||||
Api: FsReadDir
|
||||
+ FsReadFile
|
||||
+ FsReadString
|
||||
+ FsReadHead
|
||||
+ FsCreateDirAll
|
||||
+ FsWalkFiles
|
||||
+ FsWrite
|
||||
+ FsSetExecutable
|
||||
+ FsEnsureExecutableBits,
|
||||
{
|
||||
let packages = collect_packages_in_modules_dir::<Api>(modules_dir)?;
|
||||
link_bins_of_packages::<Api>(&packages, bins_dir)
|
||||
}
|
||||
|
||||
fn collect_packages_in_modules_dir<Api>(
|
||||
modules_dir: &Path,
|
||||
) -> Result<Vec<PackageBinSource>, LinkBinsError>
|
||||
where
|
||||
Api: FsReadDir + FsReadFile,
|
||||
{
|
||||
let mut packages = Vec::new();
|
||||
|
||||
let entries = match Api::read_dir(modules_dir) {
|
||||
Ok(entries) => entries,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(packages),
|
||||
Err(error) => {
|
||||
return Err(LinkBinsError::ReadModulesDir { dir: modules_dir.to_path_buf(), error });
|
||||
}
|
||||
};
|
||||
|
||||
for path in entries {
|
||||
let Some(name) = path.file_name() else {
|
||||
continue;
|
||||
};
|
||||
let name_str = name.to_string_lossy();
|
||||
if name_str.starts_with('.') {
|
||||
continue;
|
||||
}
|
||||
|
||||
if name_str.starts_with('@') {
|
||||
// Scoped: walk one level deeper. Only `NotFound` is
|
||||
// plausibly skippable (a concurrent scope-dir delete);
|
||||
// other errors — `PermissionDenied`, `EIO`, AppArmor
|
||||
// deny — would silently drop every bin under this
|
||||
// scope, so surface them as `ReadModulesDir`. Matches
|
||||
// the policy the per-`modules_dir` read above already
|
||||
// uses.
|
||||
let scope_entries = match Api::read_dir(&path) {
|
||||
Ok(entries) => entries,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => continue,
|
||||
Err(error) => {
|
||||
return Err(LinkBinsError::ReadModulesDir { dir: path.clone(), error });
|
||||
}
|
||||
};
|
||||
for sub_path in scope_entries {
|
||||
if let Some(pkg) = read_package::<Api>(&sub_path)? {
|
||||
packages.push(pkg);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if let Some(pkg) = read_package::<Api>(&path)? {
|
||||
packages.push(pkg);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(packages)
|
||||
}
|
||||
|
||||
fn read_package<Api: FsReadFile>(
|
||||
location: &Path,
|
||||
) -> Result<Option<PackageBinSource>, LinkBinsError> {
|
||||
let manifest_path = location.join("package.json");
|
||||
let bytes = match Api::read_file(&manifest_path) {
|
||||
Ok(bytes) => bytes,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None),
|
||||
Err(error) => return Err(LinkBinsError::ReadManifest { path: manifest_path, error }),
|
||||
};
|
||||
let manifest: Value = serde_json::from_slice(&bytes)
|
||||
.map_err(|error| LinkBinsError::ParseManifest { path: manifest_path, error })?;
|
||||
Ok(Some(PackageBinSource { location: location.to_path_buf(), manifest: Arc::new(manifest) }))
|
||||
}
|
||||
|
||||
/// Link every bin declared by `packages` into `bins_dir`, applying the same
|
||||
/// conflict resolution upstream uses.
|
||||
///
|
||||
/// Conflict resolution mirrors `resolveCommandConflicts`:
|
||||
///
|
||||
/// 1. Ownership wins. If exactly one package owns the bin name (via
|
||||
/// [`pkg_owns_bin`]), it wins outright.
|
||||
/// 2. Otherwise lexical comparison on the package name, lower wins. Stable
|
||||
/// and deterministic regardless of the order packages were discovered.
|
||||
///
|
||||
/// Pacquet's first iteration does not resolve same-package multi-version
|
||||
/// conflicts via semver (a feature upstream uses for hoisting), since the
|
||||
/// virtual-store layout means each bin source is a unique
|
||||
/// `(package, version)` slot already.
|
||||
pub fn link_bins_of_packages<Api>(
|
||||
packages: &[PackageBinSource],
|
||||
bins_dir: &Path,
|
||||
) -> Result<(), LinkBinsError>
|
||||
where
|
||||
Api: FsReadString
|
||||
+ FsReadHead
|
||||
+ FsCreateDirAll
|
||||
+ FsWalkFiles
|
||||
+ FsWrite
|
||||
+ FsSetExecutable
|
||||
+ FsEnsureExecutableBits,
|
||||
{
|
||||
let mut chosen: HashMap<String, (Command, &PackageBinSource)> = HashMap::new();
|
||||
|
||||
for pkg in packages {
|
||||
let pkg_name = pkg.manifest.get("name").and_then(Value::as_str).unwrap_or("");
|
||||
let commands = get_bins_from_package_manifest::<Api>(&pkg.manifest, &pkg.location);
|
||||
for command in commands {
|
||||
match chosen.get(&command.name) {
|
||||
None => {
|
||||
chosen.insert(command.name.clone(), (command, pkg));
|
||||
}
|
||||
Some((_, existing)) => {
|
||||
let existing_name =
|
||||
existing.manifest.get("name").and_then(Value::as_str).unwrap_or("");
|
||||
if pick_winner(&command.name, existing_name, pkg_name) {
|
||||
chosen.insert(command.name.clone(), (command, pkg));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if chosen.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
Api::create_dir_all(bins_dir)
|
||||
.map_err(|error| LinkBinsError::CreateBinDir { dir: bins_dir.to_path_buf(), error })?;
|
||||
|
||||
// Each shim's read-shebang + write-file + chmod sequence is independent
|
||||
// across bin names. There is no shared state, so drive them on rayon.
|
||||
// The hot path is per-package-bin; without parallelism the per-shim
|
||||
// file I/O serialised across the whole `chosen` map.
|
||||
chosen.par_iter().try_for_each(|(bin_name, (command, _pkg))| {
|
||||
write_shim::<Api>(&command.path, &bins_dir.join(bin_name))
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Return `true` when `candidate` should replace `existing` for `bin_name`.
|
||||
/// Matches the two-step ownership-then-lexical-compare in upstream's
|
||||
/// `resolveCommandConflicts`.
|
||||
fn pick_winner(bin_name: &str, existing: &str, candidate: &str) -> bool {
|
||||
let existing_owns = pkg_owns_bin(bin_name, existing);
|
||||
let candidate_owns = pkg_owns_bin(bin_name, candidate);
|
||||
match (existing_owns, candidate_owns) {
|
||||
(true, false) => false,
|
||||
(false, true) => true,
|
||||
// Both own (or neither): fall through to lexical compare. Picking the
|
||||
// smaller name keeps results deterministic across input orderings.
|
||||
_ => candidate < existing,
|
||||
}
|
||||
}
|
||||
|
||||
/// Write all three shim flavors for `target_path` (the canonical `.sh`
|
||||
/// at `shim_path`, plus the `.cmd` and `.ps1` siblings) and chmod them
|
||||
/// executable. Idempotent on warm reinstalls via [`is_shim_pointing_at`].
|
||||
///
|
||||
/// Pnpm always emits all three flavors per bin (independent of host
|
||||
/// platform), so a project installed on Linux stays usable when the
|
||||
/// same `node_modules` is reused from Windows via a network share or
|
||||
/// a `git clone` of a checked-in install. Pacquet matches that
|
||||
/// contract here: `generate_sh_shim`, `generate_cmd_shim`, and
|
||||
/// `generate_pwsh_shim` are unconditional, and the writer emits all
|
||||
/// three.
|
||||
///
|
||||
/// The chmod step (`set_executable` for the canonical shim and
|
||||
/// `ensure_executable_bits` for the target binary, matching pnpm's
|
||||
/// `fixBin(cmd.path, 0o755)` and `chmodShim`) is wired through the
|
||||
/// [`FsSetExecutable`] / [`FsEnsureExecutableBits`] capability traits.
|
||||
/// On Unix the production impls run the actual `chmod`; on Windows
|
||||
/// they are no-ops (Windows has no equivalent permission concept), so
|
||||
/// the call sites stay portable and don't need their own
|
||||
/// `#[cfg(unix)]` gating.
|
||||
fn write_shim<Api>(target_path: &Path, shim_path: &Path) -> Result<(), LinkBinsError>
|
||||
where
|
||||
Api: FsReadString + FsReadHead + FsWrite + FsSetExecutable + FsEnsureExecutableBits,
|
||||
{
|
||||
let runtime = search_script_runtime::<Api>(target_path).map_err(|error| {
|
||||
LinkBinsError::ProbeShimSource { path: target_path.to_path_buf(), error }
|
||||
})?;
|
||||
|
||||
let sh_body = generate_sh_shim(target_path, shim_path, runtime.as_ref());
|
||||
let cmd_path = with_extension_appended(shim_path, "cmd");
|
||||
let ps1_path = with_extension_appended(shim_path, "ps1");
|
||||
let cmd_body = generate_cmd_shim(target_path, &cmd_path, runtime.as_ref());
|
||||
let ps1_body = generate_pwsh_shim(target_path, &ps1_path, runtime.as_ref());
|
||||
|
||||
// Idempotent skip only fires when all three flavors are already
|
||||
// present *and pointing at the right target*. Gating on the `.sh`
|
||||
// flavor alone (an earlier version of this code) left the upgrade
|
||||
// path broken: a previous install (e.g. older pacquet,
|
||||
// partial-write crash) might have written `.sh` correctly but
|
||||
// never written `.cmd`/`.ps1`, in which case the marker check
|
||||
// would short-circuit and the missing siblings would never be
|
||||
// repaired.
|
||||
//
|
||||
// The `.sh` flavor carries a `# cmd-shim-target=<path>` trailer
|
||||
// that [`is_shim_pointing_at`] reads; the `.cmd` and `.ps1`
|
||||
// flavors don't, so we compare them byte-for-byte against the
|
||||
// freshly generated body. That catches stale/corrupted siblings
|
||||
// that an existence-only check would let slip through (Copilot
|
||||
// flagged this on
|
||||
// <https://github.com/pnpm/pacquet/pull/333#discussion_r3222744353>):
|
||||
// a manually-edited `.cmd` pointing at a stale target, or an
|
||||
// earlier pacquet write with a different relative path, would
|
||||
// bypass the rewrite under the prior `.is_ok()` gate. Generated
|
||||
// bodies are stable across pacquet versions (only the `<target>`
|
||||
// segment moves), so byte equality is a sound equivalence check.
|
||||
let sh_marker_ok = matches!(
|
||||
Api::read_to_string(shim_path),
|
||||
Ok(existing) if is_shim_pointing_at(&existing, target_path),
|
||||
);
|
||||
let cmd_ok = matches!(
|
||||
Api::read_to_string(&cmd_path),
|
||||
Ok(existing) if existing == cmd_body,
|
||||
);
|
||||
let ps1_ok = matches!(
|
||||
Api::read_to_string(&ps1_path),
|
||||
Ok(existing) if existing == ps1_body,
|
||||
);
|
||||
let already_correct = sh_marker_ok && cmd_ok && ps1_ok;
|
||||
|
||||
if !already_correct {
|
||||
Api::write(shim_path, sh_body.as_bytes())
|
||||
.map_err(|error| LinkBinsError::WriteShim { path: shim_path.to_path_buf(), error })?;
|
||||
Api::write(&cmd_path, cmd_body.as_bytes())
|
||||
.map_err(|error| LinkBinsError::WriteShim { path: cmd_path.clone(), error })?;
|
||||
Api::write(&ps1_path, ps1_body.as_bytes())
|
||||
.map_err(|error| LinkBinsError::WriteShim { path: ps1_path.clone(), error })?;
|
||||
}
|
||||
|
||||
Api::set_executable(shim_path)
|
||||
.map_err(|error| LinkBinsError::Chmod { path: shim_path.to_path_buf(), error })?;
|
||||
// Make the underlying script executable too. pnpm calls
|
||||
// `fixBin(cmd.path, 0o755)` to do this; we apply the same minimum
|
||||
// mode without rewriting CRLF shebangs (a feature pnpm inherits
|
||||
// from npm's `bin-links/lib/fix-bin.js`). Targets shipped by npm
|
||||
// already use LF in practice, so the simpler chmod-only path is
|
||||
// enough for the install tests this PR ports. `NotFound` is
|
||||
// swallowed because the target may legitimately have been
|
||||
// removed by an unrelated process between extraction and shim
|
||||
// linking. Everything else (`PermissionDenied`, `EROFS`,
|
||||
// AppArmor deny, foreign uid) surfaces as `LinkBinsError::Chmod`
|
||||
// so real failures don't disappear silently. Mirrors pnpm's
|
||||
// `fixBin` ENOENT guard.
|
||||
match Api::ensure_executable_bits(target_path) {
|
||||
Ok(()) => {}
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => {}
|
||||
Err(error) => {
|
||||
return Err(LinkBinsError::Chmod { path: target_path.to_path_buf(), error });
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Append `<ext>` to `path` as a *new* extension segment (`foo` becomes
|
||||
/// `foo.cmd`), regardless of any existing extension. `Path::with_extension`
|
||||
/// would *replace* the existing extension, which is wrong for our case.
|
||||
/// The bin name `tsc` keeps its own `tsc` and gains a sibling `tsc.cmd`,
|
||||
/// rather than turning into `tsc.cmd` and losing the original `.sh` flavor.
|
||||
fn with_extension_appended(path: &Path, ext: &str) -> PathBuf {
|
||||
let mut result = path.as_os_str().to_owned();
|
||||
result.push(".");
|
||||
result.push(ext);
|
||||
result.into()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
931
pacquet/crates/cmd-shim/src/link_bins/tests.rs
Normal file
931
pacquet/crates/cmd-shim/src/link_bins/tests.rs
Normal file
@@ -0,0 +1,931 @@
|
||||
use super::{LinkBinsError, PackageBinSource, link_bins, link_bins_of_packages};
|
||||
use crate::{
|
||||
capabilities::{
|
||||
FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadString,
|
||||
FsSetExecutable, FsWalkFiles, FsWrite, RealApi,
|
||||
},
|
||||
shim::is_shim_pointing_at,
|
||||
};
|
||||
use serde_json::{Value, json};
|
||||
use std::{
|
||||
fs::{
|
||||
create_dir_all, metadata, read as read_file, read_to_string, remove_file,
|
||||
write as write_file,
|
||||
},
|
||||
iter::{Empty, empty},
|
||||
path::{Path, PathBuf},
|
||||
sync::Arc,
|
||||
};
|
||||
use tempfile::tempdir;
|
||||
|
||||
/// All three shim flavors (`.sh` / no-extension, `.cmd`, `.ps1`) must
|
||||
/// be written for every linked bin so a project installed on Linux
|
||||
/// remains usable on Windows after a `git clone`. Mirrors pnpm's
|
||||
/// always-write-all-flavors behavior.
|
||||
#[test]
|
||||
fn writes_all_three_shim_flavors_per_bin() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg_dir = tmp.path().join("node_modules/foo");
|
||||
create_dir_all(&pkg_dir).unwrap();
|
||||
write_file(
|
||||
pkg_dir.join("package.json"),
|
||||
json!({"name": "foo", "version": "1.0.0", "bin": "cli.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(pkg_dir.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
let bins_dir = tmp.path().join("node_modules/.bin");
|
||||
let manifest_value: Value =
|
||||
serde_json::from_slice(&read_file(pkg_dir.join("package.json")).unwrap()).unwrap();
|
||||
link_bins_of_packages::<RealApi>(
|
||||
&[PackageBinSource { location: pkg_dir.clone(), manifest: Arc::new(manifest_value) }],
|
||||
&bins_dir,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let sh = bins_dir.join("foo");
|
||||
let cmd = bins_dir.join("foo.cmd");
|
||||
let ps1 = bins_dir.join("foo.ps1");
|
||||
assert!(sh.exists(), "missing .sh shim");
|
||||
assert!(cmd.exists(), "missing .cmd shim");
|
||||
assert!(ps1.exists(), "missing .ps1 shim");
|
||||
|
||||
let cmd_body = read_to_string(&cmd).unwrap();
|
||||
assert!(cmd_body.starts_with("@SETLOCAL\r\n"), "cmd shim must use CRLF SETLOCAL");
|
||||
assert!(cmd_body.contains("\"%~dp0\\..\\foo\\cli.js\""), "cmd target should be windows-style");
|
||||
|
||||
let ps1_body = read_to_string(&ps1).unwrap();
|
||||
assert!(ps1_body.starts_with("#!/usr/bin/env pwsh\n"));
|
||||
assert!(ps1_body.contains("\"$basedir/../foo/cli.js\""));
|
||||
}
|
||||
|
||||
/// End-to-end exercise: a package with a `bin` field has a shim written
|
||||
/// into the bins dir, the shim references the correct relative path,
|
||||
/// and (on Unix) both the shim and the target are executable.
|
||||
#[test]
|
||||
fn writes_shim_for_bin_string() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg_dir = tmp.path().join("node_modules/foo");
|
||||
create_dir_all(pkg_dir.join("bin")).unwrap();
|
||||
write_file(
|
||||
pkg_dir.join("package.json"),
|
||||
json!({"name": "foo", "version": "1.0.0", "bin": "bin/cli.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(pkg_dir.join("bin/cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
let bins_dir = tmp.path().join("node_modules/.bin");
|
||||
let manifest_value: Value =
|
||||
serde_json::from_slice(&read_file(pkg_dir.join("package.json")).unwrap()).unwrap();
|
||||
link_bins_of_packages::<RealApi>(
|
||||
&[PackageBinSource { location: pkg_dir.clone(), manifest: Arc::new(manifest_value) }],
|
||||
&bins_dir,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let shim_path = bins_dir.join("foo");
|
||||
assert!(shim_path.exists(), "shim should be created");
|
||||
|
||||
let body = read_to_string(&shim_path).unwrap();
|
||||
assert!(body.contains("\"$basedir/../foo/bin/cli.js\""), "shim body: {body}");
|
||||
assert!(is_shim_pointing_at(&body, &pkg_dir.join("bin/cli.js")));
|
||||
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
assert_eq!(
|
||||
metadata(&shim_path).unwrap().permissions().mode() & 0o777,
|
||||
0o755,
|
||||
"shim must be 0o755",
|
||||
);
|
||||
assert!(
|
||||
metadata(pkg_dir.join("bin/cli.js")).unwrap().permissions().mode() & 0o111 != 0,
|
||||
"target must have at least one executable bit",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// [`link_bins::<RealApi>`](link_bins) walks every package and its scoped
|
||||
/// children. Both regular and `@scope/...` packages must contribute their
|
||||
/// bins.
|
||||
#[test]
|
||||
fn link_bins_walks_modules_and_scopes() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
// Regular package
|
||||
create_dir_all(modules.join("foo")).unwrap();
|
||||
write_file(modules.join("foo/package.json"), json!({"name": "foo", "bin": "f.js"}).to_string())
|
||||
.unwrap();
|
||||
write_file(modules.join("foo/f.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
// Scoped package
|
||||
create_dir_all(modules.join("@s/bar")).unwrap();
|
||||
write_file(
|
||||
modules.join("@s/bar/package.json"),
|
||||
json!({"name": "@s/bar", "bin": "b.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(modules.join("@s/bar/b.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
// Non-package directory (no package.json) must be ignored, not error.
|
||||
create_dir_all(modules.join("not-a-package")).unwrap();
|
||||
|
||||
let bins = modules.join(".bin");
|
||||
link_bins::<RealApi>(&modules, &bins).unwrap();
|
||||
|
||||
assert!(bins.join("foo").exists(), "foo shim must exist");
|
||||
assert!(bins.join("bar").exists(), "scoped @s/bar shim must use bare name `bar`");
|
||||
}
|
||||
|
||||
/// [`link_bins`] on a missing `node_modules` directory must be a no-op
|
||||
/// (Ok with empty result), not an error. Real fs returns `NotFound`
|
||||
/// which the implementation already degrades.
|
||||
#[test]
|
||||
fn link_bins_handles_missing_modules_dir() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let bins_dir = tmp.path().join(".bin");
|
||||
link_bins::<RealApi>(&tmp.path().join("missing"), &bins_dir)
|
||||
.expect("missing modules dir is Ok");
|
||||
assert!(!bins_dir.exists(), "no shims means no bin dir created");
|
||||
}
|
||||
|
||||
/// [`link_bins_of_packages`] with no bins to link is a complete no-op.
|
||||
/// It must not even create the bins directory. The empty-`chosen`
|
||||
/// short-circuit guards a slot whose children have no bin field.
|
||||
#[test]
|
||||
fn link_bins_of_packages_no_op_when_no_bins() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("pkg");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
write_file(pkg.join("package.json"), json!({"name": "pkg"}).to_string()).unwrap();
|
||||
let bins = tmp.path().join(".bin");
|
||||
let manifest: Value =
|
||||
serde_json::from_slice(&read_file(pkg.join("package.json")).unwrap()).unwrap();
|
||||
link_bins_of_packages::<RealApi>(
|
||||
&[PackageBinSource { location: pkg, manifest: Arc::new(manifest) }],
|
||||
&bins,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(!bins.exists(), "bins dir must not be created when nothing to link");
|
||||
}
|
||||
|
||||
/// Same-name bin from two non-owner packages: lexical-compare picks the
|
||||
/// alphabetically smaller package name. Pins the
|
||||
/// `resolveCommandConflicts` fallback shape.
|
||||
#[test]
|
||||
fn lexical_compare_breaks_tie_when_neither_owns() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let alpha = tmp.path().join("alpha");
|
||||
let beta = tmp.path().join("beta");
|
||||
for d in [&alpha, &beta] {
|
||||
create_dir_all(d).unwrap();
|
||||
write_file(d.join("cmd.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
}
|
||||
write_file(
|
||||
alpha.join("package.json"),
|
||||
json!({"name": "alpha", "bin": {"shared": "cmd.js"}}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(
|
||||
beta.join("package.json"),
|
||||
json!({"name": "beta", "bin": {"shared": "cmd.js"}}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let manifest_alpha: Value =
|
||||
serde_json::from_slice(&read_file(alpha.join("package.json")).unwrap()).unwrap();
|
||||
let manifest_beta: Value =
|
||||
serde_json::from_slice(&read_file(beta.join("package.json")).unwrap()).unwrap();
|
||||
|
||||
let bins = tmp.path().join(".bin");
|
||||
// Order beta-then-alpha to verify the choice doesn't depend on
|
||||
// discovery order.
|
||||
link_bins_of_packages::<RealApi>(
|
||||
&[
|
||||
PackageBinSource { location: beta.clone(), manifest: Arc::new(manifest_beta) },
|
||||
PackageBinSource { location: alpha.clone(), manifest: Arc::new(manifest_alpha) },
|
||||
],
|
||||
&bins,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let body = read_to_string(bins.join("shared")).unwrap();
|
||||
assert!(
|
||||
body.contains("/alpha/cmd.js"),
|
||||
"lexically smaller package name `alpha` must win, got body:\n{body}",
|
||||
);
|
||||
}
|
||||
|
||||
/// A malformed `package.json` (invalid JSON) under `<modules_dir>` must
|
||||
/// surface as a [`LinkBinsError::ParseManifest`] error, not silently skip.
|
||||
#[test]
|
||||
fn link_bins_propagates_parse_manifest_error() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
create_dir_all(modules.join("broken")).unwrap();
|
||||
write_file(modules.join("broken/package.json"), "{ this is not json").unwrap();
|
||||
|
||||
let bins = modules.join(".bin");
|
||||
let err = link_bins::<RealApi>(&modules, &bins).expect_err("invalid manifest must surface");
|
||||
assert!(
|
||||
matches!(err, LinkBinsError::ParseManifest { .. }),
|
||||
"expected ParseManifest, got {err:?}",
|
||||
);
|
||||
}
|
||||
|
||||
/// [`link_bins`] must idempotently short-circuit when an existing shim
|
||||
/// already targets the same bin file. Pins [`is_shim_pointing_at`]'s
|
||||
/// integration with the writer. Mirrors pnpm's
|
||||
/// "linkBins() skips bins that already reference the correct target":
|
||||
/// <https://github.com/pnpm/pnpm/blob/4750fd370c/bins/linker/test/index.ts#L79-L99>.
|
||||
#[test]
|
||||
fn link_bins_skips_existing_shim_with_matching_marker() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
create_dir_all(modules.join("foo")).unwrap();
|
||||
write_file(modules.join("foo/package.json"), json!({"name": "foo", "bin": "f.js"}).to_string())
|
||||
.unwrap();
|
||||
write_file(modules.join("foo/f.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
let bins = modules.join(".bin");
|
||||
link_bins::<RealApi>(&modules, &bins).unwrap();
|
||||
let original = read_to_string(bins.join("foo")).unwrap();
|
||||
// Append a sentinel. If the second pass rewrites the shim, the
|
||||
// sentinel disappears.
|
||||
let sentinel = format!("{original}\n# SENTINEL");
|
||||
write_file(bins.join("foo"), &sentinel).unwrap();
|
||||
|
||||
link_bins::<RealApi>(&modules, &bins).unwrap();
|
||||
assert_eq!(read_to_string(bins.join("foo")).unwrap(), sentinel);
|
||||
}
|
||||
|
||||
/// [`link_bins`] must NOT skip when only the canonical `.sh` shim exists.
|
||||
/// The `.cmd` and `.ps1` siblings could be missing because an older
|
||||
/// pacquet wrote `.sh`-only or because a partial-write crash interrupted
|
||||
/// the writer mid-batch. Gating on the `.sh` marker alone (an earlier
|
||||
/// version of [`super::write_shim`]) caused those upgrade paths to leave
|
||||
/// the missing siblings permanently absent.
|
||||
#[test]
|
||||
fn link_bins_rewrites_when_only_sh_flavor_exists() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
create_dir_all(modules.join("foo")).unwrap();
|
||||
write_file(modules.join("foo/package.json"), json!({"name": "foo", "bin": "f.js"}).to_string())
|
||||
.unwrap();
|
||||
write_file(modules.join("foo/f.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
let bins = modules.join(".bin");
|
||||
link_bins::<RealApi>(&modules, &bins).unwrap();
|
||||
|
||||
// Simulate the partial-write / older-pacquet state: delete the
|
||||
// .cmd and .ps1 siblings, leaving only the `.sh` shim with its
|
||||
// (still correct) target marker.
|
||||
remove_file(bins.join("foo.cmd")).unwrap();
|
||||
remove_file(bins.join("foo.ps1")).unwrap();
|
||||
|
||||
link_bins::<RealApi>(&modules, &bins).unwrap();
|
||||
|
||||
assert!(bins.join("foo").exists(), ".sh shim must remain");
|
||||
assert!(bins.join("foo.cmd").exists(), ".cmd sibling must be re-created on second pass");
|
||||
assert!(bins.join("foo.ps1").exists(), ".ps1 sibling must be re-created on second pass");
|
||||
}
|
||||
|
||||
/// [`link_bins_of_packages`] propagates a non-`NotFound` `read_dir`
|
||||
/// error from the calling context. Use a fake `Api` that fails the
|
||||
/// initial `create_dir_all` to cover the [`LinkBinsError::CreateBinDir`]
|
||||
/// error variant that real fs can't trigger portably.
|
||||
#[test]
|
||||
fn link_bins_propagates_create_bin_dir_error_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct FailingCreateDir;
|
||||
impl FsReadDir for FailingCreateDir {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
impl FsReadFile for FailingCreateDir {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!("not called when chosen is empty")
|
||||
}
|
||||
}
|
||||
impl FsReadString for FailingCreateDir {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadHead for FailingCreateDir {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for FailingCreateDir {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsWrite for FailingCreateDir {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for FailingCreateDir {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for FailingCreateDir {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for FailingCreateDir {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
// A package with a bin so `chosen` is non-empty.
|
||||
let manifest = serde_json::json!({"name": "foo", "bin": "cli.js"});
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("foo");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
write_file(pkg.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
let err = link_bins_of_packages::<FailingCreateDir>(
|
||||
&[PackageBinSource { location: pkg, manifest: Arc::new(manifest) }],
|
||||
Path::new("/anything"),
|
||||
)
|
||||
.expect_err("create_dir_all error must propagate");
|
||||
assert!(matches!(err, LinkBinsError::CreateBinDir { .. }));
|
||||
}
|
||||
|
||||
/// [`link_bins_of_packages`] propagates a write failure for the `.sh`
|
||||
/// shim. Inject a fake [`FsWrite`] that always fails.
|
||||
#[test]
|
||||
fn link_bins_propagates_write_shim_error_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct FailingWrite;
|
||||
impl FsReadDir for FailingWrite {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
impl FsReadFile for FailingWrite {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadString for FailingWrite {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
// Pretend no existing shim, forcing the writer path.
|
||||
Err(io::Error::from(io::ErrorKind::NotFound))
|
||||
}
|
||||
}
|
||||
impl FsReadHead for FailingWrite {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
// Empty content → no shebang, fall through to extension.
|
||||
Ok(0)
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for FailingWrite {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsWrite for FailingWrite {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for FailingWrite {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for FailingWrite {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for FailingWrite {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let manifest = serde_json::json!({"name": "foo", "bin": "cli.js"});
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("foo");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
write_file(pkg.join("cli.js"), "").unwrap();
|
||||
let err = link_bins_of_packages::<FailingWrite>(
|
||||
&[PackageBinSource { location: pkg, manifest: Arc::new(manifest) }],
|
||||
&tmp.path().join(".bin"),
|
||||
)
|
||||
.expect_err("write error must propagate");
|
||||
assert!(matches!(err, LinkBinsError::WriteShim { .. }));
|
||||
}
|
||||
|
||||
/// [`link_bins_of_packages`] propagates a chmod failure on the shim.
|
||||
#[test]
|
||||
fn link_bins_propagates_chmod_error_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct FailingChmod;
|
||||
impl FsReadDir for FailingChmod {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
impl FsReadFile for FailingChmod {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadString for FailingChmod {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
Err(io::Error::from(io::ErrorKind::NotFound))
|
||||
}
|
||||
}
|
||||
impl FsReadHead for FailingChmod {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
Ok(0)
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for FailingChmod {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsWrite for FailingChmod {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for FailingChmod {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for FailingChmod {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for FailingChmod {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let manifest = serde_json::json!({"name": "foo", "bin": "cli.js"});
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("foo");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
write_file(pkg.join("cli.js"), "").unwrap();
|
||||
let err = link_bins_of_packages::<FailingChmod>(
|
||||
&[PackageBinSource { location: pkg, manifest: Arc::new(manifest) }],
|
||||
&tmp.path().join(".bin"),
|
||||
)
|
||||
.expect_err("chmod error must propagate");
|
||||
assert!(matches!(err, LinkBinsError::Chmod { .. }));
|
||||
}
|
||||
|
||||
/// [`super::write_shim`] propagates a non-`NotFound` IO error from
|
||||
/// [`FsSetPermissions::ensure_executable_bits`] (chmod on the *target*
|
||||
/// binary, not the shim). `NotFound` is swallowed by design, since the
|
||||
/// target may have been removed concurrently. `PermissionDenied`
|
||||
/// and friends must instead surface as [`LinkBinsError::Chmod`]. Pins
|
||||
/// the guard added in this PR (review finding #4).
|
||||
#[test]
|
||||
fn link_bins_propagates_target_chmod_error_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct FailingTargetChmod;
|
||||
impl FsReadDir for FailingTargetChmod {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
impl FsReadFile for FailingTargetChmod {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadString for FailingTargetChmod {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
Err(io::Error::from(io::ErrorKind::NotFound))
|
||||
}
|
||||
}
|
||||
impl FsReadHead for FailingTargetChmod {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
Ok(0)
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for FailingTargetChmod {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsWrite for FailingTargetChmod {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for FailingTargetChmod {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for FailingTargetChmod {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
// The target chmod returns a non-`NotFound` error; the
|
||||
// implementation must surface it rather than silently
|
||||
// dropping it.
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for FailingTargetChmod {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let manifest = serde_json::json!({"name": "foo", "bin": "cli.js"});
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("foo");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
write_file(pkg.join("cli.js"), "").unwrap();
|
||||
let err = link_bins_of_packages::<FailingTargetChmod>(
|
||||
&[PackageBinSource { location: pkg, manifest: Arc::new(manifest) }],
|
||||
&tmp.path().join(".bin"),
|
||||
)
|
||||
.expect_err("non-NotFound target chmod error must propagate as Chmod");
|
||||
assert!(matches!(err, LinkBinsError::Chmod { .. }));
|
||||
}
|
||||
|
||||
/// [`super::write_shim`] swallows `NotFound` from
|
||||
/// [`FsSetPermissions::ensure_executable_bits`] because the target may
|
||||
/// legitimately be missing (concurrent removal, race with another
|
||||
/// install). Pins this distinction so a future regression that
|
||||
/// propagates `NotFound` here would fail the test.
|
||||
#[test]
|
||||
fn link_bins_swallows_target_chmod_not_found_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct NotFoundTargetChmod;
|
||||
impl FsReadDir for NotFoundTargetChmod {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
impl FsReadFile for NotFoundTargetChmod {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadString for NotFoundTargetChmod {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
Err(io::Error::from(io::ErrorKind::NotFound))
|
||||
}
|
||||
}
|
||||
impl FsReadHead for NotFoundTargetChmod {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
Ok(0)
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for NotFoundTargetChmod {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsWrite for NotFoundTargetChmod {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for NotFoundTargetChmod {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for NotFoundTargetChmod {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
Err(io::Error::from(io::ErrorKind::NotFound))
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for NotFoundTargetChmod {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let manifest = serde_json::json!({"name": "foo", "bin": "cli.js"});
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("foo");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
write_file(pkg.join("cli.js"), "").unwrap();
|
||||
link_bins_of_packages::<NotFoundTargetChmod>(
|
||||
&[PackageBinSource { location: pkg, manifest: Arc::new(manifest) }],
|
||||
&tmp.path().join(".bin"),
|
||||
)
|
||||
.expect("NotFound on target chmod must be swallowed silently");
|
||||
}
|
||||
|
||||
/// [`link_bins_of_packages`] propagates a non-`NotFound` IO error from
|
||||
/// [`search_script_runtime`] (the [`LinkBinsError::ProbeShimSource`]
|
||||
/// variant). Forced via a fake [`FsReadHead`] that fails with
|
||||
/// permission-denied. The wider [`super::write_shim`] →
|
||||
/// [`search_script_runtime`] chain remains unchanged.
|
||||
#[test]
|
||||
fn link_bins_propagates_probe_shim_source_error_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct FailingProbe;
|
||||
impl FsReadDir for FailingProbe {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
impl FsReadFile for FailingProbe {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadString for FailingProbe {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadHead for FailingProbe {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for FailingProbe {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
impl FsWrite for FailingProbe {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for FailingProbe {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for FailingProbe {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for FailingProbe {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let manifest = serde_json::json!({"name": "foo", "bin": "cli.js"});
|
||||
let tmp = tempdir().unwrap();
|
||||
let pkg = tmp.path().join("foo");
|
||||
create_dir_all(&pkg).unwrap();
|
||||
let err = link_bins_of_packages::<FailingProbe>(
|
||||
&[PackageBinSource { location: pkg, manifest: Arc::new(manifest) }],
|
||||
&tmp.path().join(".bin"),
|
||||
)
|
||||
.expect_err("probe error must propagate");
|
||||
assert!(matches!(err, LinkBinsError::ProbeShimSource { .. }));
|
||||
}
|
||||
|
||||
/// [`link_bins`] propagates a non-`NotFound` IO error from reading a
|
||||
/// child `package.json` (the [`LinkBinsError::ReadManifest`] variant).
|
||||
/// Forced via a fake [`FsReadFile`] that always returns
|
||||
/// `PermissionDenied`.
|
||||
#[test]
|
||||
fn link_bins_propagates_read_manifest_error_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct DenyManifestRead;
|
||||
impl FsReadDir for DenyManifestRead {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Ok(vec!["foo".into()].into_iter())
|
||||
}
|
||||
}
|
||||
impl FsReadFile for DenyManifestRead {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsReadString for DenyManifestRead {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadHead for DenyManifestRead {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for DenyManifestRead {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWrite for DenyManifestRead {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for DenyManifestRead {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for DenyManifestRead {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for DenyManifestRead {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let err = link_bins::<DenyManifestRead>(Path::new("/x"), Path::new("/x/.bin"))
|
||||
.expect_err("read_manifest error must propagate");
|
||||
assert!(matches!(err, LinkBinsError::ReadManifest { .. }));
|
||||
}
|
||||
|
||||
/// [`super::pick_winner`] `(true, false)` arm. Existing owns, candidate
|
||||
/// doesn't, so existing wins. The other arm (`(false, true)`) is
|
||||
/// covered by `ownership_breaks_bin_conflicts` further down.
|
||||
///
|
||||
/// Uses `aaa-other` (lexically less than `npm`) as the non-owner so
|
||||
/// the test fails when ownership is broken: with the rule disabled
|
||||
/// the lexical fallback picks `aaa-other`, the assertion observes
|
||||
/// `/aaa-other/npx` instead of `/npm/npx`. A package named `other`
|
||||
/// would lexically lose to `npm` regardless, masking the regression.
|
||||
#[test]
|
||||
fn ownership_breaks_bin_conflicts_when_existing_owns() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let aaa_other = tmp.path().join("aaa-other");
|
||||
let npm = tmp.path().join("npm");
|
||||
for d in [&aaa_other, &npm] {
|
||||
create_dir_all(d).unwrap();
|
||||
write_file(d.join("npx"), "#!/usr/bin/env node\n").unwrap();
|
||||
}
|
||||
write_file(npm.join("package.json"), json!({"name": "npm", "bin": {"npx": "npx"}}).to_string())
|
||||
.unwrap();
|
||||
write_file(
|
||||
aaa_other.join("package.json"),
|
||||
json!({"name": "aaa-other", "bin": {"npx": "npx"}}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let manifest_other: Value =
|
||||
serde_json::from_slice(&read_file(aaa_other.join("package.json")).unwrap()).unwrap();
|
||||
let manifest_npm: Value =
|
||||
serde_json::from_slice(&read_file(npm.join("package.json")).unwrap()).unwrap();
|
||||
|
||||
// Order npm-first; this exercises the (true, false) arm because
|
||||
// `npm` (existing) owns and `aaa-other` (candidate) doesn't.
|
||||
let bins = tmp.path().join(".bin");
|
||||
link_bins_of_packages::<RealApi>(
|
||||
&[
|
||||
PackageBinSource { location: npm.clone(), manifest: Arc::new(manifest_npm) },
|
||||
PackageBinSource { location: aaa_other.clone(), manifest: Arc::new(manifest_other) },
|
||||
],
|
||||
&bins,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let body = read_to_string(bins.join("npx")).unwrap();
|
||||
assert!(body.contains("/npm/npx"), "existing-owns winner must be `npm`, body:\n{body}");
|
||||
}
|
||||
|
||||
/// [`link_bins`] propagates a non-`NotFound` `read_dir` error on
|
||||
/// `<modules_dir>` itself. Real fs can't trigger this portably; the
|
||||
/// fake forces the variant.
|
||||
#[test]
|
||||
fn link_bins_propagates_modules_dir_read_error_via_di() {
|
||||
use std::io;
|
||||
|
||||
struct FailingModulesRead;
|
||||
impl FsReadDir for FailingModulesRead {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Err::<Empty<PathBuf>, _>(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsReadFile for FailingModulesRead {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadString for FailingModulesRead {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadHead for FailingModulesRead {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for FailingModulesRead {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWrite for FailingModulesRead {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for FailingModulesRead {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for FailingModulesRead {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for FailingModulesRead {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let err = link_bins::<FailingModulesRead>(Path::new("/x"), Path::new("/x/.bin"))
|
||||
.expect_err("read_dir error must propagate");
|
||||
eprintln!("link_bins_propagates_modules_dir_read_error err={err:?}");
|
||||
assert!(matches!(err, LinkBinsError::ReadModulesDir { .. }));
|
||||
}
|
||||
|
||||
/// Conflict resolution: when two packages declare the same bin name, the
|
||||
/// owning package wins.
|
||||
///
|
||||
/// Uses `aaa-other` (lexically less than `npm`) as the non-owner so the
|
||||
/// test fails when ownership is broken: with the rule disabled the
|
||||
/// lexical fallback picks `aaa-other`, the assertion observes
|
||||
/// `/aaa-other/npx` instead of `/npm/npx`. A package named `other`
|
||||
/// would lexically lose to `npm` regardless, masking the regression.
|
||||
#[test]
|
||||
fn ownership_breaks_bin_conflicts() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let npm = tmp.path().join("npm");
|
||||
let aaa_other = tmp.path().join("aaa-other");
|
||||
for d in [&npm, &aaa_other] {
|
||||
create_dir_all(d).unwrap();
|
||||
write_file(d.join("npx"), "#!/usr/bin/env node\n").unwrap();
|
||||
}
|
||||
write_file(npm.join("package.json"), json!({"name": "npm", "bin": {"npx": "npx"}}).to_string())
|
||||
.unwrap();
|
||||
write_file(
|
||||
aaa_other.join("package.json"),
|
||||
json!({"name": "aaa-other", "bin": {"npx": "npx"}}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let manifest_npm: Value =
|
||||
serde_json::from_slice(&read_file(npm.join("package.json")).unwrap()).unwrap();
|
||||
let manifest_other: Value =
|
||||
serde_json::from_slice(&read_file(aaa_other.join("package.json")).unwrap()).unwrap();
|
||||
|
||||
let bins = tmp.path().join(".bin");
|
||||
link_bins_of_packages::<RealApi>(
|
||||
&[
|
||||
PackageBinSource { location: aaa_other.clone(), manifest: Arc::new(manifest_other) },
|
||||
PackageBinSource { location: npm.clone(), manifest: Arc::new(manifest_npm) },
|
||||
],
|
||||
&bins,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let body = read_to_string(bins.join("npx")).unwrap();
|
||||
// npm's `npx` lives at `<npm>/npx`; the shim must reference that path.
|
||||
assert!(
|
||||
body.contains("/npm/npx") || is_shim_pointing_at(&body, &npm.join("npx")),
|
||||
"ownership-aware resolution should pick npm's npx, body:\n{body}",
|
||||
);
|
||||
}
|
||||
50
pacquet/crates/cmd-shim/src/path_util.rs
Normal file
50
pacquet/crates/cmd-shim/src/path_util.rs
Normal file
@@ -0,0 +1,50 @@
|
||||
//! Path-manipulation helpers shared across [`crate::bin_resolver`] and
|
||||
//! [`crate::shim`]. Kept private to the crate.
|
||||
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
|
||||
/// Lexically resolve `.` and `..` in `path` without touching the filesystem.
|
||||
///
|
||||
/// `.` (CurDir) components are dropped. `..` (ParentDir) components pop
|
||||
/// the previous component when one exists. Otherwise the rule depends
|
||||
/// on whether `out` is already anchored:
|
||||
///
|
||||
/// - Anchored (`out` has a root or a Windows `Prefix`): drop the `..`.
|
||||
/// Node's `path.resolve("/a/../../b")` returns `/b`, not `/../b`, so
|
||||
/// the lexical normalisation has to match for `is_subdir` containment
|
||||
/// checks (a `starts_with` against the package root) to agree with
|
||||
/// pnpm. Without this branch, a path like `<pkg>/x/../../bin.js`
|
||||
/// would normalise to `/../<pkg>/bin.js` and be rejected as outside
|
||||
/// `<pkg>` even when it resolves back inside.
|
||||
/// - Unanchored: push `..` so a leading `..` survives. Relative
|
||||
/// targets like `../shared/cli` need this for
|
||||
/// `shim::relative_path_from`.
|
||||
///
|
||||
/// Filesystem-free is the whole point: callers in `bin_resolver::is_subdir`
|
||||
/// and `shim::relative_path_from` run the check before the target files
|
||||
/// exist on disk, where `std::fs::canonicalize` cannot help. Mirrors pnpm's
|
||||
/// `is-subdir`, which is also purely lexical (it uses Node's
|
||||
/// `path.resolve`).
|
||||
pub(crate) fn lexical_normalize(path: &Path) -> PathBuf {
|
||||
let mut out = PathBuf::new();
|
||||
for component in path.components() {
|
||||
match component {
|
||||
Component::ParentDir => {
|
||||
if !out.pop() && !is_anchored(&out) {
|
||||
out.push("..");
|
||||
}
|
||||
}
|
||||
Component::CurDir => {}
|
||||
other => out.push(other.as_os_str()),
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Whether `path` is anchored to a filesystem root: it either starts
|
||||
/// with a Unix `/` root or a Windows `Prefix` (drive letter, UNC
|
||||
/// share). Used by [`lexical_normalize`] to decide whether a `..` that
|
||||
/// cannot pop should be materialised or dropped.
|
||||
fn is_anchored(path: &Path) -> bool {
|
||||
path.has_root() || matches!(path.components().next(), Some(Component::Prefix(_)))
|
||||
}
|
||||
398
pacquet/crates/cmd-shim/src/shim.rs
Normal file
398
pacquet/crates/cmd-shim/src/shim.rs
Normal file
@@ -0,0 +1,398 @@
|
||||
use crate::{capabilities::FsReadHead, path_util::lexical_normalize};
|
||||
use std::{
|
||||
io,
|
||||
path::{Path, PathBuf},
|
||||
};
|
||||
|
||||
/// Detected runtime for a target script.
|
||||
///
|
||||
/// Mirrors the return shape of `searchScriptRuntime` in
|
||||
/// <https://github.com/pnpm/cmd-shim/blob/0d79ca9534/src/index.ts>.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ScriptRuntime {
|
||||
/// The interpreter to invoke. `None` means "exec the file directly".
|
||||
pub prog: Option<String>,
|
||||
/// Extra arguments declared after the interpreter in the shebang. Empty
|
||||
/// when the runtime came from the extension fallback.
|
||||
pub args: String,
|
||||
}
|
||||
|
||||
/// Map of file extensions to their default runtime when the script lacks a
|
||||
/// shebang. Mirrors `extensionToProgramMap` in upstream cmd-shim.
|
||||
fn extension_program(extension: &str) -> Option<&'static str> {
|
||||
match extension {
|
||||
"js" | "cjs" | "mjs" => Some("node"),
|
||||
"cmd" | "bat" => Some("cmd"),
|
||||
"ps1" => Some("pwsh"),
|
||||
"sh" => Some("sh"),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Read up to 512 bytes of `path` and infer the runtime.
|
||||
///
|
||||
/// Order, mirroring `searchScriptRuntime`:
|
||||
///
|
||||
/// 1. If the file exists and starts with a shebang, parse `prog` + `args` from
|
||||
/// it.
|
||||
/// 2. Otherwise look up a default runtime by file extension (e.g. `.js` →
|
||||
/// `node`, `.cmd` → `cmd`).
|
||||
/// 3. If neither yields a runtime, return `None`. [`generate_sh_shim`]
|
||||
/// handles that by exec'ing the target directly.
|
||||
///
|
||||
/// `NotFound` reading the file degrades to `Ok(None)` so a missing-bin race
|
||||
/// doesn't fail the whole install. Other IO errors propagate, since pacquet
|
||||
/// has already verified the bin path resolves under the package root by
|
||||
/// this point and a real failure deserves to surface.
|
||||
pub fn search_script_runtime<Api: FsReadHead>(path: &Path) -> io::Result<Option<ScriptRuntime>> {
|
||||
let extension = path.extension().and_then(|s| s.to_str()).unwrap_or("");
|
||||
|
||||
let runtime_from_shebang = read_shebang::<Api>(path)?;
|
||||
if let Some(rt) = runtime_from_shebang {
|
||||
return Ok(Some(rt));
|
||||
}
|
||||
|
||||
if let Some(prog) = extension_program(extension) {
|
||||
return Ok(Some(ScriptRuntime { prog: Some(prog.to_string()), args: String::new() }));
|
||||
}
|
||||
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
fn read_shebang<Api: FsReadHead>(path: &Path) -> io::Result<Option<ScriptRuntime>> {
|
||||
let mut buffer = [0u8; 512];
|
||||
let read = match read_head_filled::<Api>(path, &mut buffer) {
|
||||
Ok(read) => read,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None),
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
Ok(parse_shebang_from_bytes(&buffer[..read]))
|
||||
}
|
||||
|
||||
/// Read up to `buf.len()` bytes from `path` into `buf`, looping over
|
||||
/// the [`FsReadHead`] capability until either the buffer is full or
|
||||
/// the underlying read returns 0 (EOF). Returns the number of bytes
|
||||
/// actually filled (which can be `< buf.len()` for a short file).
|
||||
///
|
||||
/// [`FsReadHead::read_head`] mirrors a single `read(2)` syscall, which
|
||||
/// POSIX permits to return short. This loop collects short reads so
|
||||
/// the shebang parser sees a complete view of the head of the file
|
||||
/// even on pseudo-fs paths (`/proc`, `/sys`, FUSE, …) where short
|
||||
/// reads are common. On regular files at offset 0 the underlying
|
||||
/// `read` returns the whole prefix in one syscall, so the loop adds
|
||||
/// no extra syscalls in the hot path. The cost is one extra branch.
|
||||
///
|
||||
/// Kept generic over [`FsReadHead`] so tests can plug in a fake that
|
||||
/// deliberately returns short and verify the loop accumulates
|
||||
/// correctly.
|
||||
pub fn read_head_filled<Api: FsReadHead>(path: &Path, buf: &mut [u8]) -> io::Result<usize> {
|
||||
let mut total = 0;
|
||||
while total < buf.len() {
|
||||
match Api::read_head(path, total as u64, &mut buf[total..])? {
|
||||
0 => break, // EOF
|
||||
n => total += n,
|
||||
}
|
||||
}
|
||||
Ok(total)
|
||||
}
|
||||
|
||||
/// Parse the runtime out of the first line of a script's content. Pure
|
||||
/// function over bytes so the caller can plug in any I/O strategy.
|
||||
///
|
||||
/// Does **not** trim leading whitespace before looking for `#!`. The
|
||||
/// kernel and upstream cmd-shim both treat `#!` as a shebang only when
|
||||
/// it sits at byte 0 of the file; an earlier
|
||||
/// `String::from_utf8_lossy(bytes).trim_start()` accepted inputs like
|
||||
/// `" \n#!/usr/bin/env node"` as a valid shebang and could select the
|
||||
/// wrong runtime for files that just happen to mention `#!` after some
|
||||
/// whitespace. The first line is taken exactly as-is (`#!` is matched
|
||||
/// at column 0 of that line via `strip_prefix` in `parse_shebang`).
|
||||
pub fn parse_shebang_from_bytes(bytes: &[u8]) -> Option<ScriptRuntime> {
|
||||
let head = String::from_utf8_lossy(bytes);
|
||||
let first_line = head.split('\n').next().unwrap_or("").trim_end_matches('\r');
|
||||
parse_shebang(first_line)
|
||||
}
|
||||
|
||||
/// Mirrors the shebang regex in upstream cmd-shim:
|
||||
/// `^#!\s*(?:/usr/bin/env(?:\s+-S\s*)?)?\s*([^ \t]+)(.*)$`.
|
||||
///
|
||||
/// Recognises `#!/usr/bin/env <prog>`, `#!/usr/bin/env -S <prog>`, and any
|
||||
/// direct `#!/path/to/<prog>` shebang. `args` is captured **including the
|
||||
/// leading whitespace** that separates it from `prog`. That matches
|
||||
/// upstream's regex group 2 (`(.*)`), which captures everything from after
|
||||
/// `prog`'s end-of-match to end of line. Preserving the leading whitespace
|
||||
/// is what produces the byte-identical shim text upstream emits (e.g. the
|
||||
/// double space between `$basedir/sh` and `-e` in the rendered exec line).
|
||||
fn parse_shebang(line: &str) -> Option<ScriptRuntime> {
|
||||
let rest = line.strip_prefix("#!")?.trim_start();
|
||||
let (rest, _) = strip_env_prefix(rest);
|
||||
let rest = rest.trim_start();
|
||||
|
||||
// Slice at the first space or tab; the args slice keeps the separator
|
||||
// so the rendered shim matches upstream byte-for-byte. Using `splitn`
|
||||
// would discard the separator and silently drop one space from the
|
||||
// `exec` line.
|
||||
let (prog, args) = match rest.find([' ', '\t']) {
|
||||
Some(idx) => rest.split_at(idx),
|
||||
None => (rest, ""),
|
||||
};
|
||||
|
||||
if prog.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
Some(ScriptRuntime { prog: Some(prog.to_string()), args: args.to_string() })
|
||||
}
|
||||
|
||||
/// Strip a leading `/usr/bin/env`, optionally followed by `-S`, from the
|
||||
/// shebang body. Returns the remainder and whether `env` was present.
|
||||
fn strip_env_prefix(input: &str) -> (&str, bool) {
|
||||
let Some(rest) = input.strip_prefix("/usr/bin/env") else {
|
||||
return (input, false);
|
||||
};
|
||||
let trimmed = rest.trim_start();
|
||||
if let Some(after_dash_s) = trimmed.strip_prefix("-S") {
|
||||
return (after_dash_s, true);
|
||||
}
|
||||
(trimmed, true)
|
||||
}
|
||||
|
||||
/// Generate the Unix shell-shim contents for `target_path`, written to
|
||||
/// `shim_path`. Mirrors `generateShShim` in upstream cmd-shim.
|
||||
///
|
||||
/// The shim is a pure `/bin/sh` script that:
|
||||
///
|
||||
/// 1. Resolves `basedir` to its own directory (with a `cygpath` fixup for
|
||||
/// MSYS-style POSIX shells on Windows).
|
||||
/// 2. If the runtime program is colocated at `$basedir/<prog>` (a rare case,
|
||||
/// only true when the runtime was bundled alongside the shim), prefer that
|
||||
/// binary; otherwise fall through to the system PATH.
|
||||
/// 3. Forwards `"$@"` to the resolved interpreter, with the target script as
|
||||
/// the first positional argument.
|
||||
///
|
||||
/// When [`search_script_runtime`] returned `None` (no shebang, unknown
|
||||
/// extension), the shim execs the target directly via the second branch
|
||||
/// upstream uses for that case.
|
||||
pub fn generate_sh_shim(
|
||||
target_path: &Path,
|
||||
shim_path: &Path,
|
||||
runtime: Option<&ScriptRuntime>,
|
||||
) -> String {
|
||||
let mut sh = String::from(SH_SHIM_HEADER);
|
||||
|
||||
let sh_target = relative_target(target_path, shim_path);
|
||||
let quoted_target = if Path::new(&sh_target).is_absolute() {
|
||||
format!("\"{sh_target}\"")
|
||||
} else {
|
||||
format!("\"$basedir/{sh_target}\"")
|
||||
};
|
||||
|
||||
match runtime {
|
||||
Some(ScriptRuntime { prog: Some(prog), args }) => {
|
||||
// `sh_long_prog` is the `"$basedir/<prog>"` form upstream uses.
|
||||
// It always carries the leading `$basedir/` and quotes; never
|
||||
// just the program name on its own.
|
||||
let sh_long_prog = format!("\"$basedir/{prog}\"");
|
||||
sh.push_str(&format!(
|
||||
"if [ -x {sh_long_prog} ]; then\n exec {sh_long_prog} {args} {quoted_target} \"$@\"\nelse\n exec {prog} {args} {quoted_target} \"$@\"\nfi\n",
|
||||
));
|
||||
}
|
||||
// No runtime detected, so exec the target directly. Upstream still
|
||||
// emits `exit $?` on this branch for parity with non-execve POSIX
|
||||
// shells.
|
||||
runtime_opt => {
|
||||
let args = runtime_opt.map(|r| r.args.as_str()).unwrap_or("");
|
||||
sh.push_str(&format!("{quoted_target} {args} \"$@\"\nexit $?\n"));
|
||||
}
|
||||
}
|
||||
|
||||
sh.push_str(&format!("# {}\n", shim_target_marker(target_path)));
|
||||
sh
|
||||
}
|
||||
|
||||
/// Generate the Windows `.cmd` shim contents for `target_path`. Mirrors
|
||||
/// `generateCmdShim` in upstream cmd-shim. Pacquet skips the
|
||||
/// `nodePath`/`prependToPath`/`nodeExecPath`/`progArgs` features that
|
||||
/// upstream supports; we only ever write a "plain" cmd shim.
|
||||
///
|
||||
/// CRLF line endings are part of the on-disk contract for `.cmd` files
|
||||
/// on Windows, so the template uses literal `\r\n`.
|
||||
pub fn generate_cmd_shim(
|
||||
target_path: &Path,
|
||||
shim_path: &Path,
|
||||
runtime: Option<&ScriptRuntime>,
|
||||
) -> String {
|
||||
let cmd_target_rel = relative_target_windows(target_path, shim_path);
|
||||
let quoted_target = if Path::new(&cmd_target_rel).is_absolute() {
|
||||
format!("\"{cmd_target_rel}\"")
|
||||
} else {
|
||||
format!("\"%~dp0\\{cmd_target_rel}\"")
|
||||
};
|
||||
|
||||
let mut cmd = String::from("@SETLOCAL\r\n");
|
||||
|
||||
match runtime {
|
||||
Some(ScriptRuntime { prog: Some(prog), args }) => {
|
||||
let long_prog = format!("\"%~dp0\\{prog}.exe\"");
|
||||
cmd.push_str(&format!(
|
||||
"@IF EXIST {long_prog} (\r\n {long_prog} {args} {quoted_target} %*\r\n) ELSE (\r\n @SET PATHEXT=%PATHEXT:;.JS;=;%\r\n {prog} {args} {quoted_target} %*\r\n)\r\n",
|
||||
));
|
||||
}
|
||||
runtime_opt => {
|
||||
let args = runtime_opt.map(|r| r.args.as_str()).unwrap_or("");
|
||||
// No runtime detected, so exec the target directly.
|
||||
cmd.push_str(&format!("@{quoted_target} {args} %*\r\n"));
|
||||
}
|
||||
}
|
||||
|
||||
cmd
|
||||
}
|
||||
|
||||
/// Generate the cross-shell PowerShell `.ps1` shim contents for
|
||||
/// `target_path`. Mirrors `generatePwshShim` in upstream cmd-shim,
|
||||
/// minus the `nodePath`/`prependToPath`/`nodeExecPath`/`progArgs`
|
||||
/// branches we don't use. The shim self-detects Windows vs. POSIX-ish
|
||||
/// pwsh and adjusts the executable suffix accordingly.
|
||||
pub fn generate_pwsh_shim(
|
||||
target_path: &Path,
|
||||
shim_path: &Path,
|
||||
runtime: Option<&ScriptRuntime>,
|
||||
) -> String {
|
||||
let sh_target = relative_target(target_path, shim_path);
|
||||
let quoted_target = if Path::new(&sh_target).is_absolute() {
|
||||
format!("\"{sh_target}\"")
|
||||
} else {
|
||||
format!("\"$basedir/{sh_target}\"")
|
||||
};
|
||||
|
||||
use std::fmt::Write;
|
||||
let mut pwsh = String::from(PWSH_SHIM_HEADER);
|
||||
|
||||
match runtime {
|
||||
Some(ScriptRuntime { prog: Some(prog), args }) => {
|
||||
let long_prog = format!(r#""$basedir/{prog}$exe""#);
|
||||
let prog_quoted = format!(r#""{prog}$exe""#);
|
||||
writeln!(pwsh).unwrap();
|
||||
writeln!(pwsh, "$ret=0").unwrap();
|
||||
writeln!(pwsh, "if (Test-Path {long_prog}) {{").unwrap();
|
||||
writeln!(pwsh, " # Support pipeline input").unwrap();
|
||||
writeln!(pwsh, " if ($MyInvocation.ExpectingInput) {{").unwrap();
|
||||
writeln!(pwsh, " $input | & {long_prog} {args} {quoted_target} $args").unwrap();
|
||||
writeln!(pwsh, " }} else {{").unwrap();
|
||||
writeln!(pwsh, " & {long_prog} {args} {quoted_target} $args").unwrap();
|
||||
writeln!(pwsh, " }}").unwrap();
|
||||
writeln!(pwsh, " $ret=$LASTEXITCODE").unwrap();
|
||||
writeln!(pwsh, "}} else {{").unwrap();
|
||||
writeln!(pwsh, " # Support pipeline input").unwrap();
|
||||
writeln!(pwsh, " if ($MyInvocation.ExpectingInput) {{").unwrap();
|
||||
writeln!(pwsh, " $input | & {prog_quoted} {args} {quoted_target} $args").unwrap();
|
||||
writeln!(pwsh, " }} else {{").unwrap();
|
||||
writeln!(pwsh, " & {prog_quoted} {args} {quoted_target} $args").unwrap();
|
||||
writeln!(pwsh, " }}").unwrap();
|
||||
writeln!(pwsh, " $ret=$LASTEXITCODE").unwrap();
|
||||
writeln!(pwsh, "}}").unwrap();
|
||||
writeln!(pwsh, "exit $ret").unwrap();
|
||||
}
|
||||
runtime_opt => {
|
||||
let args = runtime_opt.map(|r| r.args.as_str()).unwrap_or("");
|
||||
writeln!(pwsh).unwrap();
|
||||
writeln!(pwsh, "# Support pipeline input").unwrap();
|
||||
writeln!(pwsh, "if ($MyInvocation.ExpectingInput) {{").unwrap();
|
||||
writeln!(pwsh, " $input | & {quoted_target} {args} $args").unwrap();
|
||||
writeln!(pwsh, "}} else {{").unwrap();
|
||||
writeln!(pwsh, " & {quoted_target} {args} $args").unwrap();
|
||||
writeln!(pwsh, "}}").unwrap();
|
||||
writeln!(pwsh, "exit $LASTEXITCODE").unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
pwsh
|
||||
}
|
||||
|
||||
/// `.ps1` template prelude. Sets up `$basedir` and `$exe` exactly like
|
||||
/// upstream's `generatePwshShim`.
|
||||
const PWSH_SHIM_HEADER: &str = r#"#!/usr/bin/env pwsh
|
||||
$basedir=Split-Path $MyInvocation.MyCommand.Definition -Parent
|
||||
|
||||
$exe=""
|
||||
if ($PSVersionTable.PSVersion -lt "6.0" -or $IsWindows) {
|
||||
# Fix case when both the Windows and Linux builds of Node
|
||||
# are installed in the same directory
|
||||
$exe=".exe"
|
||||
}"#;
|
||||
|
||||
/// Compute the Windows-style relative path from `shim_path`'s parent
|
||||
/// directory to `target_path`. The `.cmd` shim uses backslashes, so we
|
||||
/// convert the lexical-relative result. Falls back to the absolute path
|
||||
/// if the relative computation fails. Same shape as
|
||||
/// [`relative_target`] but with the slash direction flipped.
|
||||
fn relative_target_windows(target_path: &Path, shim_path: &Path) -> String {
|
||||
let shim_dir = shim_path.parent().unwrap_or_else(|| Path::new(""));
|
||||
let rel = relative_path_from(shim_dir, target_path);
|
||||
rel.to_string_lossy().replace('/', "\\")
|
||||
}
|
||||
|
||||
const SH_SHIM_HEADER: &str = r#"#!/bin/sh
|
||||
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
|
||||
|
||||
case `uname` in
|
||||
*CYGWIN*|*MINGW*|*MSYS*)
|
||||
if command -v cygpath > /dev/null 2>&1; then
|
||||
basedir=`cygpath -w "$basedir"`
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
"#;
|
||||
|
||||
/// Trailing `# cmd-shim-target=<rel>` marker. Upstream uses it to detect
|
||||
/// whether an existing shim already targets the same source without
|
||||
/// re-parsing its body. Pacquet uses [`is_shim_pointing_at`] for the same
|
||||
/// short-circuit on warm reinstalls.
|
||||
fn shim_target_marker(target_path: &Path) -> String {
|
||||
format!("cmd-shim-target={}", target_path.to_string_lossy().replace('\\', "/"))
|
||||
}
|
||||
|
||||
/// Whether an already-on-disk shim targets `target_path`. Mirrors
|
||||
/// `isShimPointingAt`. The check looks for the trailing marker line so the
|
||||
/// header text never has to be byte-identical between cmd-shim versions.
|
||||
pub fn is_shim_pointing_at(shim_content: &str, target_path: &Path) -> bool {
|
||||
let marker = format!("# {}", shim_target_marker(target_path));
|
||||
shim_content.lines().any(|line| line == marker)
|
||||
}
|
||||
|
||||
/// Compute the relative path from `shim_path`'s parent directory to
|
||||
/// `target_path`. Falls back to the absolute target path if the relative
|
||||
/// computation fails. That matches the `path.isAbsolute(shTarget)` guard in
|
||||
/// upstream's `generateShShim`.
|
||||
fn relative_target(target_path: &Path, shim_path: &Path) -> String {
|
||||
let shim_dir = shim_path.parent().unwrap_or_else(|| Path::new(""));
|
||||
let rel = relative_path_from(shim_dir, target_path);
|
||||
rel.to_string_lossy().replace('\\', "/")
|
||||
}
|
||||
|
||||
fn relative_path_from(from: &Path, to: &Path) -> PathBuf {
|
||||
let from = lexical_normalize(from);
|
||||
let to = lexical_normalize(to);
|
||||
|
||||
let from_components: Vec<_> = from.components().collect();
|
||||
let to_components: Vec<_> = to.components().collect();
|
||||
|
||||
let common =
|
||||
from_components.iter().zip(to_components.iter()).take_while(|(a, b)| a == b).count();
|
||||
|
||||
let mut result = PathBuf::new();
|
||||
for _ in &from_components[common..] {
|
||||
result.push("..");
|
||||
}
|
||||
for component in &to_components[common..] {
|
||||
result.push(component.as_os_str());
|
||||
}
|
||||
if result.as_os_str().is_empty() {
|
||||
result.push(".");
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
547
pacquet/crates/cmd-shim/src/shim/tests.rs
Normal file
547
pacquet/crates/cmd-shim/src/shim/tests.rs
Normal file
@@ -0,0 +1,547 @@
|
||||
use super::{
|
||||
ScriptRuntime, extension_program, generate_cmd_shim, generate_pwsh_shim, generate_sh_shim,
|
||||
is_shim_pointing_at, parse_shebang, parse_shebang_from_bytes, read_head_filled,
|
||||
relative_target, search_script_runtime,
|
||||
};
|
||||
use crate::capabilities::{FsReadHead, RealApi};
|
||||
use crate::path_util::lexical_normalize;
|
||||
use std::{
|
||||
io,
|
||||
path::{Path, PathBuf},
|
||||
};
|
||||
|
||||
#[test]
|
||||
fn parses_env_node_shebang() {
|
||||
let rt = parse_shebang("#!/usr/bin/env node").unwrap();
|
||||
assert_eq!(rt.prog.as_deref(), Some("node"));
|
||||
assert_eq!(rt.args, "");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_env_dash_s_shebang() {
|
||||
let rt = parse_shebang("#!/usr/bin/env -S node --experimental").unwrap();
|
||||
assert_eq!(rt.prog.as_deref(), Some("node"));
|
||||
// Leading space is preserved because upstream's regex group 2
|
||||
// captures the separator. Pinning `" --experimental"` (with the
|
||||
// space) is what makes the rendered shim's `exec` line match
|
||||
// upstream byte-for-byte.
|
||||
assert_eq!(rt.args, " --experimental");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_direct_shebang() {
|
||||
let rt = parse_shebang("#!/bin/sh -e").unwrap();
|
||||
assert_eq!(rt.prog.as_deref(), Some("/bin/sh"));
|
||||
// Leading space preserved (see `parses_env_dash_s_shebang`).
|
||||
assert_eq!(rt.args, " -e");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_non_shebang_lines() {
|
||||
assert!(parse_shebang("just text").is_none());
|
||||
assert!(parse_shebang("#! ").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_fallback_picks_node_for_js() {
|
||||
assert_eq!(extension_program("js"), Some("node"));
|
||||
assert_eq!(extension_program("cjs"), Some("node"));
|
||||
assert_eq!(extension_program("mjs"), Some("node"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn relative_target_traverses_into_sibling_package() {
|
||||
// shim at .../node_modules/.bin/cli; target at .../node_modules/foo/bin/cli.js
|
||||
let target = Path::new("/proj/node_modules/foo/bin/cli.js");
|
||||
let shim = Path::new("/proj/node_modules/.bin/cli");
|
||||
assert_eq!(relative_target(target, shim), "../foo/bin/cli.js");
|
||||
}
|
||||
|
||||
/// Shim body for the typical `#!/usr/bin/env node` case must match the
|
||||
/// exec template upstream produces verbatim, including the double space
|
||||
/// between `$basedir/node` and the quoted target path (upstream's
|
||||
/// `${args}` interpolates to empty between two literal spaces).
|
||||
#[test]
|
||||
fn generate_sh_shim_matches_pnpm_typical_case() {
|
||||
let target = Path::new("/proj/node_modules/typescript/bin/tsc");
|
||||
let shim = Path::new("/proj/node_modules/.bin/tsc");
|
||||
let runtime = ScriptRuntime { prog: Some("node".into()), args: String::new() };
|
||||
let body = generate_sh_shim(target, shim, Some(&runtime));
|
||||
|
||||
assert!(body.starts_with("#!/bin/sh\n"), "shebang must come first");
|
||||
assert!(
|
||||
body.contains("if [ -x \"$basedir/node\" ]; then\n exec \"$basedir/node\" \"$basedir/../typescript/bin/tsc\" \"$@\"\nelse\n exec node \"$basedir/../typescript/bin/tsc\" \"$@\"\nfi\n"),
|
||||
"exec block must match pnpm's generateShShim template, body was:\n{body}",
|
||||
);
|
||||
assert!(
|
||||
body.ends_with("# cmd-shim-target=/proj/node_modules/typescript/bin/tsc\n"),
|
||||
"trailing target marker is required for is_shim_pointing_at parity",
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn is_shim_pointing_at_round_trips_through_marker() {
|
||||
let target = Path::new("/p/node_modules/typescript/bin/tsc");
|
||||
let shim = Path::new("/p/node_modules/.bin/tsc");
|
||||
let runtime = ScriptRuntime { prog: Some("node".into()), args: String::new() };
|
||||
let body = generate_sh_shim(target, shim, Some(&runtime));
|
||||
assert!(is_shim_pointing_at(&body, target));
|
||||
assert!(!is_shim_pointing_at(&body, Path::new("/elsewhere")));
|
||||
}
|
||||
|
||||
/// Every fallback extension upstream's `extensionToProgramMap` recognises
|
||||
/// must round-trip. Guards against silent regression if an extension is
|
||||
/// removed from the table.
|
||||
#[test]
|
||||
fn extension_program_covers_every_known_extension() {
|
||||
assert_eq!(extension_program("js"), Some("node"));
|
||||
assert_eq!(extension_program("cjs"), Some("node"));
|
||||
assert_eq!(extension_program("mjs"), Some("node"));
|
||||
assert_eq!(extension_program("cmd"), Some("cmd"));
|
||||
assert_eq!(extension_program("bat"), Some("cmd"));
|
||||
assert_eq!(extension_program("ps1"), Some("pwsh"));
|
||||
assert_eq!(extension_program("sh"), Some("sh"));
|
||||
assert_eq!(extension_program("unknown"), None);
|
||||
assert_eq!(extension_program(""), None);
|
||||
}
|
||||
|
||||
/// [`super::parse_shebang`] returns None when the line lacks `#!` entirely
|
||||
/// (handled elsewhere) or when it is `#!` with only whitespace after. The
|
||||
/// `prog.is_empty()` guard catches the latter.
|
||||
#[test]
|
||||
fn parse_shebang_returns_none_for_empty_prog() {
|
||||
assert!(parse_shebang("#!\t").is_none());
|
||||
assert!(parse_shebang("#!").is_none(), "empty line after #! must yield None");
|
||||
assert!(parse_shebang("not a shebang").is_none());
|
||||
}
|
||||
|
||||
/// [`parse_shebang_from_bytes`] is the byte-level entry. It must trim a
|
||||
/// leading BOM-free CRLF first line and survive non-UTF-8 bytes (lossy
|
||||
/// decoding).
|
||||
#[test]
|
||||
fn parse_shebang_from_bytes_handles_crlf_and_lossy_utf8() {
|
||||
let bytes = b"#!/usr/bin/env node\r\nconsole.log('hi')\n";
|
||||
let rt = parse_shebang_from_bytes(bytes).expect("CRLF first line");
|
||||
assert_eq!(rt.prog.as_deref(), Some("node"));
|
||||
|
||||
// Non-UTF-8 bytes after the shebang must not break parsing.
|
||||
let mut bytes = Vec::from(*b"#!/usr/bin/env node\n");
|
||||
bytes.extend_from_slice(&[0xff, 0xfe, 0xfd]);
|
||||
let rt = parse_shebang_from_bytes(&bytes).expect("non-UTF-8 tail tolerated");
|
||||
assert_eq!(rt.prog.as_deref(), Some("node"));
|
||||
}
|
||||
|
||||
/// [`generate_sh_shim`] with `runtime: None` emits the `exit $?` arm that
|
||||
/// upstream uses when no interpreter could be inferred. The shim execs the
|
||||
/// target directly.
|
||||
#[test]
|
||||
fn generate_sh_shim_emits_direct_exec_when_no_runtime() {
|
||||
let target = Path::new("/proj/node_modules/foo/bin/cli");
|
||||
let shim = Path::new("/proj/node_modules/.bin/cli");
|
||||
let body = generate_sh_shim(target, shim, None);
|
||||
assert!(
|
||||
body.contains("\"$basedir/../foo/bin/cli\" \"$@\"\nexit $?\n"),
|
||||
"no-runtime arm must exec the target directly, body:\n{body}",
|
||||
);
|
||||
assert!(body.ends_with("# cmd-shim-target=/proj/node_modules/foo/bin/cli\n"));
|
||||
}
|
||||
|
||||
/// [`generate_sh_shim`] with `runtime: Some(.. prog: None ..)` uses the same
|
||||
/// no-runtime arm, but threading the explicit `args`. Mirrors upstream's
|
||||
/// fallback when `prog` couldn't be inferred but the runtime probe still
|
||||
/// returned a [`ScriptRuntime`] with `prog: None`.
|
||||
#[test]
|
||||
fn generate_sh_shim_threads_args_when_prog_is_none() {
|
||||
let target = Path::new("/p/cli");
|
||||
let shim = Path::new("/p/.bin/cli");
|
||||
let runtime = ScriptRuntime { prog: None, args: "--flag".to_string() };
|
||||
let body = generate_sh_shim(target, shim, Some(&runtime));
|
||||
assert!(
|
||||
body.contains("\"$basedir/../cli\" --flag \"$@\"\nexit $?\n"),
|
||||
"args must be threaded into the no-prog arm, body:\n{body}",
|
||||
);
|
||||
}
|
||||
|
||||
/// [`generate_sh_shim`] with a target that lexically resolves to an absolute
|
||||
/// path takes the `path::isAbsolute(shTarget)` branch upstream uses. The
|
||||
/// quoted target stays absolute and skips the `$basedir/` prefix.
|
||||
///
|
||||
/// Unix-only: a path like `/abs/elsewhere/cli` is "absolute" only on Unix.
|
||||
/// On Windows, `Path::is_absolute()` requires a drive letter (e.g.
|
||||
/// `C:\abs\...`), so the same input takes the relative branch. The shim
|
||||
/// produced by pacquet is a `/bin/sh` script regardless of host platform,
|
||||
/// but the absolute-vs-relative classification of bin paths is itself
|
||||
/// platform-dependent. This test pins behavior on Unix only.
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn generate_sh_shim_uses_absolute_target_when_no_common_prefix() {
|
||||
// `relative_path_from` of two paths with no common root produces an
|
||||
// absolute-ish path that still starts with `/` once joined; force the
|
||||
// absolute branch by constructing a target that's absolute and a shim
|
||||
// whose parent is empty.
|
||||
let target = Path::new("/abs/elsewhere/cli");
|
||||
let shim = Path::new("local-shim");
|
||||
let runtime = ScriptRuntime { prog: Some("node".into()), args: String::new() };
|
||||
let body = generate_sh_shim(target, shim, Some(&runtime));
|
||||
assert!(
|
||||
body.contains("\"/abs/elsewhere/cli\""),
|
||||
"absolute-target branch must skip $basedir prefix, body:\n{body}",
|
||||
);
|
||||
}
|
||||
|
||||
/// [`super::relative_target`] of `from == to_parent` collapses to `.` (the
|
||||
/// `result.is_empty()` branch in [`super::relative_path_from`]).
|
||||
#[test]
|
||||
fn relative_target_collapses_to_dot_when_paths_share_dir() {
|
||||
let target = Path::new("/proj/.bin/cli");
|
||||
let shim = Path::new("/proj/.bin/wrapper");
|
||||
assert_eq!(relative_target(target, shim), "cli");
|
||||
}
|
||||
|
||||
/// [`super::relative_path_from`] preserves a single leading `..` in the
|
||||
/// target (the `out.push("..")` fallback fires when `out.pop()` returns
|
||||
/// false on an empty buffer). Multiple consecutive leading `..`s aren't
|
||||
/// tested because [`super::lexical_normalize`] collapses them. `PathBuf::pop`
|
||||
/// does not treat a trailing `..` component as a parent reference, so the
|
||||
/// second `..` pops the first. That edge case doesn't occur in pacquet's
|
||||
/// production paths (which are always absolute under `<modules_dir>` or
|
||||
/// `<virtual_store_dir>`), so we test only the single-`..` case where
|
||||
/// the result is unambiguous.
|
||||
///
|
||||
/// Asserting the exact value catches a regression that returns the raw
|
||||
/// target unchanged (`../shared/cli`). A weaker substring assertion
|
||||
/// would pass for both correct and broken outputs.
|
||||
#[test]
|
||||
fn lexical_normalize_keeps_leading_parent_segments() {
|
||||
let target = Path::new("../shared/cli");
|
||||
let shim = Path::new("project/.bin/cli");
|
||||
let result = relative_target(target, shim);
|
||||
assert_eq!(result, "../../../shared/cli", "leading `..` must propagate");
|
||||
}
|
||||
|
||||
/// [`lexical_normalize`] drops `.` (CurDir) components. This is a direct
|
||||
/// test on the helper itself. The indirect test below pins the same
|
||||
/// behavior at the `relative_target` level, but a direct assertion makes
|
||||
/// the CurDir arm visible to coverage tooling that can't see through
|
||||
/// inlined call chains.
|
||||
#[test]
|
||||
fn lexical_normalize_drops_curdir_segments_directly() {
|
||||
assert_eq!(lexical_normalize(Path::new("a/./b")), PathBuf::from("a/b"));
|
||||
assert_eq!(lexical_normalize(Path::new("./a/b")), PathBuf::from("a/b"));
|
||||
assert_eq!(lexical_normalize(Path::new("a/b/.")), PathBuf::from("a/b"));
|
||||
assert_eq!(lexical_normalize(Path::new("./.")), PathBuf::new());
|
||||
}
|
||||
|
||||
/// [`lexical_normalize`] discards `.` (CurDir) components silently.
|
||||
/// Verify via [`relative_target`]. A target with embedded `./`
|
||||
/// resolves the same as without.
|
||||
#[test]
|
||||
fn lexical_normalize_drops_curdir_components() {
|
||||
let with_dot = relative_target(Path::new("/p/foo/./cli"), Path::new("/p/.bin/x"));
|
||||
let without_dot = relative_target(Path::new("/p/foo/cli"), Path::new("/p/.bin/x"));
|
||||
assert_eq!(with_dot, without_dot);
|
||||
}
|
||||
|
||||
/// [`search_script_runtime`] reads a real file with a shebang and returns
|
||||
/// the prog from it. End-to-end of the production path.
|
||||
#[test]
|
||||
fn search_script_runtime_reads_shebang_from_real_file() {
|
||||
use tempfile::tempdir;
|
||||
let tmp = tempdir().unwrap();
|
||||
let path = tmp.path().join("script");
|
||||
std::fs::write(&path, "#!/usr/bin/env node\nbody\n").unwrap();
|
||||
let rt = search_script_runtime::<RealApi>(&path).unwrap().expect("runtime detected");
|
||||
assert_eq!(rt.prog.as_deref(), Some("node"));
|
||||
}
|
||||
|
||||
/// [`search_script_runtime`] on a missing file must degrade to `Ok(None)`.
|
||||
/// Otherwise the install races against bin file extraction.
|
||||
#[test]
|
||||
fn search_script_runtime_returns_none_for_missing_file() {
|
||||
let nonexistent = Path::new("/definitely/not/a/real/path/cli");
|
||||
assert_eq!(search_script_runtime::<RealApi>(nonexistent).unwrap(), None);
|
||||
}
|
||||
|
||||
/// [`search_script_runtime`] falls through to extension lookup when the
|
||||
/// file has no shebang. A `.js` file without `#!` must still resolve to
|
||||
/// `node`.
|
||||
#[test]
|
||||
fn search_script_runtime_falls_back_to_extension() {
|
||||
use tempfile::tempdir;
|
||||
let tmp = tempdir().unwrap();
|
||||
let path = tmp.path().join("script.js");
|
||||
std::fs::write(&path, "console.log('no shebang')\n").unwrap();
|
||||
let rt = search_script_runtime::<RealApi>(&path).unwrap().expect("extension fallback");
|
||||
assert_eq!(rt.prog.as_deref(), Some("node"));
|
||||
}
|
||||
|
||||
/// [`search_script_runtime`] returns `Ok(None)` when neither shebang nor
|
||||
/// extension yields a runtime. Pure no-runtime path.
|
||||
#[test]
|
||||
fn search_script_runtime_returns_none_when_runtime_unknown() {
|
||||
use tempfile::tempdir;
|
||||
let tmp = tempdir().unwrap();
|
||||
let path = tmp.path().join("script.unknown_ext");
|
||||
std::fs::write(&path, "no shebang here\n").unwrap();
|
||||
assert_eq!(search_script_runtime::<RealApi>(&path).unwrap(), None);
|
||||
}
|
||||
|
||||
/// [`search_script_runtime`] propagates IO errors that aren't `NotFound`.
|
||||
/// Real-fs can't trigger e.g. `PermissionDenied` portably, so plug a
|
||||
/// fake [`FsReadHead`] per the DI principles in
|
||||
/// <https://github.com/pnpm/pacquet/pull/332#issuecomment-4345054524>.
|
||||
#[test]
|
||||
fn search_script_runtime_propagates_non_not_found_io_errors() {
|
||||
struct PermissionDeniedApi;
|
||||
impl FsReadHead for PermissionDeniedApi {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
let err = search_script_runtime::<PermissionDeniedApi>(Path::new("any"))
|
||||
.expect_err("non-NotFound IO error must propagate");
|
||||
assert_eq!(err.kind(), io::ErrorKind::PermissionDenied);
|
||||
}
|
||||
|
||||
/// A [`FsReadHead`] that returns 0 bytes (empty file) yields no shebang.
|
||||
/// The parse step then falls through to the extension fallback. Pin the
|
||||
/// behavior so a future tweak to the empty-buffer handling stays
|
||||
/// compatible with the no-shebang case.
|
||||
#[test]
|
||||
fn search_script_runtime_reads_zero_bytes_then_falls_through() {
|
||||
struct EmptyReadApi;
|
||||
impl FsReadHead for EmptyReadApi {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
Ok(0)
|
||||
}
|
||||
}
|
||||
// `.js` extension still resolves to `node` even with empty content.
|
||||
let rt =
|
||||
search_script_runtime::<EmptyReadApi>(Path::new("/x.js")).unwrap().expect("ext fallback");
|
||||
assert_eq!(rt.prog.as_deref(), Some("node"));
|
||||
|
||||
// No extension and no shebang → Ok(None).
|
||||
let rt = search_script_runtime::<EmptyReadApi>(Path::new("/x")).unwrap();
|
||||
assert_eq!(rt, None);
|
||||
}
|
||||
|
||||
/// [`RealApi::read_head`](RealApi) is the production capability. Tests
|
||||
/// that exercise it indirectly cover most paths; this one pins the
|
||||
/// contract directly.
|
||||
#[test]
|
||||
fn real_fs_read_head_reads_up_to_buffer_size() {
|
||||
use tempfile::tempdir;
|
||||
let tmp = tempdir().unwrap();
|
||||
let path = tmp.path().join("data");
|
||||
std::fs::write(&path, "hello world").unwrap();
|
||||
let mut buf = [0u8; 1024];
|
||||
let read = RealApi::read_head(&path, 0, &mut buf).unwrap();
|
||||
assert_eq!(read, 11);
|
||||
assert_eq!(&buf[..read], b"hello world");
|
||||
}
|
||||
|
||||
/// [`RealApi::read_head`](RealApi) propagates `NotFound` so the shebang reader can
|
||||
/// distinguish a missing file from a real IO error and degrade to
|
||||
/// `Ok(None)`.
|
||||
#[test]
|
||||
fn real_fs_read_head_propagates_not_found() {
|
||||
let mut buf = [0u8; 16];
|
||||
let err = RealApi::read_head(Path::new("/no/such/file"), 0, &mut buf).unwrap_err();
|
||||
assert_eq!(err.kind(), io::ErrorKind::NotFound);
|
||||
}
|
||||
|
||||
/// [`read_head_filled`] against the real filesystem fills the buffer in
|
||||
/// one underlying syscall (the common case) and returns the exact byte
|
||||
/// count.
|
||||
#[test]
|
||||
fn read_head_filled_real_fs_long_file_fills_buffer() {
|
||||
use tempfile::tempdir;
|
||||
let tmp = tempdir().unwrap();
|
||||
let path = tmp.path().join("long");
|
||||
let payload: Vec<u8> = (0..1024).map(|i| (i % 251) as u8).collect();
|
||||
std::fs::write(&path, &payload).unwrap();
|
||||
|
||||
let mut buf = [0u8; 256];
|
||||
let read = read_head_filled::<RealApi>(&path, &mut buf).unwrap();
|
||||
assert_eq!(read, 256);
|
||||
assert_eq!(&buf[..], &payload[..256]);
|
||||
}
|
||||
|
||||
/// [`read_head_filled`] against a file shorter than the buffer returns
|
||||
/// the partial count (EOF terminates the loop without erroring).
|
||||
#[test]
|
||||
fn read_head_filled_real_fs_short_file_returns_partial() {
|
||||
use tempfile::tempdir;
|
||||
let tmp = tempdir().unwrap();
|
||||
let path = tmp.path().join("short");
|
||||
std::fs::write(&path, "#!/bin/sh\n").unwrap();
|
||||
|
||||
let mut buf = [0u8; 256];
|
||||
let read = read_head_filled::<RealApi>(&path, &mut buf).unwrap();
|
||||
assert_eq!(read, 10);
|
||||
assert_eq!(&buf[..read], b"#!/bin/sh\n");
|
||||
}
|
||||
|
||||
/// [`read_head_filled`] accumulates short reads from the underlying
|
||||
/// capability. That is the very behaviour the loop exists to provide.
|
||||
/// A fake that always returns short proves the loop calls the trait
|
||||
/// repeatedly with advancing offsets until the buffer is full.
|
||||
///
|
||||
/// Pinning this with a fake is the only way to verify the loop
|
||||
/// without a pseudo-fs to test against: real filesystems essentially
|
||||
/// never return short reads at offset 0.
|
||||
#[test]
|
||||
fn read_head_filled_accumulates_short_reads_from_fake() {
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
|
||||
/// Tracks the offsets each call sees, plus how many bytes the
|
||||
/// fake produces per call. We deliver the input slice to the
|
||||
/// caller `chunk_size` bytes at a time so the loop must run
|
||||
/// multiple iterations to fill its buffer.
|
||||
static CALL_COUNT: AtomicUsize = AtomicUsize::new(0);
|
||||
static LAST_OFFSETS: [AtomicUsize; 4] = [
|
||||
AtomicUsize::new(usize::MAX),
|
||||
AtomicUsize::new(usize::MAX),
|
||||
AtomicUsize::new(usize::MAX),
|
||||
AtomicUsize::new(usize::MAX),
|
||||
];
|
||||
const PAYLOAD: &[u8] = b"abcdefghij"; // 10 bytes
|
||||
const CHUNK_SIZE: usize = 3;
|
||||
|
||||
struct ShortReader;
|
||||
impl FsReadHead for ShortReader {
|
||||
fn read_head(_: &Path, offset: u64, buf: &mut [u8]) -> io::Result<usize> {
|
||||
let i = CALL_COUNT.fetch_add(1, Ordering::Relaxed);
|
||||
if i < LAST_OFFSETS.len() {
|
||||
LAST_OFFSETS[i].store(offset as usize, Ordering::Relaxed);
|
||||
}
|
||||
let off = offset as usize;
|
||||
if off >= PAYLOAD.len() {
|
||||
return Ok(0); // EOF
|
||||
}
|
||||
let remaining = &PAYLOAD[off..];
|
||||
let take = remaining.len().min(buf.len()).min(CHUNK_SIZE);
|
||||
buf[..take].copy_from_slice(&remaining[..take]);
|
||||
Ok(take)
|
||||
}
|
||||
}
|
||||
|
||||
let mut buf = [0u8; 8];
|
||||
let read = read_head_filled::<ShortReader>(Path::new("any"), &mut buf).unwrap();
|
||||
assert_eq!(read, 8, "loop must accumulate short reads to fill the buffer");
|
||||
assert_eq!(&buf[..], b"abcdefgh");
|
||||
|
||||
// The loop made three calls (3 + 3 + 2 bytes) at offsets 0, 3, 6.
|
||||
assert_eq!(CALL_COUNT.load(Ordering::Relaxed), 3);
|
||||
assert_eq!(LAST_OFFSETS[0].load(Ordering::Relaxed), 0);
|
||||
assert_eq!(LAST_OFFSETS[1].load(Ordering::Relaxed), 3);
|
||||
assert_eq!(LAST_OFFSETS[2].load(Ordering::Relaxed), 6);
|
||||
}
|
||||
|
||||
/// [`read_head_filled`] terminates when the underlying capability
|
||||
/// returns 0 (EOF), exercised here with a file shorter than the buffer.
|
||||
#[test]
|
||||
fn read_head_filled_terminates_on_zero_byte_read_from_fake() {
|
||||
struct EofAfterOne;
|
||||
impl FsReadHead for EofAfterOne {
|
||||
fn read_head(_: &Path, offset: u64, buf: &mut [u8]) -> io::Result<usize> {
|
||||
if offset == 0 && !buf.is_empty() {
|
||||
buf[0] = b'X';
|
||||
Ok(1)
|
||||
} else {
|
||||
Ok(0) // EOF on subsequent calls
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut buf = [0u8; 16];
|
||||
let read = read_head_filled::<EofAfterOne>(Path::new("any"), &mut buf).unwrap();
|
||||
assert_eq!(read, 1, "loop must stop on EOF, returning the partial count");
|
||||
assert_eq!(buf[0], b'X');
|
||||
}
|
||||
|
||||
/// [`read_head_filled`] propagates non-EOF errors from the first call
|
||||
/// without retrying.
|
||||
#[test]
|
||||
fn read_head_filled_propagates_io_error_from_fake() {
|
||||
struct AlwaysErrors;
|
||||
impl FsReadHead for AlwaysErrors {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
Err(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
|
||||
let mut buf = [0u8; 16];
|
||||
let err = read_head_filled::<AlwaysErrors>(Path::new("any"), &mut buf).unwrap_err();
|
||||
assert_eq!(err.kind(), io::ErrorKind::PermissionDenied);
|
||||
}
|
||||
|
||||
/// [`generate_cmd_shim`] produces a Windows `.cmd` shim with CRLF line
|
||||
/// endings, `%~dp0\<rel>` for the target, and the
|
||||
/// `@IF EXIST … (… ) ELSE ( @SET PATHEXT=... … )` exec block matching
|
||||
/// upstream's template.
|
||||
#[test]
|
||||
fn generate_cmd_shim_matches_pnpm_template() {
|
||||
let target = Path::new("/proj/node_modules/typescript/bin/tsc");
|
||||
let shim = Path::new("/proj/node_modules/.bin/tsc.cmd");
|
||||
let runtime = ScriptRuntime { prog: Some("node".into()), args: String::new() };
|
||||
let body = generate_cmd_shim(target, shim, Some(&runtime));
|
||||
|
||||
assert!(body.starts_with("@SETLOCAL\r\n"), "must start with @SETLOCAL CRLF");
|
||||
assert!(
|
||||
body.contains("@IF EXIST \"%~dp0\\node.exe\" (\r\n \"%~dp0\\node.exe\" \"%~dp0\\..\\typescript\\bin\\tsc\" %*\r\n) ELSE (\r\n @SET PATHEXT=%PATHEXT:;.JS;=;%\r\n node \"%~dp0\\..\\typescript\\bin\\tsc\" %*\r\n)\r\n"),
|
||||
"exec block must match pnpm's generateCmdShim template, body was:\n{body}",
|
||||
);
|
||||
}
|
||||
|
||||
/// [`generate_cmd_shim`] with no runtime exec's the target directly via
|
||||
/// the `@<target> %*` shape.
|
||||
#[test]
|
||||
fn generate_cmd_shim_emits_direct_exec_when_no_runtime() {
|
||||
let target = Path::new("/p/cli");
|
||||
let shim = Path::new("/p/.bin/cli.cmd");
|
||||
let body = generate_cmd_shim(target, shim, None);
|
||||
assert!(
|
||||
body.contains("@\"%~dp0\\..\\cli\""),
|
||||
"no-runtime arm must exec the target directly, body:\n{body}",
|
||||
);
|
||||
}
|
||||
|
||||
/// [`generate_pwsh_shim`] produces a `.ps1` shim with the `$basedir`
|
||||
/// header, `Test-Path "$basedir/<prog>$exe"` exec block, and pipeline-
|
||||
/// input handling matching upstream.
|
||||
#[test]
|
||||
fn generate_pwsh_shim_matches_pnpm_template() {
|
||||
let target = Path::new("/proj/node_modules/typescript/bin/tsc");
|
||||
let shim = Path::new("/proj/node_modules/.bin/tsc.ps1");
|
||||
let runtime = ScriptRuntime { prog: Some("node".into()), args: String::new() };
|
||||
let body = generate_pwsh_shim(target, shim, Some(&runtime));
|
||||
|
||||
assert!(body.starts_with("#!/usr/bin/env pwsh\n"), "ps1 shim must start with pwsh shebang");
|
||||
assert!(
|
||||
body.contains("$basedir=Split-Path $MyInvocation.MyCommand.Definition -Parent"),
|
||||
"must declare $basedir from MyInvocation",
|
||||
);
|
||||
assert!(body.contains("$exe=\".exe\""), "Windows-detection branch must set $exe to .exe");
|
||||
assert!(
|
||||
body.contains(
|
||||
"if (Test-Path \"$basedir/node$exe\") {\n # Support pipeline input\n if ($MyInvocation.ExpectingInput) {\n $input | & \"$basedir/node$exe\" \"$basedir/../typescript/bin/tsc\" $args\n } else {\n & \"$basedir/node$exe\" \"$basedir/../typescript/bin/tsc\" $args\n }",
|
||||
),
|
||||
"exec-with-basedir-prog block must match pnpm's generatePwshShim template, body was:\n{body}",
|
||||
);
|
||||
assert!(body.ends_with("exit $ret\n"));
|
||||
}
|
||||
|
||||
/// [`generate_pwsh_shim`] with no runtime falls back to executing the
|
||||
/// target directly with `$LASTEXITCODE` propagation.
|
||||
#[test]
|
||||
fn generate_pwsh_shim_emits_direct_exec_when_no_runtime() {
|
||||
let target = Path::new("/p/cli");
|
||||
let shim = Path::new("/p/.bin/cli.ps1");
|
||||
let body = generate_pwsh_shim(target, shim, None);
|
||||
assert!(
|
||||
body.contains("& \"$basedir/../cli\""),
|
||||
"no-runtime arm must exec the target directly, body:\n{body}",
|
||||
);
|
||||
assert!(body.ends_with("exit $LASTEXITCODE\n"));
|
||||
}
|
||||
@@ -11,6 +11,7 @@ license.workspace = true
|
||||
repository.workspace = true
|
||||
|
||||
[dependencies]
|
||||
pacquet-cmd-shim = { workspace = true }
|
||||
pacquet-executor = { workspace = true }
|
||||
pacquet-fs = { workspace = true }
|
||||
pacquet-lockfile = { workspace = true }
|
||||
@@ -31,8 +32,8 @@ httpdate = { workspace = true }
|
||||
node-semver = { workspace = true }
|
||||
pipe-trait = { workspace = true }
|
||||
rayon = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
reflink-copy = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
tokio = { workspace = true }
|
||||
tracing = { workspace = true }
|
||||
miette = { workspace = true }
|
||||
|
||||
@@ -4,17 +4,36 @@ use crate::{
|
||||
use derive_more::{Display, Error};
|
||||
use futures_util::future;
|
||||
use miette::Diagnostic;
|
||||
use pacquet_lockfile::{LockfileResolution, PackageKey, PackageMetadata, SnapshotEntry};
|
||||
use pacquet_lockfile::{
|
||||
LockfileResolution, PackageKey, PackageMetadata, PkgNameVerPeer, SnapshotEntry,
|
||||
};
|
||||
use pacquet_network::ThrottledClient;
|
||||
use pacquet_npmrc::Npmrc;
|
||||
use pacquet_reporter::{
|
||||
LogEvent, LogLevel, ProgressLog, ProgressMessage, Reporter, StatsLog, StatsMessage,
|
||||
};
|
||||
use pacquet_store_dir::{SharedVerifiedFilesCache, StoreIndex, StoreIndexWriter, store_index_key};
|
||||
use pacquet_tarball::prefetch_cas_paths;
|
||||
use pacquet_tarball::{PrefetchResult, prefetch_cas_paths};
|
||||
use pipe_trait::Pipe;
|
||||
use std::{collections::HashMap, sync::atomic::AtomicU8};
|
||||
|
||||
/// Bundled package manifests recovered from the SQLite store index
|
||||
/// during [`CreateVirtualStore::run`], keyed by the same
|
||||
/// `PkgNameVerPeer` (without peer suffix) that
|
||||
/// [`pacquet_lockfile::Lockfile::packages`] uses. Consumed by the
|
||||
/// bin-linker so it doesn't have to re-read `package.json` per child
|
||||
/// during [`crate::LinkVirtualStoreBins::run`].
|
||||
///
|
||||
/// Only covers the warm-batch packages (those whose tarball was
|
||||
/// already in the CAFS at install start). Cold-batch packages — ones
|
||||
/// pacquet had to download — are absent and the bin linker falls
|
||||
/// back to disk reads for them. That matches pnpm's behaviour for
|
||||
/// installs that mix warm and cold packages: pnpm's bin linker
|
||||
/// reads from `pkgFilesIndex.manifest` for warm fetches and from
|
||||
/// `dep.fetching()?.bundledManifest` for cold ones, but the cold
|
||||
/// path's `bundledManifest` isn't plumbed through pacquet yet.
|
||||
pub type PackageManifests = HashMap<PkgNameVerPeer, std::sync::Arc<serde_json::Value>>;
|
||||
|
||||
/// This subroutine generates filesystem layout for the virtual store at `node_modules/.pacquet`.
|
||||
#[must_use]
|
||||
pub struct CreateVirtualStore<'a> {
|
||||
@@ -49,8 +68,11 @@ pub enum CreateVirtualStoreError {
|
||||
}
|
||||
|
||||
impl<'a> CreateVirtualStore<'a> {
|
||||
/// Execute the subroutine.
|
||||
pub async fn run<R: Reporter>(self) -> Result<(), CreateVirtualStoreError> {
|
||||
/// Execute the subroutine. Returns the set of bundled manifests
|
||||
/// recovered from `index.db` for the warm-batch slots — the
|
||||
/// bin linker uses these to avoid re-reading `package.json` per
|
||||
/// child. See [`PackageManifests`].
|
||||
pub async fn run<R: Reporter>(self) -> Result<PackageManifests, CreateVirtualStoreError> {
|
||||
let CreateVirtualStore {
|
||||
http_client,
|
||||
config,
|
||||
@@ -64,7 +86,7 @@ impl<'a> CreateVirtualStore<'a> {
|
||||
// No snapshots to install. If the lockfile also has no project deps
|
||||
// this is a valid no-op; if it does, pnpm would have populated
|
||||
// `snapshots`, so bailing out here is safe enough for v9.
|
||||
return Ok(());
|
||||
return Ok(PackageManifests::new());
|
||||
};
|
||||
let packages = packages.ok_or(CreateVirtualStoreError::MissingPackagesSection)?;
|
||||
|
||||
@@ -224,14 +246,15 @@ impl<'a> CreateVirtualStore<'a> {
|
||||
cache_key_refs.sort_unstable();
|
||||
cache_key_refs.dedup();
|
||||
let cache_keys: Vec<String> = cache_key_refs.into_iter().map(String::from).collect();
|
||||
let prefetched = prefetch_cas_paths(
|
||||
store_index.clone(),
|
||||
store_dir,
|
||||
cache_keys,
|
||||
config.verify_store_integrity,
|
||||
SharedVerifiedFilesCache::clone(&verified_files_cache),
|
||||
)
|
||||
.await;
|
||||
let PrefetchResult { cas_paths: prefetched, manifests: prefetched_manifests } =
|
||||
prefetch_cas_paths(
|
||||
store_index.clone(),
|
||||
store_dir,
|
||||
cache_keys,
|
||||
config.verify_store_integrity,
|
||||
SharedVerifiedFilesCache::clone(&verified_files_cache),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Partition snapshots by whether the prefetch covered them. The
|
||||
// warm batch — every snapshot whose tarball is already in the
|
||||
@@ -266,7 +289,25 @@ impl<'a> CreateVirtualStore<'a> {
|
||||
// local alias drifting (Copilot review on #292).
|
||||
let mut warm = Vec::with_capacity(snapshot_entries.len());
|
||||
let mut cold: Vec<(&PackageKey, &SnapshotEntry)> = Vec::new();
|
||||
// Build a `metadata_key -> manifest` lookup from the prefetched
|
||||
// index rows. Snapshot keys differ across peer-resolved
|
||||
// variants of the same package (`react-dom@17.0.2(react@...)`),
|
||||
// but the bundled manifest is identical across variants
|
||||
// because every variant resolves to the same tarball. Keying
|
||||
// by [`PkgNameVerPeer::without_peer`] collapses the variants
|
||||
// to one entry: same shape as
|
||||
// [`pacquet_lockfile::Lockfile::packages`], which is what the
|
||||
// bin linker already looks up by.
|
||||
let mut package_manifests: PackageManifests =
|
||||
HashMap::with_capacity(prefetched_manifests.len());
|
||||
for (snapshot_key, snapshot, cache_key) in &snapshot_entries {
|
||||
if let Some(cache_key) = cache_key.as_deref()
|
||||
&& let Some(manifest) = prefetched_manifests.get(cache_key)
|
||||
{
|
||||
package_manifests
|
||||
.entry(snapshot_key.without_peer())
|
||||
.or_insert_with(|| std::sync::Arc::clone(manifest));
|
||||
}
|
||||
match cache_key.as_deref().and_then(|k| prefetched.get(k)) {
|
||||
Some(cas_paths) => warm.push((snapshot_key, snapshot, cas_paths)),
|
||||
None => cold.push((snapshot_key, snapshot)),
|
||||
@@ -376,7 +417,7 @@ impl<'a> CreateVirtualStore<'a> {
|
||||
),
|
||||
}
|
||||
|
||||
Ok(())
|
||||
Ok(package_manifests)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ expression: get_all_folders(&project_root)
|
||||
---
|
||||
[
|
||||
"node_modules",
|
||||
"node_modules/.bin",
|
||||
"node_modules/.pacquet",
|
||||
"node_modules/.pacquet/@pnpm+x@1.0.0",
|
||||
"node_modules/.pacquet/@pnpm+x@1.0.0/node_modules",
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
use crate::{
|
||||
AllowBuildPolicy, BuildModules, BuildModulesError, CreateVirtualStore, CreateVirtualStoreError,
|
||||
SymlinkDirectDependencies, SymlinkDirectDependenciesError,
|
||||
LinkVirtualStoreBins, LinkVirtualStoreBinsError, SymlinkDirectDependencies,
|
||||
SymlinkDirectDependenciesError,
|
||||
};
|
||||
use derive_more::{Display, Error};
|
||||
use miette::Diagnostic;
|
||||
@@ -47,6 +48,9 @@ pub enum InstallFrozenLockfileError {
|
||||
#[diagnostic(transparent)]
|
||||
SymlinkDirectDependencies(#[error(source)] SymlinkDirectDependenciesError),
|
||||
|
||||
#[diagnostic(transparent)]
|
||||
LinkVirtualStoreBins(#[error(source)] LinkVirtualStoreBinsError),
|
||||
|
||||
#[diagnostic(transparent)]
|
||||
BuildModules(#[error(source)] BuildModulesError),
|
||||
}
|
||||
@@ -70,16 +74,42 @@ where
|
||||
|
||||
// TODO: check if the lockfile is out-of-date
|
||||
|
||||
CreateVirtualStore { http_client, config, packages, snapshots, logged_methods, requester }
|
||||
.run::<R>()
|
||||
.await
|
||||
.map_err(InstallFrozenLockfileError::CreateVirtualStore)?;
|
||||
let package_manifests = CreateVirtualStore {
|
||||
http_client,
|
||||
config,
|
||||
packages,
|
||||
snapshots,
|
||||
logged_methods,
|
||||
requester,
|
||||
}
|
||||
.run::<R>()
|
||||
.await
|
||||
.map_err(InstallFrozenLockfileError::CreateVirtualStore)?;
|
||||
|
||||
SymlinkDirectDependencies { config, importers, dependency_groups, requester }
|
||||
.run::<R>()
|
||||
.map_err(InstallFrozenLockfileError::SymlinkDirectDependencies)?;
|
||||
|
||||
// Mirrors upstream `link.ts:167-170` — `importing_done` fires once
|
||||
// Link the bins of each virtual-store slot's children into the
|
||||
// slot's own `node_modules/.bin`. Pnpm runs this from
|
||||
// `linkBinsOfDependencies` during the headless install. See
|
||||
// <https://github.com/pnpm/pnpm/blob/4750fd370c/building/during-install/src/index.ts#L258-L309>.
|
||||
// Done before `importing_done` so reporters see the import phase
|
||||
// close only after every link (including per-slot bins) is in
|
||||
// place. The manifest map threaded from `CreateVirtualStore`
|
||||
// lets the linker hit `pkgFilesIndex.manifest` directly
|
||||
// (matching pnpm's `bundledManifest`-from-CAFS path) instead
|
||||
// of re-reading every child's `package.json` from disk.
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &config.virtual_store_dir,
|
||||
snapshots,
|
||||
packages,
|
||||
package_manifests: &package_manifests,
|
||||
}
|
||||
.run()
|
||||
.map_err(InstallFrozenLockfileError::LinkVirtualStoreBins)?;
|
||||
|
||||
// Mirrors upstream `link.ts:167-170`: `importing_done` fires once
|
||||
// extraction and symlink linking are complete, before any build
|
||||
// phase. Reporters use it to close the import progress display so
|
||||
// subsequent `pnpm:lifecycle` events render in their own section.
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
use crate::{
|
||||
InstallPackageFromRegistry, InstallPackageFromRegistryError,
|
||||
store_init::init_store_dir_best_effort,
|
||||
InstallPackageFromRegistry, InstallPackageFromRegistryError, LinkVirtualStoreBins,
|
||||
LinkVirtualStoreBinsError, store_init::init_store_dir_best_effort,
|
||||
};
|
||||
use async_recursion::async_recursion;
|
||||
use dashmap::DashSet;
|
||||
use derive_more::{Display, Error};
|
||||
use futures_util::future;
|
||||
use miette::Diagnostic;
|
||||
use pacquet_cmd_shim::{LinkBinsError, RealApi, link_bins};
|
||||
use pacquet_network::ThrottledClient;
|
||||
use pacquet_npmrc::Npmrc;
|
||||
use pacquet_package_manifest::{DependencyGroup, PackageManifest};
|
||||
@@ -52,6 +53,12 @@ pub struct InstallWithoutLockfile<'a, DependencyGroupList> {
|
||||
pub enum InstallWithoutLockfileError {
|
||||
#[diagnostic(transparent)]
|
||||
InstallPackageFromRegistry(#[error(source)] InstallPackageFromRegistryError),
|
||||
|
||||
#[diagnostic(transparent)]
|
||||
LinkBins(#[error(source)] LinkBinsError),
|
||||
|
||||
#[diagnostic(transparent)]
|
||||
LinkVirtualStoreBins(#[error(source)] LinkVirtualStoreBinsError),
|
||||
}
|
||||
|
||||
impl<'a, DependencyGroupList> InstallWithoutLockfile<'a, DependencyGroupList> {
|
||||
@@ -182,7 +189,33 @@ impl<'a, DependencyGroupList> InstallWithoutLockfile<'a, DependencyGroupList> {
|
||||
),
|
||||
}
|
||||
|
||||
// Mirrors upstream `link.ts:167-170` — `importing_done` fires once
|
||||
// Link bins. Direct dependencies first (root project's
|
||||
// `node_modules/.bin`) and then per-slot children inside the
|
||||
// virtual store. Mirrors the same two-call shape as
|
||||
// `install_frozen_lockfile.rs`. We re-walk `<modules_dir>` instead
|
||||
// of replaying the manifest because the `dependency_groups`
|
||||
// iterator was already consumed by the install loop above; pnpm's
|
||||
// own `linkBins(modulesDir, binsDir)` overload uses the same
|
||||
// strategy.
|
||||
link_bins::<RealApi>(&config.modules_dir, &config.modules_dir.join(".bin"))
|
||||
.map_err(InstallWithoutLockfileError::LinkBins)?;
|
||||
|
||||
// No lockfile here, so no prefetched manifests are available —
|
||||
// fall back to the legacy readdir-driven path (slots discovered
|
||||
// by walking `<virtual_store_dir>`, child manifests read from
|
||||
// disk). The frozen-lockfile path skips both via
|
||||
// [`LinkVirtualStoreBins::snapshots`] / `package_manifests`.
|
||||
let empty_manifests = std::collections::HashMap::new();
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &config.virtual_store_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &empty_manifests,
|
||||
}
|
||||
.run()
|
||||
.map_err(InstallWithoutLockfileError::LinkVirtualStoreBins)?;
|
||||
|
||||
// Mirrors upstream `link.ts:167-170`: `importing_done` fires once
|
||||
// extraction and symlink linking are complete. The without-lockfile
|
||||
// path does not run lifecycle scripts today, so emitting here also
|
||||
// marks end-of-install for reporters.
|
||||
|
||||
@@ -12,6 +12,7 @@ mod install_frozen_lockfile;
|
||||
mod install_package_by_snapshot;
|
||||
mod install_package_from_registry;
|
||||
mod install_without_lockfile;
|
||||
mod link_bins;
|
||||
mod link_file;
|
||||
mod retry_config;
|
||||
mod store_init;
|
||||
@@ -32,6 +33,7 @@ pub use install_frozen_lockfile::*;
|
||||
pub use install_package_by_snapshot::*;
|
||||
pub use install_package_from_registry::*;
|
||||
pub use install_without_lockfile::*;
|
||||
pub use link_bins::*;
|
||||
pub use link_file::*;
|
||||
pub use symlink_direct_dependencies::*;
|
||||
pub use symlink_package::*;
|
||||
|
||||
549
pacquet/crates/package-manager/src/link_bins.rs
Normal file
549
pacquet/crates/package-manager/src/link_bins.rs
Normal file
@@ -0,0 +1,549 @@
|
||||
use crate::PackageManifests;
|
||||
use derive_more::{Display, Error};
|
||||
use miette::Diagnostic;
|
||||
use pacquet_cmd_shim::{
|
||||
FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadString,
|
||||
FsSetExecutable, FsWalkFiles, FsWrite, LinkBinsError, PackageBinSource, RealApi,
|
||||
link_bins_of_packages,
|
||||
};
|
||||
use pacquet_lockfile::{PackageKey, PackageMetadata, PkgName, SnapshotEntry};
|
||||
use rayon::prelude::*;
|
||||
use std::{
|
||||
collections::{HashMap, HashSet},
|
||||
fs, io,
|
||||
path::{Path, PathBuf},
|
||||
sync::Arc,
|
||||
};
|
||||
|
||||
/// Read the `package.json` of every direct dependency under `modules_dir`
|
||||
/// and link its bins into `<modules_dir>/.bin`.
|
||||
///
|
||||
/// `dep_names` is the list of direct-dependency keys as they appear in
|
||||
/// `package.json`, the same names already symlinked under `<modules_dir>/`
|
||||
/// by [`crate::SymlinkDirectDependencies`]. We resolve `package.json` via
|
||||
/// the symlink (`fs::read` follows it transparently) so the read targets
|
||||
/// the real package contents in the virtual store.
|
||||
///
|
||||
/// Driven on rayon because each location's read+parse is independent.
|
||||
/// Mirrors pnpm v11's `linkBinsOfPackages` call site for direct deps:
|
||||
/// <https://github.com/pnpm/pnpm/blob/4750fd370c/installing/deps-installer/src/install/index.ts#L1539>.
|
||||
pub fn link_direct_dep_bins(modules_dir: &Path, dep_names: &[String]) -> Result<(), LinkBinsError> {
|
||||
let direct_dep_locations: Vec<PathBuf> =
|
||||
dep_names.iter().map(|name| modules_dir.join(name)).collect();
|
||||
// Swallow only `NotFound`: a direct-dep symlink target can
|
||||
// legitimately be missing right after a partial pacquet run, or
|
||||
// be an in-progress install. Every other IO error (permission
|
||||
// denied, EIO, etc.) and every JSON parse error must surface as
|
||||
// `LinkBinsError::{ReadManifest, ParseManifest}` so the failure
|
||||
// is diagnosable rather than hiding behind a missing `.bin`
|
||||
// entry. Matches the read-side error policy in
|
||||
// `pacquet_cmd_shim::link_bins`.
|
||||
let bin_sources: Vec<PackageBinSource> = direct_dep_locations
|
||||
.par_iter()
|
||||
.filter_map(|location| {
|
||||
let manifest_path = location.join("package.json");
|
||||
let bytes = match fs::read(&manifest_path) {
|
||||
Ok(bytes) => bytes,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => return None,
|
||||
Err(error) => {
|
||||
return Some(Err(LinkBinsError::ReadManifest { path: manifest_path, error }));
|
||||
}
|
||||
};
|
||||
let manifest: serde_json::Value = match serde_json::from_slice(&bytes) {
|
||||
Ok(manifest) => manifest,
|
||||
Err(error) => {
|
||||
return Some(Err(LinkBinsError::ParseManifest { path: manifest_path, error }));
|
||||
}
|
||||
};
|
||||
Some(Ok(PackageBinSource { location: location.clone(), manifest: Arc::new(manifest) }))
|
||||
})
|
||||
.collect::<Result<_, _>>()?;
|
||||
if bin_sources.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
link_bins_of_packages::<RealApi>(&bin_sources, &modules_dir.join(".bin"))
|
||||
}
|
||||
|
||||
/// Error type of [`LinkVirtualStoreBins`].
|
||||
#[derive(Debug, Display, Error, Diagnostic)]
|
||||
pub enum LinkVirtualStoreBinsError {
|
||||
#[display("Failed to read virtual store directory at {dir:?}: {error}")]
|
||||
#[diagnostic(code(pacquet_package_manager::read_virtual_store))]
|
||||
ReadVirtualStore {
|
||||
dir: PathBuf,
|
||||
#[error(source)]
|
||||
error: io::Error,
|
||||
},
|
||||
|
||||
#[diagnostic(transparent)]
|
||||
LinkBins(#[error(source)] LinkBinsError),
|
||||
}
|
||||
|
||||
/// For every package slot under `<virtual_store_dir>/<pkg>@<ver>/node_modules`,
|
||||
/// link the bins of that slot's child packages into the slot's *own*
|
||||
/// `node_modules/.bin` directory.
|
||||
///
|
||||
/// This mirrors `linkBinsOfDependencies` in pnpm's `building/during-install`
|
||||
/// (see <https://github.com/pnpm/pnpm/blob/4750fd370c/building/during-install/src/index.ts#L258-L309>).
|
||||
/// pnpm walks each `depNode`, takes its `children` (its direct deps in the
|
||||
/// resolved graph) and writes their bins into
|
||||
/// `<depNode.dir>/node_modules/.bin`.
|
||||
///
|
||||
/// Pacquet's virtual store layout already exposes a slot's children as
|
||||
/// siblings via [`create_symlink_layout`](crate::create_symlink_layout()).
|
||||
/// So once the symlinks exist, walking
|
||||
/// the slot's `node_modules` and excluding the package itself gives the same
|
||||
/// child-set pnpm uses, and the bins go into the package's own
|
||||
/// `node_modules/.bin` (i.e. nested *one level deeper* than the slot's
|
||||
/// `node_modules` directory).
|
||||
///
|
||||
/// Path layout produced for a slot `A@1.0.0`:
|
||||
///
|
||||
/// ```text
|
||||
/// <virtual>/A@1.0.0/node_modules/A/node_modules/.bin/<bin>
|
||||
/// ```
|
||||
///
|
||||
/// When `snapshots` is `Some` (the frozen-lockfile case), the slot
|
||||
/// set is taken from the lockfile and each child's manifest is
|
||||
/// looked up in `package_manifests` rather than read off disk —
|
||||
/// matching pnpm's `linkBinsOfDependencies` which consumes
|
||||
/// `bundledManifest` straight out of the SQLite store index (see
|
||||
/// <https://github.com/pnpm/pnpm/blob/4750fd370c/building/during-install/src/index.ts#L289>).
|
||||
/// When `snapshots` is `None` (install without a lockfile), the
|
||||
/// linker falls back to enumerating slots and reading manifests via
|
||||
/// the filesystem, the shape this code had before the
|
||||
/// lockfile-driven path landed.
|
||||
#[must_use]
|
||||
pub struct LinkVirtualStoreBins<'a> {
|
||||
pub virtual_store_dir: &'a Path,
|
||||
/// `Some` when the install is lockfile-driven. Iterating the
|
||||
/// snapshot map (instead of `read_dir(virtual_store_dir)`)
|
||||
/// removes the per-slot directory enumeration and lets us walk
|
||||
/// each slot's children from its `dependencies` /
|
||||
/// `optionalDependencies` lists without touching the filesystem.
|
||||
pub snapshots: Option<&'a HashMap<PackageKey, SnapshotEntry>>,
|
||||
/// Lockfile `packages:` section, indexed by `PkgNameVerPeer`
|
||||
/// (without peer suffix). Used to filter children by
|
||||
/// `hasBin == true` *before* any per-child IO — mirrors pnpm's
|
||||
/// `dep.hasBin` filter in
|
||||
/// [`linkBinsOfDependencies`](https://github.com/pnpm/pnpm/blob/4750fd370c/building/during-install/src/index.ts#L283).
|
||||
/// Most packages don't declare a bin, so this short-circuits the
|
||||
/// bulk of the per-slot work before any path-building or manifest
|
||||
/// lookup happens.
|
||||
pub packages: Option<&'a HashMap<PackageKey, PackageMetadata>>,
|
||||
/// Bundled manifests recovered from the warm-cache prefetch of
|
||||
/// `index.db` ([`crate::PackageManifests`]). A hit lets the
|
||||
/// linker skip the `package.json` read for that child entirely;
|
||||
/// a miss falls back to a disk read so cold-batch packages
|
||||
/// installed earlier in the same run still get their bins
|
||||
/// linked.
|
||||
pub package_manifests: &'a PackageManifests,
|
||||
}
|
||||
|
||||
impl<'a> LinkVirtualStoreBins<'a> {
|
||||
pub fn run(self) -> Result<(), LinkVirtualStoreBinsError> {
|
||||
self.run_with::<RealApi>()
|
||||
}
|
||||
|
||||
/// DI-driven entry. Production callers go through [`Self::run`] which
|
||||
/// turbofishes [`RealApi`]; tests inject fakes that fail specific fs
|
||||
/// operations to cover error paths the real fs can't trigger
|
||||
/// portably. See the per-capability DI pattern at
|
||||
/// <https://github.com/pnpm/pacquet/pull/332#issuecomment-4345054524>.
|
||||
pub fn run_with<Api>(self) -> Result<(), LinkVirtualStoreBinsError>
|
||||
where
|
||||
Api: FsReadDir
|
||||
+ FsReadFile
|
||||
+ FsReadString
|
||||
+ FsReadHead
|
||||
+ FsCreateDirAll
|
||||
+ FsWalkFiles
|
||||
+ FsWrite
|
||||
+ FsSetExecutable
|
||||
+ FsEnsureExecutableBits,
|
||||
{
|
||||
let LinkVirtualStoreBins { virtual_store_dir, snapshots, packages, package_manifests } =
|
||||
self;
|
||||
if let Some(snapshots) = snapshots {
|
||||
let has_bin_set = build_has_bin_set(packages);
|
||||
run_lockfile_driven::<Api>(
|
||||
virtual_store_dir,
|
||||
snapshots,
|
||||
has_bin_set.as_ref(),
|
||||
package_manifests,
|
||||
)
|
||||
} else {
|
||||
run_with_readdir::<Api>(virtual_store_dir)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Pre-compute the set of package keys whose lockfile metadata sets
|
||||
/// `hasBin: true`. Mirrors pnpm's filter at
|
||||
/// [`during-install/src/index.ts:283`](https://github.com/pnpm/pnpm/blob/4750fd370c/building/during-install/src/index.ts#L283):
|
||||
/// most packages don't declare a bin, so short-circuiting the
|
||||
/// per-child manifest lookup with this set is the cheapest win on
|
||||
/// warm-cache installs.
|
||||
///
|
||||
/// Return-value semantics distinguish "lockfile metadata absent"
|
||||
/// from "lockfile metadata says no package has a bin":
|
||||
///
|
||||
/// - `None` — the lockfile's `packages:` section wasn't supplied
|
||||
/// (pathological lockfile shape). We have no info, so the bin
|
||||
/// linker falls back to the conservative "process every child"
|
||||
/// path and lets the per-package bin resolver sort it out.
|
||||
/// - `Some(set)` — the section was present and we used it. The
|
||||
/// `set` contains only entries with `hasBin == Some(true)`; an
|
||||
/// *empty* `Some(set)` is authoritative: the lockfile says no
|
||||
/// package has a bin, and every slot should short-circuit
|
||||
/// immediately. Conflating this case with `None` (the bug Copilot
|
||||
/// flagged at <https://github.com/pnpm/pacquet/pull/333#discussion_r3222807548>)
|
||||
/// would force per-child work the lockfile already ruled out.
|
||||
fn build_has_bin_set(
|
||||
packages: Option<&HashMap<PackageKey, PackageMetadata>>,
|
||||
) -> Option<HashSet<PackageKey>> {
|
||||
let packages = packages?;
|
||||
Some(
|
||||
packages
|
||||
.iter()
|
||||
.filter(|(_, meta)| meta.has_bin == Some(true))
|
||||
.map(|(key, _)| key.clone())
|
||||
.collect(),
|
||||
)
|
||||
}
|
||||
|
||||
/// Walk the lockfile's `snapshots:` map, build each slot's bin output
|
||||
/// directory lexically, and link every child's bins into it. The
|
||||
/// child set comes from `snapshot.dependencies` +
|
||||
/// `snapshot.optional_dependencies`, filtered by `has_bin_set` so
|
||||
/// packages that don't declare a bin never make it into the
|
||||
/// per-slot path-building or manifest-lookup work. The corresponding
|
||||
/// manifest comes from [`PackageManifests`] (no disk read) or, for
|
||||
/// cold-batch packages that prefetch missed, a fallback
|
||||
/// `package.json` read through the existing symlink at
|
||||
/// `<slot>/node_modules/<alias>`.
|
||||
fn run_lockfile_driven<Api>(
|
||||
virtual_store_dir: &Path,
|
||||
snapshots: &HashMap<PackageKey, SnapshotEntry>,
|
||||
has_bin_set: Option<&HashSet<PackageKey>>,
|
||||
package_manifests: &PackageManifests,
|
||||
) -> Result<(), LinkVirtualStoreBinsError>
|
||||
where
|
||||
Api: FsReadFile
|
||||
+ FsReadString
|
||||
+ FsReadHead
|
||||
+ FsCreateDirAll
|
||||
+ FsWalkFiles
|
||||
+ FsWrite
|
||||
+ FsSetExecutable
|
||||
+ FsEnsureExecutableBits,
|
||||
{
|
||||
// `has_bin_set` is `Some` exactly when the lockfile's `packages:`
|
||||
// section was present at install start — in which case the set
|
||||
// is authoritative and every slot is filtered through it (an
|
||||
// empty `Some(set)` means "no package declares a bin", which
|
||||
// short-circuits every slot below). When the section was
|
||||
// missing we have no info and fall through to processing every
|
||||
// child. See [`build_has_bin_set`] for the rationale.
|
||||
// Materialise as a `Vec` so rayon can split the work; iterating
|
||||
// a `HashMap` directly with `par_iter` would require collecting
|
||||
// anyway, and explicit collection here keeps the parallelism
|
||||
// contract obvious.
|
||||
let slot_entries: Vec<(&PackageKey, &SnapshotEntry)> = snapshots.iter().collect();
|
||||
slot_entries.par_iter().try_for_each(|(slot_key, snapshot)| {
|
||||
let children = snapshot
|
||||
.dependencies
|
||||
.iter()
|
||||
.flatten()
|
||||
.chain(snapshot.optional_dependencies.iter().flatten());
|
||||
|
||||
// First pass: figure out which children (if any) have a bin
|
||||
// declared. Cheap — just hash-set lookups against the
|
||||
// pre-built `has_bin_set` and a `without_peer` materialisation
|
||||
// per child. If no child has a bin, skip the slot entirely —
|
||||
// we don't even build the slot's path. Slots in this category
|
||||
// are the bulk of a real lockfile (~95% in the integrated
|
||||
// benchmark fixture); skipping them removes the dominant
|
||||
// chunk of the per-install bin-link work.
|
||||
let with_bin: Vec<(&PkgName, PackageKey)> = children
|
||||
.filter_map(|(alias, dep_ref)| {
|
||||
let child_key = dep_ref.resolve(alias);
|
||||
let metadata_key = child_key.without_peer();
|
||||
let keep = match has_bin_set {
|
||||
Some(set) => set.contains(&metadata_key),
|
||||
None => true,
|
||||
};
|
||||
keep.then_some((alias, metadata_key))
|
||||
})
|
||||
.collect();
|
||||
if with_bin.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let slot_dir = virtual_store_dir.join(slot_key.to_virtual_store_name());
|
||||
let modules_dir = slot_dir.join("node_modules");
|
||||
let self_pkg_dir = slot_own_pkg_dir(&modules_dir, slot_key);
|
||||
let bins_dir = self_pkg_dir.join("node_modules/.bin");
|
||||
|
||||
let mut bin_sources: Vec<PackageBinSource> = Vec::with_capacity(with_bin.len());
|
||||
for (alias, metadata_key) in with_bin {
|
||||
let child_location = pkg_dir_under(&modules_dir, alias);
|
||||
if let Some(manifest) = package_manifests.get(&metadata_key) {
|
||||
// Hot path: parsed manifest already in memory from
|
||||
// the warm-cache prefetch. Both the prefetch map
|
||||
// and `PackageBinSource` hold the manifest via
|
||||
// [`Arc`], so this is a refcount bump rather than a
|
||||
// deep clone of the JSON tree. Avoids the
|
||||
// `slots × children`-sized clone fan-out that
|
||||
// dominated the previous version of this path on
|
||||
// warm-cache installs.
|
||||
bin_sources.push(PackageBinSource {
|
||||
location: child_location,
|
||||
manifest: Arc::clone(manifest),
|
||||
});
|
||||
} else {
|
||||
// Cold-batch fallback: package was downloaded
|
||||
// earlier in the run, so its row isn't in the
|
||||
// prefetched manifest map yet. Reading from disk
|
||||
// here is the same code path as the non-lockfile
|
||||
// install — see [`run_with_readdir`].
|
||||
match read_package::<Api>(&child_location) {
|
||||
Ok(Some(pkg)) => bin_sources.push(pkg),
|
||||
Ok(None) => {}
|
||||
Err(error) => return Err(LinkVirtualStoreBinsError::LinkBins(error)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if bin_sources.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
link_bins_of_packages::<Api>(&bin_sources, &bins_dir)
|
||||
.map_err(LinkVirtualStoreBinsError::LinkBins)
|
||||
})
|
||||
}
|
||||
|
||||
/// Compute `<slot>/node_modules/<pkg-or-@scope/pkg>` for the slot's
|
||||
/// own package. The slot's package name lives on the lockfile key,
|
||||
/// so no filesystem probing is needed (the directory is an invariant
|
||||
/// maintained by [`crate::create_virtual_dir_by_snapshot`]). Scoped
|
||||
/// names land at `<modules>/@scope/<name>`, unscoped names at
|
||||
/// `<modules>/<name>`.
|
||||
fn slot_own_pkg_dir(modules_dir: &Path, slot_key: &PackageKey) -> PathBuf {
|
||||
pkg_dir_under(modules_dir, &slot_key.name)
|
||||
}
|
||||
|
||||
/// Join a package name onto a `node_modules` directory, handling the
|
||||
/// `@scope/name` split into two path components. Operates on the raw
|
||||
/// [`PkgName`] (whose `scope` and `bare` fields are already split),
|
||||
/// not on the virtual-store-name form — for instance the input
|
||||
/// represents `@types/node`, **not** `@types+node`.
|
||||
fn pkg_dir_under(modules_dir: &Path, name: &PkgName) -> PathBuf {
|
||||
match &name.scope {
|
||||
Some(scope) => modules_dir.join(format!("@{scope}")).join(&name.bare),
|
||||
None => modules_dir.join(&name.bare),
|
||||
}
|
||||
}
|
||||
|
||||
/// Fallback (non-lockfile) path: enumerate slots via `read_dir`,
|
||||
/// then walk each slot's `node_modules` to discover children. Used
|
||||
/// only by [`crate::InstallWithoutLockfile`] today; the lockfile
|
||||
/// path bypasses every directory enumeration in here.
|
||||
fn run_with_readdir<Api>(virtual_store_dir: &Path) -> Result<(), LinkVirtualStoreBinsError>
|
||||
where
|
||||
Api: FsReadDir
|
||||
+ FsReadFile
|
||||
+ FsReadString
|
||||
+ FsReadHead
|
||||
+ FsCreateDirAll
|
||||
+ FsWalkFiles
|
||||
+ FsWrite
|
||||
+ FsSetExecutable
|
||||
+ FsEnsureExecutableBits,
|
||||
{
|
||||
let slots = match Api::read_dir(virtual_store_dir) {
|
||||
Ok(slots) => slots,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()),
|
||||
Err(error) => {
|
||||
return Err(LinkVirtualStoreBinsError::ReadVirtualStore {
|
||||
dir: virtual_store_dir.to_path_buf(),
|
||||
error,
|
||||
});
|
||||
}
|
||||
};
|
||||
let slots: Vec<PathBuf> = slots.collect();
|
||||
slots.par_iter().try_for_each(|slot_dir| {
|
||||
let modules_dir = slot_dir.join("node_modules");
|
||||
let Some(self_pkg_dir) = find_slot_own_package_dir(slot_dir, &modules_dir) else {
|
||||
return Ok(());
|
||||
};
|
||||
// Probe the slot's own package directory before walking its
|
||||
// children. Without the probe, an incomplete slot whose
|
||||
// `node_modules/<pkg>` is missing but whose sibling deps are
|
||||
// still present would have `link_bins_excluding` collect the
|
||||
// siblings and `create_dir_all` the missing `<pkg>` chain to
|
||||
// hold the shims, leaving an orphan package directory on
|
||||
// disk. This path runs only for [`crate::InstallWithoutLockfile`]
|
||||
// and visits ~direct-deps slots (small N), so the probe cost
|
||||
// is trivial; the lockfile-driven path bypasses this by
|
||||
// treating the slot's own pkg dir as an invariant of
|
||||
// [`crate::create_virtual_dir_by_snapshot`].
|
||||
if Api::read_dir(&self_pkg_dir).is_err() {
|
||||
return Ok(());
|
||||
}
|
||||
let bins_dir = self_pkg_dir.join("node_modules/.bin");
|
||||
link_bins_excluding::<Api>(&modules_dir, &bins_dir, &self_pkg_dir)
|
||||
.map_err(LinkVirtualStoreBinsError::LinkBins)
|
||||
})
|
||||
}
|
||||
|
||||
/// Locate the slot's own package directory inside `<slot>/node_modules`.
|
||||
///
|
||||
/// The slot directory's name encodes the package name as
|
||||
/// `<scope>+<name>@<version>` for the simple case (see
|
||||
/// [`pacquet_lockfile::PkgNameVerPeer::to_virtual_store_name`]). For
|
||||
/// peer-resolved slots the version segment itself contains additional
|
||||
/// `@`-separated peer specs joined by `_`, e.g.
|
||||
/// `ts-node@10.9.1_@types+node@18.7.19_typescript@5.1.6`. The `@` after
|
||||
/// `typescript` is part of a peer's version, not the package-name
|
||||
/// boundary. Parsing from the right (`rfind('@')`) would split there
|
||||
/// and silently break peer-resolved slots; parse from the left
|
||||
/// instead, skipping a leading `@` that belongs to a scoped package.
|
||||
///
|
||||
/// Returns `None` only when the slot name fails to parse — there's no
|
||||
/// filesystem probe for the resolved candidate. The previous version
|
||||
/// stat-equivalent-ed the path with `Api::read_dir` to short-circuit
|
||||
/// missing slots, but on a 1267-package fixture that was 1267
|
||||
/// wasted `open(O_DIRECTORY) + close` round-trips on the hot path of
|
||||
/// every warm install. The slot's own package directory is an
|
||||
/// invariant of [`crate::create_virtual_dir_by_snapshot`]; the
|
||||
/// downstream `link_bins_excluding` handles `NotFound` from its own
|
||||
/// `read_dir` of `<slot>/node_modules` cleanly when the invariant
|
||||
/// ever does break, so the probe is pure overhead.
|
||||
fn find_slot_own_package_dir(slot_dir: &Path, modules_dir: &Path) -> Option<PathBuf> {
|
||||
let slot_name = slot_dir.file_name()?.to_str()?;
|
||||
|
||||
// The package-name half is everything before the **first** `@`,
|
||||
// ignoring a single leading `@` that belongs to a scoped name
|
||||
// (`@scope+pkg@...` → start the `@` search at offset 1).
|
||||
// After `to_virtual_store_name`, `/` in scoped names becomes `+`,
|
||||
// so the package-name half can never contain `@` itself.
|
||||
let scoped = slot_name.starts_with('@');
|
||||
let search_start = usize::from(scoped);
|
||||
let at = search_start + slot_name[search_start..].find('@')?;
|
||||
let name_part = &slot_name[..at];
|
||||
|
||||
// `+` separates `<scope>+<name>` for scoped packages, and *only*
|
||||
// for scoped packages. Gating on `scoped` avoids misparsing a
|
||||
// hypothetical unscoped name that contains `+`: `PkgName::parse`
|
||||
// does not reject non-URL-safe characters (only npm's
|
||||
// `validate-npm-package-name` warns about them), so an unscoped
|
||||
// name like `foo+bar` could in principle reach here and would
|
||||
// otherwise be split into `foo` / `bar`.
|
||||
let pkg_dir = match scoped.then(|| name_part.split_once('+')).flatten() {
|
||||
Some((scope, name)) => modules_dir.join(scope).join(name),
|
||||
None => modules_dir.join(name_part),
|
||||
};
|
||||
Some(pkg_dir)
|
||||
}
|
||||
|
||||
/// Like [`pacquet_cmd_shim::link_bins`] but skipping the slot's own package
|
||||
/// from the candidate set. Without this, a slot for `tsc@5.0.0` would link
|
||||
/// its own `tsc` bin into its own `node_modules/.bin`, which pnpm doesn't.
|
||||
fn link_bins_excluding<Api>(
|
||||
modules_dir: &Path,
|
||||
bins_dir: &Path,
|
||||
exclude: &Path,
|
||||
) -> Result<(), LinkBinsError>
|
||||
where
|
||||
Api: FsReadDir
|
||||
+ FsReadFile
|
||||
+ FsReadString
|
||||
+ FsReadHead
|
||||
+ FsCreateDirAll
|
||||
+ FsWalkFiles
|
||||
+ FsWrite
|
||||
+ FsSetExecutable
|
||||
+ FsEnsureExecutableBits,
|
||||
{
|
||||
let mut packages: Vec<PackageBinSource> = Vec::new();
|
||||
|
||||
let entries = match Api::read_dir(modules_dir) {
|
||||
Ok(entries) => entries,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()),
|
||||
Err(error) => {
|
||||
return Err(LinkBinsError::ReadModulesDir { dir: modules_dir.to_path_buf(), error });
|
||||
}
|
||||
};
|
||||
|
||||
for path in entries {
|
||||
let Some(name) = path.file_name() else {
|
||||
continue;
|
||||
};
|
||||
let name_str = name.to_string_lossy();
|
||||
if name_str.starts_with('.') {
|
||||
continue;
|
||||
}
|
||||
|
||||
if name_str.starts_with('@') {
|
||||
// Only `NotFound` is plausibly skippable here (a
|
||||
// concurrent scope-dir delete). Other errors —
|
||||
// permission denied, EIO, AppArmor deny — would mean
|
||||
// the bins for every package under this scope silently
|
||||
// disappear, so surface them instead of letting them
|
||||
// hide.
|
||||
let scope_entries = match Api::read_dir(&path) {
|
||||
Ok(entries) => entries,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => continue,
|
||||
Err(error) => {
|
||||
return Err(LinkBinsError::ReadModulesDir { dir: path.clone(), error });
|
||||
}
|
||||
};
|
||||
for sub_path in scope_entries {
|
||||
if paths_eq(&sub_path, exclude) {
|
||||
continue;
|
||||
}
|
||||
if let Some(pkg) = read_package::<Api>(&sub_path)? {
|
||||
packages.push(pkg);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if paths_eq(&path, exclude) {
|
||||
continue;
|
||||
}
|
||||
if let Some(pkg) = read_package::<Api>(&path)? {
|
||||
packages.push(pkg);
|
||||
}
|
||||
}
|
||||
|
||||
if packages.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
link_bins_of_packages::<Api>(&packages, bins_dir)
|
||||
}
|
||||
|
||||
fn read_package<Api: FsReadFile>(
|
||||
location: &Path,
|
||||
) -> Result<Option<PackageBinSource>, LinkBinsError> {
|
||||
let manifest_path = location.join("package.json");
|
||||
let bytes = match Api::read_file(&manifest_path) {
|
||||
Ok(bytes) => bytes,
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None),
|
||||
Err(error) => return Err(LinkBinsError::ReadManifest { path: manifest_path, error }),
|
||||
};
|
||||
let manifest: serde_json::Value = serde_json::from_slice(&bytes)
|
||||
.map_err(|error| LinkBinsError::ParseManifest { path: manifest_path, error })?;
|
||||
Ok(Some(PackageBinSource { location: location.to_path_buf(), manifest: Arc::new(manifest) }))
|
||||
}
|
||||
|
||||
fn paths_eq(a: &Path, b: &Path) -> bool {
|
||||
// Lexical comparison is enough; both paths come from the same
|
||||
// `node_modules` walk and don't go through canonicalisation.
|
||||
a == b
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
475
pacquet/crates/package-manager/src/link_bins/tests.rs
Normal file
475
pacquet/crates/package-manager/src/link_bins/tests.rs
Normal file
@@ -0,0 +1,475 @@
|
||||
use super::{LinkVirtualStoreBins, LinkVirtualStoreBinsError, link_direct_dep_bins};
|
||||
use pacquet_cmd_shim::is_shim_pointing_at;
|
||||
use serde_json::json;
|
||||
use std::{
|
||||
fs::{create_dir_all, read_to_string, write as write_file},
|
||||
iter::{Empty, empty},
|
||||
path::{Path, PathBuf},
|
||||
};
|
||||
use tempfile::tempdir;
|
||||
|
||||
/// End-to-end exercise of [`LinkVirtualStoreBins`] against a hand-built
|
||||
/// virtual store. Slot `parent@1.0.0` has one child `child` declaring a
|
||||
/// bin; after the run, the child's shim must land at
|
||||
/// `parent@1.0.0/node_modules/parent/node_modules/.bin/child` and *not*
|
||||
/// at the slot's own `node_modules/.bin` (which is what would happen if
|
||||
/// we accidentally pointed at the wrong directory).
|
||||
#[test]
|
||||
fn writes_child_bins_into_slot_own_package_node_modules() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let virtual_dir = tmp.path().join(".pacquet");
|
||||
|
||||
// The slot for `parent@1.0.0`. pnpm uses `+` for scope separator.
|
||||
let slot = virtual_dir.join("parent@1.0.0");
|
||||
let modules = slot.join("node_modules");
|
||||
let parent_dir = modules.join("parent");
|
||||
let child_dir = modules.join("child");
|
||||
create_dir_all(&parent_dir).unwrap();
|
||||
create_dir_all(&child_dir).unwrap();
|
||||
|
||||
write_file(
|
||||
parent_dir.join("package.json"),
|
||||
json!({"name": "parent", "version": "1.0.0"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(
|
||||
child_dir.join("package.json"),
|
||||
json!({"name": "child", "version": "1.0.0", "bin": "cli.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(child_dir.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &virtual_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.unwrap();
|
||||
|
||||
let shim_path = parent_dir.join("node_modules/.bin/child");
|
||||
assert!(shim_path.exists(), "expected shim at {shim_path:?}");
|
||||
let body = read_to_string(&shim_path).unwrap();
|
||||
// Layout, with shim at A and target at B, relative path `A → B`:
|
||||
//
|
||||
// <slot>/node_modules/parent/node_modules/.bin/child (shim, A)
|
||||
// <slot>/node_modules/child/cli.js (target, B)
|
||||
//
|
||||
// Common prefix is `<slot>/node_modules`. A has three extra
|
||||
// segments after that (`parent`, `node_modules`, `.bin`); B has
|
||||
// two (`child`, `cli.js`). Relative = `../../../child/cli.js`.
|
||||
assert!(
|
||||
body.contains("\"$basedir/../../../child/cli.js\""),
|
||||
"shim must reference the sibling child via the right number of `..`s, got:\n{body}",
|
||||
);
|
||||
}
|
||||
|
||||
/// A slot whose own package also declares a bin must NOT have that bin
|
||||
/// linked into its own `node_modules/.bin`. pnpm only links *children*
|
||||
/// of a slot, so a tsc slot does not redundantly produce a shim for
|
||||
/// its own tsc binary.
|
||||
///
|
||||
/// To distinguish the exclusion logic from "the slot wasn't processed
|
||||
/// at all," the slot has a real child (`other`) whose bin SHOULD be
|
||||
/// linked. The assertions then check both directions:
|
||||
///
|
||||
/// 1. The child bin appears in `<slot>/node_modules/<own>/node_modules/.bin/`.
|
||||
/// 2. The slot's own bin does NOT appear there.
|
||||
///
|
||||
/// If [`super::find_slot_own_package_dir`] returns `None` (slot skipped),
|
||||
/// (1) fails. If the exclusion logic is dropped, (2) fails. Either
|
||||
/// failure surfaces the regression.
|
||||
#[test]
|
||||
fn skips_slot_own_package_when_walking_children() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let virtual_dir = tmp.path().join(".pacquet");
|
||||
|
||||
let slot = virtual_dir.join("tsc@5.0.0");
|
||||
let modules = slot.join("node_modules");
|
||||
let pkg_dir = modules.join("tsc");
|
||||
let other_dir = modules.join("other");
|
||||
create_dir_all(&pkg_dir).unwrap();
|
||||
create_dir_all(&other_dir).unwrap();
|
||||
|
||||
write_file(
|
||||
pkg_dir.join("package.json"),
|
||||
json!({"name": "tsc", "version": "5.0.0", "bin": "tsc.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(pkg_dir.join("tsc.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
write_file(
|
||||
other_dir.join("package.json"),
|
||||
json!({"name": "other", "version": "1.0.0", "bin": "other.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(other_dir.join("other.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &virtual_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.unwrap();
|
||||
|
||||
let bin_dir = pkg_dir.join("node_modules/.bin");
|
||||
assert!(
|
||||
bin_dir.join("other").exists(),
|
||||
"child bin `other` must be linked under the slot's own package",
|
||||
);
|
||||
assert!(!bin_dir.join("tsc").exists(), "self-bin `tsc` must not be linked into own slot");
|
||||
}
|
||||
|
||||
/// [`LinkVirtualStoreBins`] with a non-existent virtual-store directory
|
||||
/// must be a no-op (`Ok`). A fresh install where the dir doesn't exist
|
||||
/// yet must not error out.
|
||||
#[test]
|
||||
fn link_virtual_store_bins_no_op_when_dir_missing() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let nonexistent = tmp.path().join("does-not-exist");
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &nonexistent,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.expect("missing dir is Ok");
|
||||
}
|
||||
|
||||
/// Slot whose name has a `+` (scope separator) resolves to
|
||||
/// `node_modules/<scope>/<name>`. Pins [`super::find_slot_own_package_dir`]'s
|
||||
/// scoped branch. The un-scoped branch is exercised by the existing
|
||||
/// `writes_child_bins_into_slot_own_package_node_modules` test.
|
||||
#[test]
|
||||
fn link_virtual_store_bins_handles_scoped_slot_name() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let virtual_dir = tmp.path().join(".pacquet");
|
||||
let slot = virtual_dir.join("@scope+parent@1.0.0");
|
||||
let modules = slot.join("node_modules");
|
||||
let parent_dir = modules.join("@scope/parent");
|
||||
let child_dir = modules.join("child");
|
||||
create_dir_all(&parent_dir).unwrap();
|
||||
create_dir_all(&child_dir).unwrap();
|
||||
|
||||
write_file(
|
||||
parent_dir.join("package.json"),
|
||||
json!({"name": "@scope/parent", "version": "1.0.0"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(
|
||||
child_dir.join("package.json"),
|
||||
json!({"name": "child", "version": "1.0.0", "bin": "cli.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(child_dir.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &virtual_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.unwrap();
|
||||
|
||||
let shim = parent_dir.join("node_modules/.bin/child");
|
||||
assert!(shim.exists(), "scoped-slot bin linking must produce a shim at {shim:?}");
|
||||
}
|
||||
|
||||
/// Peer-resolved slots have version segments that contain additional
|
||||
/// `@` characters (one per peer spec). [`super::find_slot_own_package_dir`]
|
||||
/// must parse the package-name boundary from the **left**, not the
|
||||
/// right, otherwise it splits inside a peer spec and silently fails
|
||||
/// to locate the own package. Bins of children of the slot then
|
||||
/// never get linked.
|
||||
///
|
||||
/// Slot name shape verified against
|
||||
/// `pacquet_lockfile::pkg_name_ver_peer::tests::to_virtual_store_name`.
|
||||
/// Pins review finding #5.
|
||||
#[test]
|
||||
fn link_virtual_store_bins_handles_peer_resolved_slot_name() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let virtual_dir = tmp.path().join(".pacquet");
|
||||
|
||||
// pnpm format: `ts-node@10.9.1(@types/node@18.7.19)(typescript@5.1.6)`.
|
||||
// Pacquet's `to_virtual_store_name`: `_` joins peers, `(` and `)`
|
||||
// are stripped, `/` becomes `+`.
|
||||
let slot = virtual_dir.join("ts-node@10.9.1_@types+node@18.7.19_typescript@5.1.6");
|
||||
let modules = slot.join("node_modules");
|
||||
let pkg_dir = modules.join("ts-node");
|
||||
let child_dir = modules.join("child");
|
||||
create_dir_all(&pkg_dir).unwrap();
|
||||
create_dir_all(&child_dir).unwrap();
|
||||
|
||||
write_file(
|
||||
pkg_dir.join("package.json"),
|
||||
json!({"name": "ts-node", "version": "10.9.1"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(
|
||||
child_dir.join("package.json"),
|
||||
json!({"name": "child", "version": "1.0.0", "bin": "cli.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(child_dir.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &virtual_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.unwrap();
|
||||
|
||||
let shim = pkg_dir.join("node_modules/.bin/child");
|
||||
assert!(
|
||||
shim.exists(),
|
||||
"peer-resolved slot's child bin must be linked under the own package's `node_modules/.bin`, \
|
||||
not silently dropped because the slot-name parser split inside a peer spec",
|
||||
);
|
||||
}
|
||||
|
||||
/// An unscoped slot name that contains `+` must be treated as a
|
||||
/// single bare package name, not split on `+` like the scoped form
|
||||
/// `<scope>+<name>`. `PkgName::parse` does not reject non-URL-safe
|
||||
/// characters (npm's `validate-npm-package-name` only emits a
|
||||
/// warning), so a manifest with `"name": "foo+bar"` reaches
|
||||
/// `to_virtual_store_name` unchanged and the slot ends up as
|
||||
/// `foo+bar@1.0.0`. Without gating the `+` split on `scoped`, the
|
||||
/// own-package lookup would build `node_modules/foo/bar` and miss
|
||||
/// the actual `node_modules/foo+bar` directory.
|
||||
#[test]
|
||||
fn link_virtual_store_bins_handles_unscoped_name_with_plus() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let virtual_dir = tmp.path().join(".pacquet");
|
||||
let slot = virtual_dir.join("foo+bar@1.0.0");
|
||||
let modules = slot.join("node_modules");
|
||||
let pkg_dir = modules.join("foo+bar");
|
||||
let child_dir = modules.join("child");
|
||||
create_dir_all(&pkg_dir).unwrap();
|
||||
create_dir_all(&child_dir).unwrap();
|
||||
|
||||
write_file(
|
||||
pkg_dir.join("package.json"),
|
||||
json!({"name": "foo+bar", "version": "1.0.0"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(
|
||||
child_dir.join("package.json"),
|
||||
json!({"name": "child", "version": "1.0.0", "bin": "cli.js"}).to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
write_file(child_dir.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &virtual_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.unwrap();
|
||||
|
||||
let shim = pkg_dir.join("node_modules/.bin/child");
|
||||
assert!(
|
||||
shim.exists(),
|
||||
"unscoped slot name containing `+` must not be split as `<scope>+<name>`; \
|
||||
expected shim at {shim:?}",
|
||||
);
|
||||
}
|
||||
|
||||
/// A virtual-store slot whose `node_modules/` is missing must be skipped
|
||||
/// without error.
|
||||
#[test]
|
||||
fn link_virtual_store_bins_skips_slot_without_node_modules() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let virtual_dir = tmp.path().join(".pacquet");
|
||||
create_dir_all(virtual_dir.join("incomplete@1.0.0")).unwrap();
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &virtual_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
/// A slot whose `node_modules/` exists but lacks the expected own-package
|
||||
/// subdirectory must be skipped without error. The outer `is_dir()`
|
||||
/// check on `<slot>/node_modules` passes, so the slot is processed,
|
||||
/// but [`super::find_slot_own_package_dir`] returns `None` once
|
||||
/// `<slot>/node_modules/<own>` turns out to not be a directory. That
|
||||
/// drives the `let-else return Ok(())` at the call site, the
|
||||
/// uncovered branch real-fs tests with a well-formed slot can never
|
||||
/// reach.
|
||||
#[test]
|
||||
fn link_virtual_store_bins_skips_slot_without_own_package_dir() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let virtual_dir = tmp.path().join(".pacquet");
|
||||
// Slot directory and its `node_modules/` exist, but the expected
|
||||
// `<slot>/node_modules/foo` (matching the slot name `foo@1.0.0`)
|
||||
// is missing, so `find_slot_own_package_dir` returns `None`.
|
||||
create_dir_all(virtual_dir.join("foo@1.0.0/node_modules")).unwrap();
|
||||
LinkVirtualStoreBins {
|
||||
virtual_store_dir: &virtual_dir,
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run()
|
||||
.expect("missing own-package dir is skipped silently");
|
||||
}
|
||||
|
||||
/// [`link_direct_dep_bins`] walks the project's `node_modules/<dep>`
|
||||
/// symlinks and writes a shim per declared bin. End-to-end exercise of
|
||||
/// the path that runs after [`crate::SymlinkDirectDependencies`].
|
||||
#[test]
|
||||
fn link_direct_dep_bins_writes_shims_for_each_dep() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
let foo_dir = modules.join("foo");
|
||||
create_dir_all(&foo_dir).unwrap();
|
||||
write_file(foo_dir.join("package.json"), json!({"name": "foo", "bin": "cli.js"}).to_string())
|
||||
.unwrap();
|
||||
write_file(foo_dir.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
link_direct_dep_bins(&modules, &["foo".to_string()]).unwrap();
|
||||
|
||||
let shim = modules.join(".bin/foo");
|
||||
assert!(shim.exists(), "shim should be created at {shim:?}");
|
||||
let body = read_to_string(&shim).unwrap();
|
||||
assert!(is_shim_pointing_at(&body, &foo_dir.join("cli.js")));
|
||||
}
|
||||
|
||||
/// [`link_direct_dep_bins`] with no deps is a no-op. It must not even
|
||||
/// create the `.bin` directory. Mirrors the early-return of
|
||||
/// [`pacquet_cmd_shim::link_bins_of_packages`].
|
||||
#[test]
|
||||
fn link_direct_dep_bins_no_op_for_empty_dep_list() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
create_dir_all(&modules).unwrap();
|
||||
link_direct_dep_bins(&modules, &[]).unwrap();
|
||||
assert!(!modules.join(".bin").exists());
|
||||
}
|
||||
|
||||
/// [`link_direct_dep_bins`] resolves the dep name through the symlink
|
||||
/// pacquet creates under `<modules_dir>/<name>`. Pin that the manifest
|
||||
/// is read from the symlink's *target*, not the symlink path itself.
|
||||
#[test]
|
||||
fn link_direct_dep_bins_follows_symlink_to_real_package() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
create_dir_all(&modules).unwrap();
|
||||
|
||||
// The "real" package contents live elsewhere (mimics pacquet's
|
||||
// virtual-store layout).
|
||||
let real_pkg = tmp.path().join("virtual/foo@1.0.0/node_modules/foo");
|
||||
create_dir_all(&real_pkg).unwrap();
|
||||
write_file(real_pkg.join("package.json"), json!({"name": "foo", "bin": "cli.js"}).to_string())
|
||||
.unwrap();
|
||||
write_file(real_pkg.join("cli.js"), "#!/usr/bin/env node\n").unwrap();
|
||||
|
||||
// Use the same approach pacquet uses in production: symlink on
|
||||
// Unix, junction on Windows. Plain `std::os::windows::fs::symlink_dir`
|
||||
// requires the `SeCreateSymbolicLinkPrivilege` (off by default on
|
||||
// CI runners), so the test would fail there even though production
|
||||
// never hits that code path.
|
||||
let symlink = modules.join("foo");
|
||||
pacquet_fs::symlink_dir(&real_pkg, &symlink).unwrap();
|
||||
|
||||
link_direct_dep_bins(&modules, &["foo".to_string()]).unwrap();
|
||||
|
||||
assert!(modules.join(".bin/foo").exists(), "symlinked dep must produce a shim");
|
||||
}
|
||||
|
||||
/// Skip dep names whose symlink points at a non-existent target.
|
||||
/// [`link_direct_dep_bins`] filters those silently because the
|
||||
/// surrounding install pipeline has already populated whatever it could.
|
||||
#[test]
|
||||
fn link_direct_dep_bins_skips_dep_with_missing_manifest() {
|
||||
let tmp = tempdir().unwrap();
|
||||
let modules = tmp.path().join("node_modules");
|
||||
create_dir_all(&modules).unwrap();
|
||||
// No `<modules>/foo` directory at all.
|
||||
link_direct_dep_bins(&modules, &["foo".to_string()]).unwrap();
|
||||
assert!(!modules.join(".bin").exists());
|
||||
}
|
||||
|
||||
/// [`LinkVirtualStoreBins::run_with`] propagates a non-`NotFound`
|
||||
/// `read_dir` error on the virtual-store directory itself. Real fs
|
||||
/// can't trigger this portably; the fake forces the
|
||||
/// [`LinkVirtualStoreBinsError::ReadVirtualStore`] variant.
|
||||
#[test]
|
||||
fn link_virtual_store_bins_propagates_read_error_via_di() {
|
||||
use pacquet_cmd_shim::{
|
||||
FsCreateDirAll, FsEnsureExecutableBits, FsReadDir, FsReadFile, FsReadHead, FsReadString,
|
||||
FsSetExecutable, FsWalkFiles, FsWrite,
|
||||
};
|
||||
use std::io;
|
||||
|
||||
struct DenyVirtualStore;
|
||||
impl FsReadDir for DenyVirtualStore {
|
||||
fn read_dir(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
Err::<Empty<PathBuf>, _>(io::Error::from(io::ErrorKind::PermissionDenied))
|
||||
}
|
||||
}
|
||||
impl FsReadFile for DenyVirtualStore {
|
||||
fn read_file(_: &Path) -> io::Result<Vec<u8>> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadString for DenyVirtualStore {
|
||||
fn read_to_string(_: &Path) -> io::Result<String> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsReadHead for DenyVirtualStore {
|
||||
fn read_head(_: &Path, _: u64, _: &mut [u8]) -> io::Result<usize> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsCreateDirAll for DenyVirtualStore {
|
||||
fn create_dir_all(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWrite for DenyVirtualStore {
|
||||
fn write(_: &Path, _: &[u8]) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsSetExecutable for DenyVirtualStore {
|
||||
fn set_executable(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsEnsureExecutableBits for DenyVirtualStore {
|
||||
fn ensure_executable_bits(_: &Path) -> io::Result<()> {
|
||||
unreachable!()
|
||||
}
|
||||
}
|
||||
impl FsWalkFiles for DenyVirtualStore {
|
||||
fn walk_files(_: &Path) -> io::Result<impl Iterator<Item = PathBuf>> {
|
||||
unreachable!("directories.bin not exercised by this test");
|
||||
#[expect(unreachable_code)]
|
||||
Ok(empty())
|
||||
}
|
||||
}
|
||||
|
||||
let err = LinkVirtualStoreBins {
|
||||
virtual_store_dir: Path::new("/anything"),
|
||||
snapshots: None,
|
||||
packages: None,
|
||||
package_manifests: &Default::default(),
|
||||
}
|
||||
.run_with::<DenyVirtualStore>()
|
||||
.expect_err("read_dir error must propagate");
|
||||
assert!(matches!(err, LinkVirtualStoreBinsError::ReadVirtualStore { .. }));
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
use crate::symlink_package;
|
||||
use crate::{link_direct_dep_bins, symlink_package};
|
||||
use derive_more::{Display, Error};
|
||||
use miette::Diagnostic;
|
||||
use pacquet_cmd_shim::LinkBinsError;
|
||||
use pacquet_lockfile::{Lockfile, PkgName, PkgNameVerPeer, ProjectSnapshot};
|
||||
use pacquet_npmrc::Npmrc;
|
||||
use pacquet_package_manifest::DependencyGroup;
|
||||
@@ -37,6 +38,9 @@ pub enum SymlinkDirectDependenciesError {
|
||||
)]
|
||||
#[diagnostic(code(pacquet_package_manager::missing_root_importer))]
|
||||
MissingRootImporter { root_key: String },
|
||||
|
||||
#[diagnostic(transparent)]
|
||||
LinkBins(#[error(source)] LinkBinsError),
|
||||
}
|
||||
|
||||
impl<'a, DependencyGroupList> SymlinkDirectDependencies<'a, DependencyGroupList>
|
||||
@@ -54,7 +58,7 @@ where
|
||||
})?;
|
||||
|
||||
// Iterate per group so each emit can label the dependency
|
||||
// with its [`DependencyType`] — pnpm's reporter renders the
|
||||
// with its [`DependencyType`]. pnpm's reporter renders the
|
||||
// diff with that hint, so dropping it would silently
|
||||
// misclassify devDependencies as prod.
|
||||
// [`ProjectSnapshot::dependencies_by_groups`] flattens the
|
||||
@@ -65,22 +69,22 @@ where
|
||||
// for peer dependencies (they're materialised through their
|
||||
// host package, not directly under `node_modules/`), and
|
||||
// [`ProjectSnapshot::get_map_by_group`] also returns `None`
|
||||
// for `Peer` so this filter is belt-and-braces — it lets
|
||||
// for `Peer` so this filter is belt-and-braces. It lets
|
||||
// the per-group → [`DependencyType`] match below stay
|
||||
// exhaustive without a misleading `Peer` arm that maps to
|
||||
// an "absent" type.
|
||||
//
|
||||
// Dedup with a `HashSet<PkgName>`, first-wins. A v9 lockfile
|
||||
// pnpm itself wrote shouldn't list the same package across
|
||||
// multiple importer sections — pnpm's resolver normalises
|
||||
// (a package with `optional: true` lands in
|
||||
// multiple importer sections (pnpm's resolver normalises:
|
||||
// a package with `optional: true` lands in
|
||||
// `optionalDependencies` only). But pacquet ingests
|
||||
// user-supplied lockfiles, and a malformed one with the same
|
||||
// key in two sections would race two `symlink_package` calls
|
||||
// to the same `node_modules/<name>` and emit duplicate
|
||||
// `pnpm:root added` events. First-wins picks up the highest-
|
||||
// priority group from the caller-supplied
|
||||
// `dependency_groups` order — the CLI today passes
|
||||
// `dependency_groups` order. The CLI today passes
|
||||
// `[Prod, Dev, Optional]`, matching pnpm's
|
||||
// dependencies-over-optional precedence.
|
||||
let mut seen: HashSet<&PkgName> = HashSet::new();
|
||||
@@ -127,14 +131,14 @@ where
|
||||
DependencyGroup::Prod => DependencyType::Prod,
|
||||
DependencyGroup::Dev => DependencyType::Dev,
|
||||
DependencyGroup::Optional => DependencyType::Optional,
|
||||
// Filtered upfront — see the comment on the
|
||||
// `entries` builder above.
|
||||
// Filtered upfront. See the comment on the `entries`
|
||||
// builder above.
|
||||
DependencyGroup::Peer => unreachable!("peers are filtered out before this point"),
|
||||
};
|
||||
// Pacquet's lockfile snapshot doesn't track the
|
||||
// npm-alias key separately from the resolved package
|
||||
// name at this layer, so `name` and `real_name` carry
|
||||
// the same value — clone the already-built string
|
||||
// the same value. Clone the already-built string
|
||||
// instead of formatting `name` a second time.
|
||||
let real_name = name_str.clone();
|
||||
R::emit(&LogEvent::Root(RootLog {
|
||||
@@ -154,6 +158,14 @@ where
|
||||
}));
|
||||
});
|
||||
|
||||
// After the symlinks exist, walk them to discover each
|
||||
// direct dep's `package.json` and link declared bins into
|
||||
// `<modules_dir>/.bin`. Mirrors pnpm v11's `linkBinsOfPackages`
|
||||
// call site for direct deps.
|
||||
let dep_names: Vec<String> = entries.iter().map(|(name, _, _)| name.to_string()).collect();
|
||||
link_direct_dep_bins(&config.modules_dir, &dep_names)
|
||||
.map_err(SymlinkDirectDependenciesError::LinkBins)?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -77,6 +77,7 @@
|
||||
use crate::{CafsFileInfo, PackageFilesIndex, SideEffectsDiff};
|
||||
use derive_more::{Display, Error};
|
||||
use miette::Diagnostic;
|
||||
use serde_json::Value;
|
||||
use smart_default::SmartDefault;
|
||||
use std::{collections::HashMap, rc::Rc};
|
||||
|
||||
@@ -611,11 +612,14 @@ fn write_str(w: &mut Vec<u8>, s: &str) {
|
||||
/// ## Optional-field handling
|
||||
///
|
||||
/// - **`PackageFilesIndex`**: `algo` and `files` are always emitted;
|
||||
/// `requires_build` and `side_effects` are included in the record
|
||||
/// schema only when `Some`. `manifest` is always `None` in pacquet
|
||||
/// today and not yet wired through; the encoder returns
|
||||
/// [`EncodeError::ManifestNotSupported`] if it ever gets a `Some`,
|
||||
/// which is louder than silently dropping it.
|
||||
/// `requires_build`, `manifest`, and `side_effects` are included
|
||||
/// in the record schema only when `Some`. The `manifest`
|
||||
/// ([`serde_json::Value`]) is encoded recursively, with every
|
||||
/// nested JSON object record-encoded so a pnpm reader sees them as
|
||||
/// JS `Object`s (which `manifest.bin` / `manifest.directories?.bin`
|
||||
/// property access can reach) rather than plain msgpack maps
|
||||
/// (which msgpackr decodes as JS `Map`s, leaving those property
|
||||
/// reads `undefined`).
|
||||
/// - **`CafsFileInfo`**: optional `checkedAt` is omitted from the
|
||||
/// record schema entirely when `None` rather than written as `nil`,
|
||||
/// so the presence of `checkedAt` determines the shape and thus
|
||||
@@ -643,25 +647,16 @@ pub fn encode_package_files_index(index: &PackageFilesIndex) -> Result<Vec<u8>,
|
||||
#[derive(Debug, Display, Error, Diagnostic)]
|
||||
#[non_exhaustive]
|
||||
pub enum EncodeError {
|
||||
#[display(
|
||||
"PackageFilesIndex.manifest is Some, but the msgpackr-records \
|
||||
encoder doesn't yet know how to serialize `serde_json::Value` \
|
||||
— pacquet doesn't populate this field today, so the code path \
|
||||
is unimplemented. Add manifest encoding if/when pacquet starts \
|
||||
writing bundled manifests."
|
||||
)]
|
||||
#[diagnostic(code(pacquet_store_dir::msgpackr_records::manifest_not_supported))]
|
||||
ManifestNotSupported,
|
||||
|
||||
#[display(
|
||||
"Ran out of msgpackr record slots: encountered more than \
|
||||
{max} distinct record shapes (slot range is 0x41..=0x7f). \
|
||||
This shouldn't happen for current pacquet payloads — \
|
||||
`CafsFileInfo` has at most 2 shapes and `SideEffectsDiff` \
|
||||
at most 4. Reaching this error likely means a new record \
|
||||
type was added to the encoder without bumping the shape \
|
||||
accounting, or the encoder is being reused for a schema \
|
||||
it wasn't designed for."
|
||||
`CafsFileInfo` contributes at most 2 shapes and \
|
||||
`SideEffectsDiff` at most 4; the rest are allocated lazily \
|
||||
from `PackageFilesIndex.manifest`'s nested object shapes, \
|
||||
which in practice fit comfortably inside the remaining \
|
||||
range for a single tarball's manifest. Reaching this error \
|
||||
likely means the encoder is being reused for a payload it \
|
||||
wasn't designed for."
|
||||
)]
|
||||
#[diagnostic(code(pacquet_store_dir::msgpackr_records::out_of_record_slots))]
|
||||
OutOfRecordSlots { max: usize },
|
||||
@@ -698,6 +693,17 @@ struct EncodeState {
|
||||
/// Same for `SideEffectsDiff`, indexed by [`side_effects_shape`]
|
||||
/// (4 possible values).
|
||||
side_effects_slots: [Option<u8>; 4],
|
||||
/// Field-name vector → slot for every JSON-object shape seen so
|
||||
/// far inside a manifest value. Shape keys are owned `Vec<String>`
|
||||
/// because the field names are read from a borrowed
|
||||
/// `serde_json::Map` whose lifetime ends before the next encode
|
||||
/// call wants the lookup. msgpackr does the equivalent thing for
|
||||
/// arbitrary JS objects under `useRecords: true`; pacquet has to
|
||||
/// match so a pnpm reader sees the manifest's nested objects as JS
|
||||
/// `Object`s (record-decoded) rather than `Map`s (plain-msgpack-
|
||||
/// decoded), which is what pnpm's bin linker reads with
|
||||
/// `manifest.bin` / `manifest.directories?.bin` property access.
|
||||
json_object_slots: HashMap<Vec<String>, u8>,
|
||||
/// Next unused slot in the 0x41..=0x7f range. Starts above
|
||||
/// `PKG_FILES_INDEX_SLOT` because the top-level record always
|
||||
/// takes slot 0x40.
|
||||
@@ -736,23 +742,18 @@ fn encode_pkg_files_index_value(
|
||||
state: &mut EncodeState,
|
||||
idx: &PackageFilesIndex,
|
||||
) -> Result<(), EncodeError> {
|
||||
if idx.manifest.is_some() {
|
||||
return Err(EncodeError::ManifestNotSupported);
|
||||
}
|
||||
|
||||
// Field order `[algo, requiresBuild?, files, sideEffects?]`
|
||||
// matches what pnpm's msgpackr emits, as verified by the
|
||||
// `decodes_requires_build_true` and `decodes_side_effects`
|
||||
// fixtures captured from msgpackr 1.11.8 right here in this
|
||||
// module. Optional fields are omitted from the schema when
|
||||
// `None`. `manifest` stays out of the schema entirely until
|
||||
// pacquet starts populating it (the `Some` check above bails
|
||||
// before we get here).
|
||||
let mut fields: Vec<&str> = Vec::with_capacity(4);
|
||||
// Field order `[algo, requiresBuild?, manifest?, files, sideEffects?]`.
|
||||
// Optional fields are omitted from the schema when `None`, matching
|
||||
// msgpackr's field-omit-when-absent shape so a pnpm reader sees the
|
||||
// same JS object regardless of whether pacquet or pnpm wrote the row.
|
||||
let mut fields: Vec<&str> = Vec::with_capacity(5);
|
||||
fields.push("algo");
|
||||
if idx.requires_build.is_some() {
|
||||
fields.push("requiresBuild");
|
||||
}
|
||||
if idx.manifest.is_some() {
|
||||
fields.push("manifest");
|
||||
}
|
||||
fields.push("files");
|
||||
if idx.side_effects.is_some() {
|
||||
fields.push("sideEffects");
|
||||
@@ -765,6 +766,9 @@ fn encode_pkg_files_index_value(
|
||||
if let Some(rb) = idx.requires_build {
|
||||
write_bool(w, rb);
|
||||
}
|
||||
if let Some(manifest) = &idx.manifest {
|
||||
encode_json_value(w, state, manifest)?;
|
||||
}
|
||||
write_map_header(w, idx.files.len());
|
||||
for (name, info) in &idx.files {
|
||||
write_str(w, name);
|
||||
@@ -781,6 +785,94 @@ fn encode_pkg_files_index_value(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Emit one JSON value as msgpack inside an active records stream.
|
||||
/// Scalars use the smallest slot-safe encoding (no bare positive
|
||||
/// fixints in `0x40..=0x7f`, which would otherwise be misread as
|
||||
/// record-slot references — see [`write_uint`]). Arrays are plain
|
||||
/// msgpack `fixarray` / `array16` / `array32`. Objects are
|
||||
/// **record-encoded** via [`encode_json_object`] so that
|
||||
/// `useRecords: true` decoders see them as JS `Object` rather than
|
||||
/// JS `Map` — necessary for pnpm's bin linker to find
|
||||
/// `manifest.bin` / `manifest.directories?.bin` via property
|
||||
/// access.
|
||||
fn encode_json_value(
|
||||
w: &mut Vec<u8>,
|
||||
state: &mut EncodeState,
|
||||
value: &Value,
|
||||
) -> Result<(), EncodeError> {
|
||||
match value {
|
||||
Value::Null => w.push(0xc0),
|
||||
Value::Bool(b) => write_bool(w, *b),
|
||||
Value::Number(n) => encode_json_number(w, n),
|
||||
Value::String(s) => write_str(w, s),
|
||||
Value::Array(arr) => {
|
||||
write_array_header(w, arr.len());
|
||||
for item in arr {
|
||||
encode_json_value(w, state, item)?;
|
||||
}
|
||||
}
|
||||
Value::Object(obj) => encode_json_object(w, state, obj)?,
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Record-encode a JSON object: allocate one slot per distinct key
|
||||
/// set seen in the current stream, emit a record def the first time
|
||||
/// each shape appears, and emit a bare slot byte on subsequent
|
||||
/// instances of the same shape. The slot table lives on
|
||||
/// [`EncodeState::json_object_slots`] so reuse compresses repeated
|
||||
/// nested-object shapes (e.g. multiple `bin: { command: path }`
|
||||
/// objects with the same single command name) the same way
|
||||
/// msgpackr's records mode does.
|
||||
///
|
||||
/// Field iteration order is the [`serde_json::Map`]'s own order;
|
||||
/// pacquet builds with `serde_json/preserve_order`, so that's the
|
||||
/// insertion order from parsing the original `package.json` — the
|
||||
/// same order pnpm itself observes when packing the manifest.
|
||||
fn encode_json_object(
|
||||
w: &mut Vec<u8>,
|
||||
state: &mut EncodeState,
|
||||
obj: &serde_json::Map<String, Value>,
|
||||
) -> Result<(), EncodeError> {
|
||||
let fields: Vec<String> = obj.keys().cloned().collect();
|
||||
if let Some(&slot) = state.json_object_slots.get(&fields) {
|
||||
w.push(slot);
|
||||
} else {
|
||||
let slot = state.allocate_slot()?;
|
||||
let field_refs: Vec<&str> = fields.iter().map(String::as_str).collect();
|
||||
write_record_def_header(w, slot, &field_refs);
|
||||
state.json_object_slots.insert(fields, slot);
|
||||
}
|
||||
for value in obj.values() {
|
||||
encode_json_value(w, state, value)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Encode a [`serde_json::Number`] using the smallest slot-safe
|
||||
/// MessagePack form. The branch order matches what pnpm's msgpackr
|
||||
/// itself picks: any integer value first (so a JSON `1.0` parsed as
|
||||
/// `Number(1)` stays an integer on the wire), falling through to
|
||||
/// `float 64` only when the number genuinely needs the precision.
|
||||
fn encode_json_number(w: &mut Vec<u8>, n: &serde_json::Number) {
|
||||
if let Some(u) = n.as_u64() {
|
||||
write_uint(w, u);
|
||||
return;
|
||||
}
|
||||
if let Some(i) = n.as_i64() {
|
||||
write_int(w, i);
|
||||
return;
|
||||
}
|
||||
if let Some(f) = n.as_f64() {
|
||||
write_float64(w, f);
|
||||
}
|
||||
// Unreachable: a `serde_json::Number` is always one of the three
|
||||
// cases above. If it isn't (a future serde_json release adds a
|
||||
// new representation), the wire output would be missing a value
|
||||
// for this field, which would surface as a deserialize error on
|
||||
// round-trip — louder than silent corruption.
|
||||
}
|
||||
|
||||
fn encode_cafs_file_info(
|
||||
w: &mut Vec<u8>,
|
||||
state: &mut EncodeState,
|
||||
@@ -924,6 +1016,32 @@ fn write_float64(w: &mut Vec<u8>, v: f64) {
|
||||
w.extend_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
|
||||
/// Write a signed integer in the smallest MessagePack encoding.
|
||||
/// Negative values use the int 8/16/32/64 family (`0xd0..=0xd3`),
|
||||
/// whose header bytes are outside the records-mode slot range so
|
||||
/// they're always safe to emit. Non-negative values delegate to
|
||||
/// [`write_uint`] which handles the slot-byte fixint promotion.
|
||||
fn write_int(w: &mut Vec<u8>, v: i64) {
|
||||
if v >= 0 {
|
||||
write_uint(w, v as u64);
|
||||
} else if v >= -32 {
|
||||
// Negative fixint `0xe0..=0xff`; outside slot range.
|
||||
w.push(v as i8 as u8);
|
||||
} else if v >= i8::MIN as i64 {
|
||||
w.push(0xd0);
|
||||
w.push(v as i8 as u8);
|
||||
} else if v >= i16::MIN as i64 {
|
||||
w.push(0xd1);
|
||||
w.extend_from_slice(&(v as i16).to_be_bytes());
|
||||
} else if v >= i32::MIN as i64 {
|
||||
w.push(0xd2);
|
||||
w.extend_from_slice(&(v as i32).to_be_bytes());
|
||||
} else {
|
||||
w.push(0xd3);
|
||||
w.extend_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
}
|
||||
|
||||
fn write_bool(w: &mut Vec<u8>, b: bool) {
|
||||
w.push(if b { 0xc3 } else { 0xc2 });
|
||||
}
|
||||
|
||||
@@ -594,19 +594,137 @@ fn allocate_slot_returns_error_past_0x7f() {
|
||||
assert!(matches!(err, EncodeError::OutOfRecordSlots { max: 63 }), "got {err:?}");
|
||||
}
|
||||
|
||||
/// A `manifest: Some(_)` must round-trip through encode →
|
||||
/// transcode → `rmp_serde::from_slice` unchanged. This is the basic
|
||||
/// "the encoder doesn't drop or mangle JSON values" smoke test.
|
||||
#[test]
|
||||
fn encode_rejects_manifest_some() {
|
||||
// pacquet doesn't populate `manifest` today; encoding a
|
||||
// `Some` value is unimplemented. Fail loudly rather than
|
||||
// silently dropping it — if/when the field starts carrying
|
||||
// real data, this test trips and we implement the path.
|
||||
let idx = PackageFilesIndex {
|
||||
manifest: Some(serde_json::json!({ "name": "x" })),
|
||||
fn encode_roundtrips_simple_manifest() {
|
||||
let manifest = serde_json::json!({
|
||||
"name": "pkg",
|
||||
"version": "1.0.0",
|
||||
"bin": "cli.js",
|
||||
});
|
||||
let original = PackageFilesIndex {
|
||||
manifest: Some(manifest),
|
||||
requires_build: None,
|
||||
algo: "sha512".to_string(),
|
||||
files: HashMap::new(),
|
||||
side_effects: None,
|
||||
};
|
||||
let err = encode_package_files_index(&idx).unwrap_err();
|
||||
assert!(matches!(err, EncodeError::ManifestNotSupported), "got {err:?}");
|
||||
assert_eq!(roundtrip(&original), original);
|
||||
}
|
||||
|
||||
/// Nested objects inside the manifest must be **record-encoded**, not
|
||||
/// emitted as plain msgpack maps — otherwise msgpackr in
|
||||
/// `useRecords: true` mode would decode them as JS `Map` and
|
||||
/// `manifest.bin.tsc` (the property access pnpm's bin linker uses
|
||||
/// for object-form `bin` fields) would be `undefined`. The encoder
|
||||
/// signals this by emitting one `d4 72 <slot>` def per distinct
|
||||
/// nested-object shape.
|
||||
#[test]
|
||||
fn encode_record_encodes_nested_objects_in_manifest() {
|
||||
let manifest = serde_json::json!({
|
||||
"name": "tsc",
|
||||
"version": "5.0.0",
|
||||
"bin": { "tsc": "bin/tsc.js", "tsserver": "bin/tsserver.js" },
|
||||
"directories": { "bin": "bin" },
|
||||
});
|
||||
let idx = PackageFilesIndex {
|
||||
manifest: Some(manifest),
|
||||
requires_build: None,
|
||||
algo: "sha512".to_string(),
|
||||
files: HashMap::new(),
|
||||
side_effects: None,
|
||||
};
|
||||
let bytes = encode_package_files_index(&idx).unwrap();
|
||||
|
||||
// Outer PackageFilesIndex def + nested manifest object def + nested
|
||||
// `bin` object def + nested `directories` object def = 4 defs.
|
||||
let record_defs = bytes.windows(2).filter(|w| *w == [0xd4, RECORD_DEF_EXT_TYPE]).count();
|
||||
assert_eq!(
|
||||
record_defs, 4,
|
||||
"expected 4 record defs (outer + manifest + bin + directories), got bytes {bytes:02x?}",
|
||||
);
|
||||
|
||||
// Round-trip the manifest through the transcoder to verify the
|
||||
// bytes a msgpackr 1.11.8 reader would consume parse cleanly.
|
||||
assert_eq!(roundtrip(&idx), idx);
|
||||
}
|
||||
|
||||
/// Two nested objects with the **same** key set within the same
|
||||
/// manifest must share a slot. Verifies record-reuse — the whole
|
||||
/// point of records. Shape here is the *key set*, not just the
|
||||
/// arity: `{left-pad}` and `{right-pad}` are different shapes
|
||||
/// (different keys), so the test uses two objects that genuinely
|
||||
/// share keys.
|
||||
#[test]
|
||||
fn encode_shares_slot_for_same_shaped_nested_objects() {
|
||||
let manifest = serde_json::json!({
|
||||
"bin": { "cli": "bin/cli.js" },
|
||||
"directories": { "cli": "src" },
|
||||
});
|
||||
let idx = PackageFilesIndex {
|
||||
manifest: Some(manifest),
|
||||
requires_build: None,
|
||||
algo: "sha512".to_string(),
|
||||
files: HashMap::new(),
|
||||
side_effects: None,
|
||||
};
|
||||
let bytes = encode_package_files_index(&idx).unwrap();
|
||||
let record_defs = bytes.windows(2).filter(|w| *w == [0xd4, RECORD_DEF_EXT_TYPE]).count();
|
||||
// Outer + manifest + ONE shape `{ cli }` shared by both nested
|
||||
// maps = 3 defs. If the encoder allocated a new slot per
|
||||
// instance instead of sharing, this would be 4.
|
||||
assert_eq!(
|
||||
record_defs, 3,
|
||||
"expected slot reuse for same-shape objects, got bytes {bytes:02x?}",
|
||||
);
|
||||
assert_eq!(roundtrip(&idx), idx);
|
||||
}
|
||||
|
||||
/// All JSON value kinds (null, bool, number, string, array, object)
|
||||
/// must round-trip. Numbers cover the slot-byte fixint range so the
|
||||
/// `0x40..=0x7f` → `uint 8` promotion in [`write_uint`] is exercised
|
||||
/// from inside the manifest encoding path too.
|
||||
#[test]
|
||||
fn encode_roundtrips_all_json_value_kinds() {
|
||||
let manifest = serde_json::json!({
|
||||
"string": "hello",
|
||||
"null": null,
|
||||
"true": true,
|
||||
"false": false,
|
||||
"small_int": 0,
|
||||
"slot_range_int": 0x42,
|
||||
"big_int": 1_000_000_u64,
|
||||
"negative_int": -5,
|
||||
"float": 3.5,
|
||||
"array": ["a", 1, null, true, [1, 2], { "k": "v" }],
|
||||
"empty_array": [],
|
||||
"empty_object": {},
|
||||
});
|
||||
let idx = PackageFilesIndex {
|
||||
manifest: Some(manifest),
|
||||
requires_build: None,
|
||||
algo: "sha512".to_string(),
|
||||
files: HashMap::new(),
|
||||
side_effects: None,
|
||||
};
|
||||
assert_eq!(roundtrip(&idx), idx);
|
||||
}
|
||||
|
||||
/// Manifest must round-trip alongside `requiresBuild`, `files`, and
|
||||
/// `sideEffects` — the encoder has to emit all five fields in the
|
||||
/// expected order and the decoder still needs to recover each.
|
||||
#[test]
|
||||
fn encode_roundtrips_manifest_with_other_fields() {
|
||||
let mut files = HashMap::new();
|
||||
files.insert("package.json".to_string(), sample_cafs(42, false));
|
||||
let original = PackageFilesIndex {
|
||||
manifest: Some(serde_json::json!({ "name": "x", "bin": "cli.js" })),
|
||||
requires_build: Some(true),
|
||||
algo: "sha512".to_string(),
|
||||
files,
|
||||
side_effects: None,
|
||||
};
|
||||
assert_eq!(roundtrip(&original), original);
|
||||
}
|
||||
|
||||
@@ -357,7 +357,39 @@ impl StoreIndex {
|
||||
&self,
|
||||
keys: &[String],
|
||||
) -> Result<HashMap<String, PackageFilesIndex>, StoreIndexError> {
|
||||
let mut out = HashMap::with_capacity(keys.len());
|
||||
let raw = self.get_many_raw(keys)?;
|
||||
let mut out = HashMap::with_capacity(raw.len());
|
||||
for (key, bytes) in raw {
|
||||
match decode_index_value(&bytes) {
|
||||
Ok(entry) => {
|
||||
out.insert(key, entry);
|
||||
}
|
||||
Err(error) => tracing::debug!(
|
||||
target: "pacquet::store_index",
|
||||
?key,
|
||||
?error,
|
||||
"skipping undecodable package_index row in get_many",
|
||||
),
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Batched read that returns the **undecoded** value bytes for each
|
||||
/// hit. Callers run `decode_index_value` themselves — either inline
|
||||
/// (matching the old [`Self::get_many`] shape) or, more usefully,
|
||||
/// in parallel across a rayon pool after releasing the
|
||||
/// [`SharedReadonlyStoreIndex`] mutex.
|
||||
///
|
||||
/// The decode is the dominant CPU cost of [`Self::get_many`] for
|
||||
/// rows that carry a `manifest` field — msgpackr-records transcode
|
||||
/// plus a `rmp_serde::from_slice` of a nested JSON tree per row,
|
||||
/// times ~1k rows on a real lockfile. Doing that work under the
|
||||
/// `SharedReadonlyStoreIndex` lock serialises N installs back to
|
||||
/// one thread; doing it after the lock releases lets each prefetch
|
||||
/// fan out across the rayon pool.
|
||||
pub fn get_many_raw(&self, keys: &[String]) -> Result<Vec<(String, Vec<u8>)>, StoreIndexError> {
|
||||
let mut out = Vec::with_capacity(keys.len());
|
||||
if keys.is_empty() {
|
||||
return Ok(out);
|
||||
}
|
||||
@@ -381,18 +413,8 @@ impl StoreIndex {
|
||||
.query_map(params, |row| Ok((row.get::<_, String>(0)?, row.get::<_, Vec<u8>>(1)?)))
|
||||
.map_err(|source| StoreIndexError::Read { source })?;
|
||||
for row in rows {
|
||||
let (key, bytes) = row.map_err(|source| StoreIndexError::Read { source })?;
|
||||
match decode_index_value(&bytes) {
|
||||
Ok(entry) => {
|
||||
out.insert(key, entry);
|
||||
}
|
||||
Err(error) => tracing::debug!(
|
||||
target: "pacquet::store_index",
|
||||
?key,
|
||||
?error,
|
||||
"skipping undecodable package_index row in get_many",
|
||||
),
|
||||
}
|
||||
let row = row.map_err(|source| StoreIndexError::Read { source })?;
|
||||
out.push(row);
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
@@ -508,6 +530,16 @@ impl StoreIndex {
|
||||
}
|
||||
}
|
||||
|
||||
/// Decode one `package_index.data` blob into a [`PackageFilesIndex`].
|
||||
/// Exposed publicly so callers reading raw rows via
|
||||
/// [`StoreIndex::get_many_raw`] can run the decode outside the
|
||||
/// store-index mutex (typically across a rayon pool — the decode
|
||||
/// is the dominant CPU cost for rows that carry a `manifest`
|
||||
/// field).
|
||||
pub fn decode_package_files_index(bytes: &[u8]) -> Result<PackageFilesIndex, StoreIndexError> {
|
||||
decode_index_value(bytes)
|
||||
}
|
||||
|
||||
fn decode_index_value(bytes: &[u8]) -> Result<PackageFilesIndex, StoreIndexError> {
|
||||
// `transcode_to_plain_msgpack` tracks records-mode internally and
|
||||
// only reinterprets `0x40..=0x7f` as slot references after a record
|
||||
|
||||
@@ -23,6 +23,7 @@ futures-util = { workspace = true }
|
||||
miette = { workspace = true }
|
||||
num_cpus = { workspace = true }
|
||||
pipe-trait = { workspace = true }
|
||||
rayon = { workspace = true }
|
||||
reqwest = { workspace = true }
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
|
||||
@@ -20,6 +20,7 @@ use pacquet_store_dir::{
|
||||
StoreIndexError, StoreIndexWriter, WriteCasFileError, store_index_key,
|
||||
};
|
||||
use pipe_trait::Pipe;
|
||||
use rayon::prelude::*;
|
||||
use smart_default::SmartDefault;
|
||||
use ssri::Integrity;
|
||||
use tar::Archive;
|
||||
@@ -294,6 +295,101 @@ fn decompress_gzip(gz_data: &[u8], unpacked_size: Option<usize>) -> Result<Vec<u
|
||||
.map_err(TarballError::DecodeGzip)
|
||||
}
|
||||
|
||||
/// Mirror of pnpm's `normalizeBundledManifest` at
|
||||
/// <https://github.com/pnpm/pnpm/blob/4750fd370c/store/cafs/src/normalizeBundledManifest.ts>:
|
||||
/// pick the subset of `package.json` fields that downstream code
|
||||
/// (bin linking, dependency resolution, build-script detection)
|
||||
/// actually reads, and discard the rest. Two reasons for the
|
||||
/// subset: (1) `index.db` row size on disk — a full manifest can be
|
||||
/// tens of KB; (2) msgpackr-records slot pressure on the encoder
|
||||
/// (record slots top out at `0x7f`, see [`pacquet_store_dir::EncodeError::OutOfRecordSlots`]).
|
||||
///
|
||||
/// `scripts` is further narrowed to just the three lifecycle hooks
|
||||
/// pnpm actually executes — `preinstall`, `install`, `postinstall`.
|
||||
/// Other script keys (test, build, lint, etc.) are dev ergonomics
|
||||
/// that the installer never invokes.
|
||||
///
|
||||
/// Returns `None` when nothing survives the pick. In practice this
|
||||
/// only happens for inputs that aren't a JSON object at all (the
|
||||
/// type guard at the top of the function), or for objects whose
|
||||
/// every field is either absent or `null`. A real `package.json`
|
||||
/// from an npm tarball always carries at least `name` and `version`
|
||||
/// (both kept by the pick), so the typical npm-published manifest
|
||||
/// surfaces as `Some(...)`. Matches upstream's
|
||||
/// `if (!result && !scripts) return undefined` shape: empty inputs
|
||||
/// degrade to `None` rather than a `Some(Object({}))` that would
|
||||
/// round-trip as a zero-field record def.
|
||||
fn normalize_bundled_manifest(value: &serde_json::Value) -> Option<serde_json::Value> {
|
||||
/// Fields kept verbatim from the source manifest.
|
||||
///
|
||||
/// Order matters for the on-wire byte sequence — msgpackr emits
|
||||
/// fields in JS object insertion order, and pacquet's encoder
|
||||
/// follows the [`serde_json::Map`] iteration order — but it
|
||||
/// does *not* matter for property-access correctness on the
|
||||
/// pnpm side. The order below mirrors pnpm's
|
||||
/// `BUNDLED_MANIFEST_FIELDS` array so a side-by-side byte diff
|
||||
/// against a pnpm-written row is shallower.
|
||||
const BUNDLED_MANIFEST_FIELDS: &[&str] = &[
|
||||
"bin",
|
||||
"bundledDependencies",
|
||||
"bundleDependencies",
|
||||
"cpu",
|
||||
"dependencies",
|
||||
"devDependencies",
|
||||
"directories",
|
||||
"engines",
|
||||
"libc",
|
||||
"name",
|
||||
"optionalDependencies",
|
||||
"os",
|
||||
"peerDependencies",
|
||||
"peerDependenciesMeta",
|
||||
];
|
||||
const LIFECYCLE_SCRIPTS: &[&str] = &["preinstall", "install", "postinstall"];
|
||||
|
||||
let serde_json::Value::Object(map) = value else { return None };
|
||||
let mut picked = serde_json::Map::new();
|
||||
|
||||
// pnpm emits `version` first regardless of whether it was first
|
||||
// in the source object. Keep the same ordering so a byte diff
|
||||
// against a pnpm-written row stays minimal. Version normalization
|
||||
// via `semver.clean(...)` (pnpm only loose-cleans for the bundled
|
||||
// row, not for resolution) is intentionally skipped: the inputs
|
||||
// from a real npm tarball are already semver-clean in practice,
|
||||
// and pulling `node-semver` into `pacquet-tarball` purely for
|
||||
// this normalization would carry more risk than the deviation it
|
||||
// closes.
|
||||
if let Some(v) = map.get("version")
|
||||
&& !v.is_null()
|
||||
{
|
||||
picked.insert("version".to_string(), v.clone());
|
||||
}
|
||||
|
||||
for &key in BUNDLED_MANIFEST_FIELDS {
|
||||
if let Some(v) = map.get(key)
|
||||
&& !v.is_null()
|
||||
{
|
||||
picked.insert(key.to_string(), v.clone());
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(serde_json::Value::Object(scripts)) = map.get("scripts") {
|
||||
let mut sub = serde_json::Map::new();
|
||||
for &key in LIFECYCLE_SCRIPTS {
|
||||
if let Some(s) = scripts.get(key)
|
||||
&& !s.is_null()
|
||||
{
|
||||
sub.insert(key.to_string(), s.clone());
|
||||
}
|
||||
}
|
||||
if !sub.is_empty() {
|
||||
picked.insert("scripts".to_string(), serde_json::Value::Object(sub));
|
||||
}
|
||||
}
|
||||
|
||||
if picked.is_empty() { None } else { Some(serde_json::Value::Object(picked)) }
|
||||
}
|
||||
|
||||
/// Walk a decompressed tar archive, writing each regular-file entry
|
||||
/// into the CAFS and returning the `{in-tarball path → CAFS path}` map
|
||||
/// plus the per-tarball [`PackageFilesIndex`] row to hand off to the
|
||||
@@ -411,6 +507,44 @@ fn extract_tarball_entries(
|
||||
tracing::warn!(?previous, "Duplication detected. Old entry has been ejected");
|
||||
}
|
||||
|
||||
// Capture the parsed manifest whenever we see `package.json`.
|
||||
// Mirrors pnpm's `bundledManifest` pass-through at
|
||||
// [pnpm/pnpm@4750fd370c]: pnpm stuffs the narrowed manifest
|
||||
// into `pkgFilesIndex.manifest` so install-side consumers
|
||||
// (notably `linkBinsOfDependencies`) can avoid re-reading
|
||||
// the file from disk. The [`normalize_bundled_manifest`]
|
||||
// pick drops fields downstream code doesn't use, keeping
|
||||
// `index.db` rows tight.
|
||||
//
|
||||
// **Last-entry wins.** Pnpm's [`addFilesFromTarball`] always
|
||||
// overwrites `manifestBuffer = fileBuffer` per `package.json`
|
||||
// entry (no `if (manifestBuffer === undefined)` guard), so
|
||||
// when a tarball contains duplicate `package.json` entries
|
||||
// the final one is canonical — same shape as
|
||||
// `filesIndex.set(...)` which already overwrites duplicates.
|
||||
// Real npm tarballs never publish multiple `package.json`
|
||||
// entries, but the consistency with the `files` map is what
|
||||
// matters: `manifest` and `files` must describe the same
|
||||
// file. Failed JSON parses degrade the field to `None` (the
|
||||
// manifest is best-effort; a corrupt `package.json` is the
|
||||
// publisher's fault and downstream code can fall back to
|
||||
// disk reads).
|
||||
//
|
||||
// [pnpm/pnpm@4750fd370c]: https://github.com/pnpm/pnpm/blob/4750fd370c/worker/src/start.ts#L218
|
||||
// [`addFilesFromTarball`]: https://github.com/pnpm/pnpm/blob/4750fd370c/store/cafs/src/addFilesFromTarball.ts#L41-L43
|
||||
if cleaned_entry_path == "package.json" {
|
||||
match serde_json::from_slice::<serde_json::Value>(&buffer) {
|
||||
Ok(parsed) => pkg_files_idx.manifest = normalize_bundled_manifest(&parsed),
|
||||
Err(error) => {
|
||||
tracing::debug!(
|
||||
?error,
|
||||
"package.json in tarball failed to parse as JSON; bundled manifest cleared",
|
||||
);
|
||||
pkg_files_idx.manifest = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// `as_millis()` returns `u128`; narrow to `u64` to match the
|
||||
// store index schema — see `CafsFileInfo::checked_at` for why
|
||||
// `u64` is used. Using `u64::try_from` rather than `as u64`
|
||||
@@ -470,6 +604,35 @@ fn extract_tarball_entries(
|
||||
/// as a hot-path cost).
|
||||
pub type PrefetchedCasPaths = HashMap<String, Arc<HashMap<String, PathBuf>>>;
|
||||
|
||||
/// Bundled package manifests recovered from the SQLite store index,
|
||||
/// keyed by the same `<integrity>\t<pkg_id>` string [`PrefetchedCasPaths`]
|
||||
/// uses. Mirrors pnpm's `bundledManifest` cache in
|
||||
/// [`worker/src/start.ts`](https://github.com/pnpm/pnpm/blob/4750fd370c/worker/src/start.ts#L144):
|
||||
/// pnpm reads the parsed manifest out of `pkgFilesIndex.manifest` so
|
||||
/// `linkBinsOfDependencies` doesn't have to re-read `package.json`
|
||||
/// from disk per child. Each value is `Arc`-wrapped so multiple
|
||||
/// bin-link consumers can hold the same parsed manifest without
|
||||
/// deep-cloning.
|
||||
///
|
||||
/// Only keys whose row carried a manifest blob appear in the map —
|
||||
/// a missing key means either "row exists but has no manifest" (old
|
||||
/// pacquet write, or a tarball whose `package.json` failed to
|
||||
/// parse) or "package wasn't prefetched at all". Callers that need
|
||||
/// to tell those apart cross-reference with [`PrefetchedCasPaths`]
|
||||
/// from the same [`PrefetchResult`].
|
||||
pub type PrefetchedManifests = HashMap<String, Arc<serde_json::Value>>;
|
||||
|
||||
/// Output of [`prefetch_cas_paths`]: the warm-cache filesystem map
|
||||
/// plus any bundled manifests recovered from the same `index.db`
|
||||
/// rows. Bundled in a single struct so callers can destructure both
|
||||
/// after one `await`, rather than the function having to thread two
|
||||
/// separate spawn_blocking round-trips through.
|
||||
#[derive(Default)]
|
||||
pub struct PrefetchResult {
|
||||
pub cas_paths: PrefetchedCasPaths,
|
||||
pub manifests: PrefetchedManifests,
|
||||
}
|
||||
|
||||
/// Batch the entire warm-cache lookup phase into one `spawn_blocking`
|
||||
/// task at install start: collect every row the lockfile is going to
|
||||
/// ask about under a single `index.lock()` round-trip, drop the lock,
|
||||
@@ -505,55 +668,100 @@ pub async fn prefetch_cas_paths(
|
||||
cache_keys: Vec<String>,
|
||||
verify_store_integrity: bool,
|
||||
verified_files_cache: SharedVerifiedFilesCache,
|
||||
) -> PrefetchedCasPaths {
|
||||
let Some(index) = index else { return HashMap::new() };
|
||||
) -> PrefetchResult {
|
||||
let Some(index) = index else { return PrefetchResult::default() };
|
||||
if cache_keys.is_empty() {
|
||||
return HashMap::new();
|
||||
return PrefetchResult::default();
|
||||
}
|
||||
let result = tokio::task::spawn_blocking(move || -> PrefetchedCasPaths {
|
||||
// Phase 1: read every row under the mutex; drop the guard
|
||||
// before running any filesystem work. One batched
|
||||
// `SELECT ... WHERE key IN (?, ?, ...)` per `GET_MANY_CHUNK`
|
||||
// (see `StoreIndex::get_many`) collapses what used to be N
|
||||
// round-trips into one — see #294 for the cold-cache regression
|
||||
// the per-key loop introduced when every key missed.
|
||||
let entries: HashMap<String, PackageFilesIndex> = {
|
||||
let result = tokio::task::spawn_blocking(move || -> PrefetchResult {
|
||||
// Phase 1: read every row's *raw bytes* under the mutex.
|
||||
// Splitting raw-read from decode means the
|
||||
// `SharedReadonlyStoreIndex` lock is held only for the
|
||||
// SELECT loop, not for the per-row msgpackr decode — which
|
||||
// is the dominant CPU cost once rows carry a `manifest`
|
||||
// field (transcode + `rmp_serde::from_slice` of a nested
|
||||
// JSON tree per row, times ~1k rows on a real lockfile).
|
||||
// Doing the decode after the guard drops lets it fan out
|
||||
// across rayon below.
|
||||
//
|
||||
// One batched `SELECT ... WHERE key IN (?, ?, ...)` per
|
||||
// `GET_MANY_CHUNK` (see `StoreIndex::get_many_raw`)
|
||||
// collapses what used to be N round-trips into one — see
|
||||
// #294 for the cold-cache regression the per-key loop
|
||||
// introduced when every key missed.
|
||||
let raw: Vec<(String, Vec<u8>)> = {
|
||||
let Ok(guard) = index.lock() else {
|
||||
tracing::debug!(
|
||||
target: "pacquet::download",
|
||||
"store-index mutex poisoned at prefetch start; falling back to per-snapshot lookups",
|
||||
);
|
||||
return HashMap::new();
|
||||
return PrefetchResult::default();
|
||||
};
|
||||
match guard.get_many(&cache_keys) {
|
||||
Ok(map) => map,
|
||||
match guard.get_many_raw(&cache_keys) {
|
||||
Ok(rows) => rows,
|
||||
Err(error) => {
|
||||
tracing::debug!(
|
||||
target: "pacquet::download",
|
||||
?error,
|
||||
"store-index batched read failed at prefetch start; falling back to per-snapshot lookups",
|
||||
);
|
||||
return HashMap::new();
|
||||
return PrefetchResult::default();
|
||||
}
|
||||
}
|
||||
};
|
||||
// Phase 2: integrity-check each entry without holding the lock.
|
||||
let mut out = HashMap::with_capacity(entries.len());
|
||||
for (cache_key, entry) in entries {
|
||||
let verify_result = if verify_store_integrity {
|
||||
pacquet_store_dir::check_pkg_files_integrity(
|
||||
store_dir,
|
||||
entry,
|
||||
&verified_files_cache,
|
||||
)
|
||||
} else {
|
||||
pacquet_store_dir::build_file_maps_from_index(store_dir, entry)
|
||||
};
|
||||
// Phase 2: decode each row's msgpackr-records bytes into a
|
||||
// `PackageFilesIndex`, then run the integrity check. Both
|
||||
// steps are per-row CPU work with no shared state, so we
|
||||
// fan out across rayon. With manifests included in the
|
||||
// payload, decoding 1k+ rows serially had become the
|
||||
// dominant chunk of the prefetch wall (single-threaded
|
||||
// `spawn_blocking`); the par-iter recovers the per-row
|
||||
// parallelism the warm-batch link phase already uses.
|
||||
//
|
||||
// The bundled manifest is split off the decoded entry via
|
||||
// `Option::take` so it travels back to the caller without
|
||||
// an intermediate `Value::clone` of the JSON tree — the
|
||||
// verify function only inspects `files`, never `manifest`.
|
||||
let decoded: Vec<(String, Option<Arc<serde_json::Value>>, pacquet_store_dir::VerifyResult)> = raw
|
||||
.into_par_iter()
|
||||
.filter_map(|(cache_key, bytes)| {
|
||||
let mut entry: PackageFilesIndex = match pacquet_store_dir::decode_package_files_index(&bytes) {
|
||||
Ok(entry) => entry,
|
||||
Err(error) => {
|
||||
tracing::debug!(
|
||||
target: "pacquet::download",
|
||||
?cache_key,
|
||||
?error,
|
||||
"skipping undecodable package_index row at prefetch",
|
||||
);
|
||||
return None;
|
||||
}
|
||||
};
|
||||
let manifest = entry.manifest.take().map(Arc::new);
|
||||
let verify_result = if verify_store_integrity {
|
||||
pacquet_store_dir::check_pkg_files_integrity(
|
||||
store_dir,
|
||||
entry,
|
||||
&verified_files_cache,
|
||||
)
|
||||
} else {
|
||||
pacquet_store_dir::build_file_maps_from_index(store_dir, entry)
|
||||
};
|
||||
Some((cache_key, manifest, verify_result))
|
||||
})
|
||||
.collect();
|
||||
|
||||
let mut cas_paths = HashMap::with_capacity(decoded.len());
|
||||
let mut manifests = HashMap::new();
|
||||
for (cache_key, manifest, verify_result) in decoded {
|
||||
if verify_result.passed {
|
||||
out.insert(cache_key, Arc::new(verify_result.files_map));
|
||||
if let Some(manifest) = manifest {
|
||||
manifests.insert(cache_key.clone(), manifest);
|
||||
}
|
||||
cas_paths.insert(cache_key, Arc::new(verify_result.files_map));
|
||||
}
|
||||
}
|
||||
out
|
||||
PrefetchResult { cas_paths, manifests }
|
||||
})
|
||||
.await;
|
||||
result.unwrap_or_else(|error| {
|
||||
@@ -562,7 +770,7 @@ pub async fn prefetch_cas_paths(
|
||||
?error,
|
||||
"store-index prefetch task failed; falling back to per-snapshot lookups",
|
||||
);
|
||||
HashMap::new()
|
||||
PrefetchResult::default()
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -411,7 +411,7 @@ async fn prefetch_cas_paths_returns_hits_for_live_index_rows() {
|
||||
)
|
||||
.await;
|
||||
|
||||
let map = prefetched.get(&index_key).expect("hit");
|
||||
let map = prefetched.cas_paths.get(&index_key).expect("hit");
|
||||
assert_eq!(map.get("package.json"), Some(&pkg_json_path));
|
||||
drop(store_dir);
|
||||
}
|
||||
@@ -467,7 +467,7 @@ async fn prefetch_cas_paths_omits_failed_integrity_entries() {
|
||||
.await;
|
||||
|
||||
assert!(
|
||||
!prefetched.contains_key(&index_key),
|
||||
!prefetched.cas_paths.contains_key(&index_key),
|
||||
"row that fails integrity must not appear in prefetch result",
|
||||
);
|
||||
drop(store_dir);
|
||||
@@ -523,7 +523,7 @@ async fn prefetch_cas_paths_skips_filesystem_checks_when_verify_disabled() {
|
||||
)
|
||||
.await;
|
||||
|
||||
let map = prefetched.get(&index_key).expect(
|
||||
let map = prefetched.cas_paths.get(&index_key).expect(
|
||||
"verify=false should trust the index row and surface the entry without checking disk",
|
||||
);
|
||||
assert!(map.contains_key("package.json"));
|
||||
|
||||
Reference in New Issue
Block a user