mirror/pnpm
1
0
Fork 0
mirror of https://github.com/pnpm/pnpm.git synced 2026-06-29 18:35:18 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(pnpr): forward credentials and add per-user access grants for external private registries (#12184) (#12189)

Browse Source 
Closes #12184 (part 2).

#12181 shipped the per-caller access gate on `POST /v1/install`, which authorizes every served package against pnpr's own `packages:` policy — the complete answer **while pnpr fetches anonymously**. This PR adds the remaining piece: forwarding the caller's per-registry credentials so the accelerator can resolve/fetch **external private** content as the caller, and gating that content per user against the registry that actually owns it.

## Credential forwarding (issue steps 1–2)

- **Wire:** `POST /v1/install` gains an `authHeaders` body map (`{ "//host/path/": "Bearer …" }`, the shape `AuthHeaders::from_map` consumes / `getAuthHeadersFromCreds` produces) plus an HTTP `Authorization` header. The body map carries the *upstream* registry tokens; the header identifies the caller to pnpr's own gate and keys the grant table.
- **pacquet plumbing:** a request-scoped `Arc<AuthHeaders>` is threaded via a new `Install.auth_override` field and an `auth_override` param on `build_resolution_verifiers`, so resolution/verification run as the caller **without** baking per-user auth into the interned `&'static Config` (which would leak one config per user).
- **Server:** `handle_install` builds the per-request `AuthHeaders` and threads it through resolve, verify, and `fetch_uncached` (which now returns the freshly-fetched set).
- **Clients:** pacquet `pnpr-client` and `@pnpm/pnpr.client` send `registry` / `namedRegistries` / `authHeaders` + `Authorization`; the TS path sources them from the caller's registry credentials via `@pnpm/network.auth-header` (`getAuthHeadersFromCreds` is newly re-exported). `@pnpm/worker` is unchanged — downloads happen server-side.
- **Credential scope:** both clients forward the caller's *full* credential map, not a subset scoped to the declared registries. The registries a dependency graph touches aren't knowable up front — a transitive package can be scope-routed to another registry or pinned to a tarball URL on a host that's in `.npmrc` but isn't a declared registry — so pnpr attaches the right token per fetched URL exactly as a local install does. These are package-fetch credentials going to the very service the caller configured to fetch its packages.

## Per-user grant table (issue steps 3–4)

Externally-resolved private content carries no pnpr policy, so the store's possession of the bytes must not authorize a user the upstream never cleared. A served package is dispatched by **whether a forwarded credential was used to fetch it**:

- **No forwarded cred → pnpr-as-authority:** the existing local `packages:` policy check, unchanged.
- **Forwarded cred → upstream-as-authority:** gated against a persistent `(user, name@version)` grant table (SQLite, modeled on `VerdictCache`). Freshly fetched this request ⇒ record + allow (the upstream just accepted the token). Cache hit with a standing grant ⇒ allow, no upstream trip. Cache hit, no grant ⇒ re-verify against the owning registry with the caller's credential — record on success; **clear-on-discovery** (purge the user's grants for the package) + deny on `401`/`403`. TTL is the `installAccelerator.grantTtl` config knob (default: permanent).

## Public vs private (no per-user gating for public packages)

A forwarded credential matching a registry doesn't mean a package is *private* — in a mixed proxy (one registry serving a company's private packages **and** public ones), the token matches everything, and gating public content per user would cost a grant row and a re-verify round trip per user for bytes anyone may read. So before the per-user path, a not-yet-classified cache hit is probed **anonymously**: a `2xx` classifies the package public in a global set (no user pays for it again, no grant, no further round trip); a `401`/`403` means it's genuinely private and falls through to the grant / re-verify path above. Public packages thus cost **one anonymous probe across the whole fleet**, not one per user.

## Tests

- pnpr: grant-table + public-set mechanics, regime dispatch, the upstream-authorization paths (fresh-fetch, granted cache hit, private re-verify-and-record, denied-clears-grants, public-classified-once-then-free), and forwarded-cred-routes-around-local-policy.
- pacquet `pnpr-client`: a test asserting `authHeaders` + `Authorization` travel on the wire.
- Full suites green: `pnpr` (237), `pacquet-package-manager` (389), `pacquet-pnpr-client` (12), `pacquet-network`/`config` (325); clippy `-D warnings`, `cargo fmt`, rustdoc `-D warnings --document-private-items`, `typos`, and the TS compile all clean.

## Scoped follow-ups (not in this PR)

- Clear-on-discovery fires at the re-verify hook only. A `401`/`403` during the cold resolve aborts the request anyway (nothing is served); threading the offending package out of the deep resolve error to also clear stale grants for *future* requests needs structured auth errors.
- Per-scope external registries route via the default registry, since pacquet doesn't yet surface `@scope:registry` routing in `collect_packages`.
This commit is contained in:
Zoltan Kochan
2026-06-04 18:45:56 +02:00
committed by GitHub
parent 2bcd69fd8c
commit 5192edf40e
29 changed files with 1205 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
.changeset/pnpr-forward-credentials.md Normal file
View File

@@ -0,0 +1,8 @@
---
"@pnpm/installing.deps-installer": minor
"@pnpm/network.auth-header": minor
"@pnpm/pnpr.client": minor
"pnpm": minor
---
The pnpr install accelerator now forwards the caller's per-registry credentials on `POST /v1/install`, so it can resolve, verify, and fetch private dependencies from external registries as the caller. The client sends an `Authorization` header identifying itself to the pnpr server plus an `authHeaders` map of the registry tokens (built with `@pnpm/network.auth-header`), and the server threads those credentials through resolution and fetch instead of reaching the registry anonymously. Externally-resolved private content carries no pnpr access policy, so the server gates it per user against the owning registry — serving a cache hit only to a user the registry has cleared — and re-checks access (clearing it on a `401`/`403`) rather than letting the store's possession of the bytes authorize anyone. Packages the registry serves anonymously are classified public once (globally) and then served to everyone without per-user access checks, so a registry that mixes public and private packages doesn't pay the per-user cost for its public ones.

1
installing/deps-installer/package.json
View File

@@ -98,6 +98,7 @@
"@pnpm/lockfile.utils": "workspace:*",
"@pnpm/lockfile.verification": "workspace:*",
"@pnpm/lockfile.walker": "workspace:*",
"@pnpm/network.auth-header": "workspace:*",
"@pnpm/npm-package-arg": "catalog:",
"@pnpm/patching.config": "workspace:*",
"@pnpm/pkg-manifest.utils": "workspace:*",

12
installing/deps-installer/src/install/index.ts
View File

@@ -2346,8 +2346,16 @@ async function installFromPnpmRegistry (
)
}
const { fetchFromPnpmRegistry } = await import('@pnpm/pnpr.client')
const { createGetAuthHeaderByURI, getAuthHeadersFromCreds } = await import('@pnpm/network.auth-header')
const { StoreIndex } = await import('@pnpm/store.index')
const { setImportConcurrency } = await import('@pnpm/worker')
// Forward the whole credential map (the registries a graph touches
// aren't known up front), so the server attaches the right token per
// URL. `authorization` also identifies the caller to pnpr's gate.
const configByUri = opts.configByUri ?? {}
const forwardedAuthHeaders = getAuthHeadersFromCreds(configByUri)
const pnprAuthorization = createGetAuthHeaderByURI(configByUri)(opts.pnprServer!)
// Raise import concurrency for this install only — the pnpr server path has no
// concurrent fetching competing for workers. Restore afterwards so we
// don't leak a process-wide mutation to other installs (e.g. tests).
@@ -2392,6 +2400,10 @@ async function installFromPnpmRegistry (
devDependencies: projectsList ? undefined : manifest.devDependencies,
optionalDependencies: projectsList ? undefined : manifest.optionalDependencies,
projects: projectsList,
registry: opts.registries?.default,
namedRegistries: opts.namedRegistries,
authHeaders: forwardedAuthHeaders,
authorization: pnprAuthorization,
overrides: opts.overrides,
minimumReleaseAge: opts.minimumReleaseAge,
lockfile: existingLockfile ?? undefined,

3
installing/deps-installer/tsconfig.json
View File

@@ -132,6 +132,9 @@
{
"path": "../../lockfile/walker"
},
{
"path": "../../network/auth-header"
},
{
"path": "../../network/git-utils"
},

5
network/auth-header/src/index.ts
View File

@@ -4,6 +4,11 @@ import type { RegistryConfig } from '@pnpm/types'
import { getAuthHeadersFromCreds } from './getAuthHeadersFromConfig.js'
import { removePort } from './helpers/removePort.js'
// Re-exported so callers that need the whole nerf-darted-URI → header map
// (e.g. forwarding every registry credential to the pnpr install
// accelerator) can build it without re-implementing `credsToHeader`.
export { getAuthHeadersFromCreds }
export function createGetAuthHeaderByURI (
configByUri: Record<string, RegistryConfig>
): (uri: string) => string | undefined {

12
pacquet/crates/cli/src/cli_args/install.rs
View File

@@ -358,6 +358,7 @@ impl InstallArgs {
node_linker,
lockfile_only,
update_seed_policy: UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<Reporter>()
.await
@@ -468,6 +469,16 @@ async fn install_via_pnpr<Reporter: self::Reporter + 'static>(
dev_dependencies,
registry: state.config.registry.clone(),
named_registries: state.config.named_registries.clone(),
// Forward the whole credential map: the registries a graph
// touches aren't known up front (scope-routed or tarball-URL
// sub-deps), so the server attaches the right token per URL.
auth_headers: state
.config
.auth_headers
.entries()
.map(|(uri, value)| (uri.to_string(), value.to_string()))
.collect(),
authorization: state.config.auth_headers.for_url(pnpr_server),
overrides,
lockfile: state.lockfile.clone(),
frozen_lockfile: link.frozen_lockfile,
@@ -544,6 +555,7 @@ async fn install_via_pnpr<Reporter: self::Reporter + 'static>(
node_linker: link.node_linker,
lockfile_only: false,
update_seed_policy: UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<Reporter>()
.await

7
pacquet/crates/network/src/auth.rs
View File

@@ -87,6 +87,13 @@ impl AuthHeaders {
AuthHeaders { by_uri, max_parts }
}
/// The `(nerf_darted_uri, header_value)` pairs backing this lookup, so
/// a caller can forward the whole set to another process (the pnpr
/// accelerator) and rebuild it with [`Self::from_map`].
pub fn entries(&self) -> impl Iterator<Item = (&str, &str)> {
self.by_uri.iter().map(|(uri, value)| (uri.as_str(), value.as_str()))
}
/// Resolve an `Authorization` header for `url`, mirroring pnpm's
/// `getAuthHeaderByURI`:
///

1
pacquet/crates/package-manager/src/add.rs
View File

@@ -124,6 +124,7 @@ where
// is the only thing that re-resolves. `update`'s bump is a
// separate operation.
update_seed_policy: UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<Reporter>()
.await

5
pacquet/crates/package-manager/src/build_resolution_verifiers.rs
View File

@@ -23,7 +23,7 @@ use pacquet_config::{
Config, TrustPolicy,
version_policy::{PackageVersionPolicy, VersionPolicyError, create_package_version_policy},
};
use pacquet_network::ThrottledClient;
use pacquet_network::{AuthHeaders, ThrottledClient};
use pacquet_resolving_npm_resolver::{
CreateNpmResolutionVerifierOptions, PackageMetaCache, create_npm_resolution_verifier,
};
@@ -71,6 +71,7 @@ pub fn build_resolution_verifiers(
config: &Config,
http_client: Arc<ThrottledClient>,
meta_cache: Option<Arc<dyn PackageMetaCache>>,
auth_override: Option<Arc<AuthHeaders>>,
) -> Result<Vec<Arc<dyn ResolutionVerifier>>, BuildVerifiersError> {
let mut verifiers: Vec<Arc<dyn ResolutionVerifier>> = Vec::new();
@@ -119,7 +120,7 @@ pub fn build_resolution_verifiers(
.map(|(name, url)| (name.clone(), url.clone()))
.collect(),
http_client,
auth_headers: Arc::clone(&config.auth_headers),
auth_headers: auth_override.unwrap_or_else(|| Arc::clone(&config.auth_headers)),
cache_dir: Some(config.cache_dir.clone()),
meta_cache,
retry_opts: retry_opts_from_config(config),

11
pacquet/crates/package-manager/src/install.rs
View File

@@ -25,7 +25,7 @@ use pacquet_modules_yaml::{
Host, IncludedDependencies, LayoutVersion, Modules, NodeLinker as ModulesNodeLinker,
WriteModulesError, read_modules_manifest, write_modules_manifest,
};
use pacquet_network::ThrottledClient;
use pacquet_network::{AuthHeaders, ThrottledClient};
use pacquet_package_manifest::{DependencyGroup, PackageManifest};
use pacquet_reporter::{
ContextLog, LogEvent, LogLevel, PackageManifestLog, PackageManifestMessage, PnpmLog, Reporter,
@@ -177,6 +177,12 @@ where
/// short-circuit is also bypassed so an `update` that finds newer
/// in-range versions isn't skipped as "already up to date".
pub update_seed_policy: UpdateSeedPolicy,
/// Per-invocation `Authorization`-header override for resolve/verify;
/// `None` (every local install) uses `config.auth_headers`. The pnpr
/// accelerator threads request-scoped [`AuthHeaders`] here so it
/// resolves a caller's private content without baking per-user auth
/// into the shared `&'static Config`.
pub auth_override: Option<Arc<AuthHeaders>>,
}
/// Error type of [`Install`].
@@ -365,6 +371,7 @@ where
node_linker,
lockfile_only,
update_seed_policy,
auth_override,
} = self;
// `--lockfile-only` with `lockfile: false` (pnpm's
@@ -625,6 +632,7 @@ where
Arc::clone(&http_client_arc),
Some(Arc::clone(&meta_cache)
as Arc<dyn pacquet_resolving_npm_resolver::PackageMetaCache>),
auth_override.clone(),
)
.map_err(InstallError::BuildVerifiers)?;
verify_lockfile_resolutions::<Reporter>(
@@ -961,6 +969,7 @@ where
supported_architectures: supported_architectures.as_ref(),
lockfile_only,
update_seed_policy,
auth_override,
}
.run::<Reporter>()
.await

62
pacquet/crates/package-manager/src/install/tests.rs
View File

@@ -72,6 +72,7 @@ async fn should_install_dependencies() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -144,6 +145,7 @@ async fn should_error_when_frozen_lockfile_is_requested_but_none_exists() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -191,6 +193,7 @@ async fn should_error_when_frozen_lockfile_and_update_checksums_are_both_set() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -267,6 +270,7 @@ async fn frozen_lockfile_flag_overrides_config_lockfile_false() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -336,6 +340,7 @@ async fn npm_alias_dependency_installs_under_alias_key() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -424,6 +429,7 @@ async fn unversioned_npm_alias_defaults_to_latest() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -495,6 +501,7 @@ async fn frozen_lockfile_flag_with_no_lockfile_errors() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -586,6 +593,7 @@ async fn install_emits_pnpm_event_sequence() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -734,6 +742,7 @@ async fn install_writes_modules_yaml() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -838,6 +847,7 @@ async fn install_writes_workspace_state() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -1065,6 +1075,7 @@ async fn install_optional_failing_postinstall_dep_via_registry_mock_succeeds() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -1141,6 +1152,7 @@ async fn auto_install_peers_does_not_cascade_optional_peers() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -1240,6 +1252,7 @@ async fn auto_install_peers_skips_meta_only_optional_peers() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -1375,6 +1388,7 @@ async fn warm_reinstall_skips_snapshot_when_current_lockfile_matches() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -1476,6 +1490,7 @@ async fn warm_reinstall_emits_broken_modules_when_dir_is_missing() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await;
@@ -1585,6 +1600,7 @@ async fn context_log_reflects_current_lockfile_after_first_install() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -1638,6 +1654,7 @@ async fn context_log_reflects_current_lockfile_after_first_install() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -1733,6 +1750,7 @@ async fn warm_reinstall_reports_added_zero_and_emits_no_imported_events() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -1838,6 +1856,7 @@ async fn frozen_lockfile_errors_when_manifest_drifts_from_lockfile() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -1905,6 +1924,7 @@ async fn ignore_manifest_check_bypasses_manifest_freshness_gate() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -1973,6 +1993,7 @@ async fn frozen_lockfile_errors_when_overrides_drift_from_lockfile() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -2067,6 +2088,7 @@ async fn frozen_lockfile_applies_overrides_to_manifest_before_freshness_check()
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -2177,6 +2199,7 @@ async fn frozen_lockfile_resolves_catalog_protocol_in_overrides_before_freshness
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -2241,6 +2264,7 @@ async fn frozen_lockfile_errors_when_lockfile_has_no_root_importer() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -2332,6 +2356,7 @@ async fn frozen_lockfile_under_gvs_registers_project_and_runs_clean() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -2442,6 +2467,7 @@ async fn gvs_persists_global_virtual_store_dir_in_modules_yaml_and_context_log()
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -2559,6 +2585,7 @@ async fn frozen_lockfile_with_gvs_off_skips_project_registry() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -2642,6 +2669,7 @@ async fn frozen_lockfile_under_gvs_registers_workspace_root_only() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -2845,6 +2873,7 @@ async fn frozen_install_preserves_seeded_skipped_across_reinstall() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -2972,6 +3001,7 @@ async fn frozen_install_silently_swallows_unreachable_optional_tarball() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -3075,6 +3105,7 @@ async fn frozen_install_propagates_non_optional_fetch_failure() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -3184,6 +3215,7 @@ async fn frozen_install_no_optional_drops_optional_only_snapshots() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -3278,6 +3310,7 @@ async fn frozen_install_optional_included_surfaces_missing_metadata() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -3375,6 +3408,7 @@ async fn frozen_install_no_optional_keeps_shared_non_optional_snapshot() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -3471,6 +3505,7 @@ async fn hoisted_node_linker_empty_lockfile_writes_modules_yaml() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -3562,6 +3597,7 @@ async fn hoisted_node_linker_does_not_create_virtual_store_root() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -3661,6 +3697,7 @@ async fn frozen_lockfile_install_errors_when_no_variant_matches_host() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -3758,6 +3795,7 @@ async fn frozen_lockfile_install_skips_runtime_when_skip_runtimes_set() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -3859,6 +3897,7 @@ async fn install_rejects_invalid_minimum_release_age_exclude_pattern() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -3962,6 +4001,7 @@ async fn frozen_lockfile_gate_rejects_under_huge_minimum_release_age() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -4051,6 +4091,7 @@ async fn fresh_install_writes_pnpm_lock_yaml_with_expected_shape() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4138,6 +4179,7 @@ async fn fresh_install_splits_dev_and_prod_dependency_sections() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4211,6 +4253,7 @@ async fn fresh_install_records_user_written_specifier() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4280,6 +4323,7 @@ async fn fresh_install_lockfile_round_trips_through_load_save_load() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4348,6 +4392,7 @@ async fn fresh_install_with_lockfile_disabled_does_not_write_a_lockfile() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4419,6 +4464,7 @@ async fn fresh_install_also_writes_current_lockfile_under_virtual_store() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4506,6 +4552,7 @@ async fn fresh_install_with_lockfile_disabled_skips_current_lockfile_too() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4571,6 +4618,7 @@ async fn fresh_install_marks_optional_snapshots_in_pnpm_lock_yaml() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4661,6 +4709,7 @@ async fn fresh_install_hoisted_node_linker_records_modules_yaml() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4731,6 +4780,7 @@ async fn fresh_install_refuses_skip_runtimes_before_writing_state() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -4805,6 +4855,7 @@ async fn prefer_frozen_lockfile_takes_frozen_path_when_lockfile_is_fresh() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -4880,6 +4931,7 @@ async fn no_prefer_frozen_lockfile_flag_forces_fresh_resolve() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -4949,6 +5001,7 @@ async fn stale_lockfile_under_no_flag_falls_through_to_fresh_resolve() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -5204,6 +5257,7 @@ async fn frozen_install_short_circuits_when_modules_and_lockfile_are_consistent(
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -5388,6 +5442,7 @@ async fn optimistic_repeat_install_skips_entire_pipeline_when_state_is_fresh() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -5539,6 +5594,7 @@ async fn frozen_lockfile_disables_optimistic_short_circuit() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -5691,6 +5747,7 @@ async fn optimistic_repeat_install_does_not_short_circuit_when_lockfile_missing(
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await;
@@ -5772,6 +5829,7 @@ async fn optimistic_repeat_install_round_trips_on_single_project_install() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -5826,6 +5884,7 @@ async fn optimistic_repeat_install_round_trips_on_single_project_install() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<RecordingReporter>()
.await
@@ -5936,6 +5995,7 @@ async fn fresh_install_applies_package_extensions_to_dependency_manifest() {
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await
@@ -6034,6 +6094,7 @@ async fn frozen_lockfile_errors_when_package_extensions_drift_from_lockfile() {
node_linker: pacquet_config::NodeLinker::default(),
lockfile_only: false,
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<SilentReporter>()
.await;
@@ -6114,6 +6175,7 @@ async fn install_with_pnpmfile_reporter<Reporter: self::Reporter + 'static>(
lockfile_only: false,
resolved_packages: &Default::default(),
update_seed_policy: crate::UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<Reporter>()
.await

16
pacquet/crates/package-manager/src/install_with_fresh_lockfile.rs
View File

@@ -17,7 +17,7 @@ use pacquet_engine_runtime_deno_resolver::DenoResolver;
use pacquet_engine_runtime_node_resolver::NodeResolver;
use pacquet_hooks::finder;
use pacquet_lockfile::{Lockfile, LockfileResolution, SaveLockfileError};
use pacquet_network::ThrottledClient;
use pacquet_network::{AuthHeaders, ThrottledClient};
use pacquet_package_manifest::{DependencyGroup, PackageManifest};
use pacquet_reporter::{HookLog, LogEvent, LogLevel, Reporter, Stage, StageLog};
use pacquet_resolving_default_resolver::DefaultResolver;
@@ -170,6 +170,9 @@ pub struct InstallWithFreshLockfile<'a, DependencyGroupList> {
/// satisfying their manifest range. Drives `pacquet update`'s
/// compatible bump; see [`UpdateSeedPolicy`].
pub update_seed_policy: UpdateSeedPolicy,
/// Per-invocation `Authorization`-header override; `None` uses
/// `config.auth_headers`. See [`crate::Install::auth_override`].
pub auth_override: Option<Arc<AuthHeaders>>,
}
/// Which lockfile-pinned `(name, version)` pairs to *withhold* from the
@@ -426,7 +429,12 @@ impl<'a, DependencyGroupList> InstallWithFreshLockfile<'a, DependencyGroupList>
supported_architectures,
lockfile_only,
update_seed_policy,
auth_override,
} = self;
// The pnpr override when supplied, else the config's npmrc headers;
// shared by every registry-touching resolver below.
let auth_headers = auth_override.unwrap_or_else(|| Arc::clone(&config.auth_headers));
let is_hoisted = matches!(node_linker, NodeLinker::Hoisted);
// Materialise the caller's iterator into a `Vec` so the same
// group set can be replayed into both the resolver (consumes
@@ -532,7 +540,7 @@ impl<'a, DependencyGroupList> InstallWithFreshLockfile<'a, DependencyGroupList>
registries,
named_registries: merged_named_registries.clone(),
http_client: Arc::clone(&http_client_arc),
auth_headers: Arc::clone(&config.auth_headers),
auth_headers: Arc::clone(&auth_headers),
meta_cache: Arc::clone(&meta_cache),
fetch_locker: Arc::clone(&fetch_locker),
picked_manifest_cache: Arc::clone(&picked_manifest_cache),
@@ -593,7 +601,7 @@ impl<'a, DependencyGroupList> InstallWithFreshLockfile<'a, DependencyGroupList>
store_dir,
store_index_writer: Some(Arc::clone(&store_index_writer)),
mem_cache: Some(Arc::clone(&tarball_mem_cache)),
auth_headers: Arc::clone(&config.auth_headers),
auth_headers: Arc::clone(&auth_headers),
retry_opts: crate::retry_config::retry_opts_from_config(config),
store_index: store_index.clone(),
verify_store_integrity: config.verify_store_integrity,
@@ -620,7 +628,7 @@ impl<'a, DependencyGroupList> InstallWithFreshLockfile<'a, DependencyGroupList>
named_registries: merged_named_registries,
registry_names: named_registry_aliases,
http_client: Arc::clone(&http_client_arc),
auth_headers: Arc::clone(&config.auth_headers),
auth_headers: Arc::clone(&auth_headers),
meta_cache: Arc::clone(&meta_cache),
fetch_locker: Arc::clone(&fetch_locker),
picked_manifest_cache: Arc::clone(&picked_manifest_cache),

1
pacquet/crates/package-manager/src/remove.rs
View File

@@ -139,6 +139,7 @@ impl<'a> Remove<'a> {
// every remaining lockfile pin in the preferred-versions
// seed, same as `install` / `add`.
update_seed_policy: UpdateSeedPolicy::KeepAll,
auth_override: None,
}
.run::<Reporter>()
.await

1
pacquet/crates/package-manager/src/update.rs
View File

@@ -392,6 +392,7 @@ impl Update<'_> {
node_linker: config.node_linker,
lockfile_only,
update_seed_policy: seed_policy,
auth_override: None,
}
.run::<Reporter>()
.await

2
pacquet/crates/pnpr-client/Cargo.toml
View File

@@ -28,6 +28,8 @@ tracing = { workspace = true }
pacquet-testing-utils = { workspace = true }
mockito = { workspace = true }
pnpr = { workspace = true }
reqwest = { workspace = true }
serde_json = { workspace = true }
tempfile = { workspace = true }
tokio = { workspace = true }

16
pacquet/crates/pnpr-client/src/lib.rs
View File

@@ -55,6 +55,14 @@ pub struct InstallOptions<'a> {
pub registry: String,
/// The client's named-registry aliases.
pub named_registries: DepMap,
/// The caller's forwarded upstream credentials, keyed by nerf-darted
/// registry URI, so the server resolves/fetches private content as the
/// caller. Distinct from [`Self::authorization`] (pnpr identity).
pub auth_headers: DepMap,
/// `Authorization` for the pnpr server's own URL (`None` if it needs
/// none): identifies the caller to pnpr's gate and keys the grant
/// table. Distinct from the upstream creds in [`Self::auth_headers`].
pub authorization: Option<String>,
/// The client's `overrides` (selector -> spec) as raw JSON, applied
/// at resolve time server-side.
pub overrides: Option<serde_json::Value>,
@@ -213,6 +221,7 @@ impl PnprClient {
"storeIntegrities": store_integrities,
"registry": opts.registry,
"namedRegistries": opts.named_registries,
"authHeaders": opts.auth_headers,
"overrides": opts.overrides,
"lockfile": opts.lockfile,
"frozenLockfile": opts.frozen_lockfile,
@@ -229,8 +238,11 @@ impl PnprClient {
"inlineFiles": true,
});
let response =
self.http.post(format!("{}v1/install", self.base_url)).json(&request).send().await?;
let mut post = self.http.post(format!("{}v1/install", self.base_url)).json(&request);
if let Some(authorization) = opts.authorization.as_deref() {
post = post.header("authorization", authorization);
}
let response = post.send().await?;
if !response.status().is_success() {
let status = response.status();
let body = response.text().await.unwrap_or_default();

112
pacquet/crates/pnpr-client/tests/integration.rs
View File

@@ -50,6 +50,31 @@ fn deps<const COUNT: usize>(entries: [(&str, &str); COUNT]) -> BTreeMap<String,
entries.into_iter().map(|(name, range)| (name.to_string(), range.to_string())).collect()
}
/// The nerf-darted key (`//host[:port]/path/`) a forwarded credential for
/// `url` is keyed by, mirroring `AuthHeaders`' lookup on the server —
/// keeping any registry path prefix so the key isn't wrong for one.
fn nerf_key(url: &str) -> String {
let authority_and_path = url.split("://").nth(1).unwrap_or(url);
let (authority, path) = authority_and_path.split_once('/').unwrap_or((authority_and_path, ""));
let path = path.split(['?', '#']).next().unwrap_or("").trim_matches('/');
if path.is_empty() { format!("//{authority}/") } else { format!("//{authority}/{path}/") }
}
/// Register a user with the shared test registry and return its bearer
/// token, so a test can forward it as the caller's upstream credential.
async fn register_token(registry_url: &str, username: &str) -> String {
let body = serde_json::json!({ "name": username, "password": "password123" });
let response = reqwest::Client::new()
.put(format!("{registry_url}-/user/org.couchdb.user:{username}"))
.json(&body)
.send()
.await
.expect("adduser request");
assert!(response.status().is_success(), "adduser returned {}", response.status());
let json: serde_json::Value = response.json().await.expect("adduser response json");
json["token"].as_str().expect("token in adduser response").to_string()
}
fn options<'a>(
store: &'a StoreDir,
registry: &str,
@@ -61,6 +86,8 @@ fn options<'a>(
dev_dependencies: BTreeMap::new(),
registry: registry.to_string(),
named_registries: BTreeMap::new(),
auth_headers: BTreeMap::new(),
authorization: None,
overrides: None,
lockfile: None,
frozen_lockfile: false,
@@ -77,6 +104,91 @@ fn options<'a>(
}
}
/// The forwarded per-registry credentials and the pnpr-server identity
/// header must travel on the wire: `authHeaders` in the body (so the
/// server resolves/fetches private content as the caller) and
/// `Authorization` on the request (so pnpr's gate + grant table key on
/// the right user). A `mockito` server captures the request and asserts
/// both are present; the canned 500 just short-circuits the client after
/// the match.
#[tokio::test]
async fn forwards_credentials_and_the_identity_header() {
let mut server = mockito::Server::new_async().await;
let mock = server
.mock("POST", "/v1/install")
.match_header("authorization", "Bearer pnpr-token")
.match_body(mockito::Matcher::PartialJsonString(
r#"{"authHeaders":{"//npm.acme.test/":"Bearer upstream-token"}}"#.to_string(),
))
.with_status(500)
.with_body("stop")
.create_async()
.await;
let client_store = TempDir::new().unwrap();
let store = StoreDir::new(client_store.path().to_path_buf());
let client = PnprClient::new(format!("{}/", server.url()));
let mut opts = options(&store, "https://npm.acme.test/", deps([("@acme/foo", "1.0.0")]));
opts.auth_headers = deps([("//npm.acme.test/", "Bearer upstream-token")]);
opts.authorization = Some("Bearer pnpr-token".to_string());
let result = client.install(opts).await;
assert!(result.is_err(), "the canned 500 should surface as an error");
mock.assert_async().await;
}
/// End-to-end: the test registry gates `@pnpm.e2e/needs-auth` behind
/// `$authenticated`, so resolving it through the accelerator only works
/// when the caller's upstream token is forwarded and the server fetches
/// the packument + tarball as the caller.
#[tokio::test]
async fn a_forwarded_credential_resolves_a_private_package() {
let registry = TestRegistry::start();
let token = register_token(&registry.url(), "needs-auth-forwarder").await;
let (pnpr_url, _storage) = start_pnpr().await;
let client_store = TempDir::new().unwrap();
let store = StoreDir::new(client_store.path().to_path_buf());
let client = PnprClient::new(pnpr_url);
let mut opts = options(&store, &registry.url(), deps([("@pnpm.e2e/needs-auth", "1.0.0")]));
let mut auth = BTreeMap::new();
auth.insert(nerf_key(&registry.url()), format!("Bearer {token}"));
opts.auth_headers = auth;
let outcome = client.install(opts).await.expect("forwarded credential should resolve it");
let packages = outcome.lockfile.packages.as_ref().expect("lockfile has packages");
assert!(
packages.keys().any(|key| key.to_string().starts_with("@pnpm.e2e/needs-auth@1.0.0")),
"lockfile should contain the authed package, got: {:?}",
packages.keys().map(ToString::to_string).collect::<Vec<_>>(),
);
assert!(outcome.files_written >= 1, "its files should be materialized");
}
/// The same install without a forwarded credential fails: the registry
/// won't serve the gated packument anonymously, so resolution can't read
/// it.
#[tokio::test]
async fn a_private_package_fails_without_a_forwarded_credential() {
let registry = TestRegistry::start();
let (pnpr_url, _storage) = start_pnpr().await;
let client_store = TempDir::new().unwrap();
let store = StoreDir::new(client_store.path().to_path_buf());
let client = PnprClient::new(pnpr_url);
let opts = options(&store, &registry.url(), deps([("@pnpm.e2e/needs-auth", "1.0.0")]));
let Err(PnprClientError::Server(message)) = client.install(opts).await else {
panic!("expected the gated install to fail with a server error");
};
assert!(
message.contains("401"),
"expected an auth denial without a forwarded credential, got: {message}",
);
}
#[tokio::test]
async fn resolves_and_downloads_a_package() {
let registry = TestRegistry::start();

1
pacquet/crates/resolving-npm-resolver/src/lib.rs
View File

@@ -69,6 +69,7 @@ pub use pick_package_from_meta::{
RegistryPackageSpec, RegistryPackageSpecType, filter_pkg_metadata_by_publish_date,
pick_lowest_version_by_version_range, pick_package_from_meta, pick_version_by_version_range,
};
pub use registry_url::to_registry_url;
pub use resolve_from_workspace::{
ResolveFromWorkspaceError, ResolveFromWorkspaceOptions, try_resolve_from_workspace,
};

3
pnpm-lock.yaml generated
View File

@@ -5673,6 +5673,9 @@ importers:
'@pnpm/lockfile.walker':
specifier: workspace:*
version: link:../../lockfile/walker
'@pnpm/network.auth-header':
specifier: workspace:*
version: link:../../network/auth-header
'@pnpm/npm-package-arg':
specifier: 'catalog:'
version: 2.0.0

42
pnpr/client/src/fetchFromPnpmRegistry.ts
View File

@@ -33,6 +33,24 @@ export interface FetchFromPnpmRegistryOptions {
optionalDependencies?: Record<string, string>
/** Multiple projects in a workspace */
projects?: PnprProject[]
/**
* The client's default registry. The server resolves against this
* (and `namedRegistries`) rather than its own configuration.
*/
registry?: string
/** The client's named-registry aliases (`namedRegistries`). */
namedRegistries?: Record<string, string>
/**
* The caller's forwarded upstream credentials, keyed by nerf-darted
* registry URI, so the server resolves/fetches private content as the
* caller. Distinct from `authorization` (pnpr identity).
*/
authHeaders?: Record<string, string>
/**
* `Authorization` for the pnpr server's own URL (`undefined` if none):
* identifies the caller to pnpr's gate and keys the grant table.
*/
authorization?: string
/** Overrides */
overrides?: Record<string, string>
/** Node.js version for resolution */
@@ -94,6 +112,9 @@ export async function fetchFromPnpmRegistry (
const requestBody = JSON.stringify({
projects,
registry: opts.registry,
namedRegistries: opts.namedRegistries,
authHeaders: opts.authHeaders,
overrides: opts.overrides,
nodeVersion: opts.nodeVersion ?? process.version.slice(1),
os: process.platform,
@@ -108,7 +129,7 @@ export async function fetchFromPnpmRegistry (
inlineFiles: true,
})
const body = await postInstall(opts.registryUrl, requestBody)
const body = await postInstall(opts.registryUrl, requestBody, opts.authorization)
// The combined response is `[u32 header length][header JSON][file frames]`.
if (body.length < 4) {
@@ -179,20 +200,27 @@ const REQUEST_TIMEOUT = 600_000 // 10 minutes — server-side resolution can be
* prefix configured on the pnpr server URL (e.g. https://host/pnpr/) is
* preserved.
*/
async function postInstall (registryUrl: string, body: string): Promise<Buffer> {
async function postInstall (registryUrl: string, body: string, authorization?: string): Promise<Buffer> {
const base = registryUrl.endsWith('/') ? registryUrl : `${registryUrl}/`
const url = new URL('v1/install', base)
const requestFn = url.protocol === 'https:' ? https.request : http.request
const headers: http.OutgoingHttpHeaders = {
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(body),
'Accept-Encoding': 'gzip',
}
// Identify the caller to the pnpr server's access gate so protected
// packages resolve and the per-user grant table keys on the right user.
if (authorization != null) {
headers.Authorization = authorization
}
return new Promise<Buffer>((resolve, reject) => {
const req = requestFn(url, {
method: 'POST',
timeout: REQUEST_TIMEOUT,
headers: {
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(body),
'Accept-Encoding': 'gzip',
},
headers,
}, (res) => {
const chunks: Buffer[] = []
res.on('data', (chunk: Buffer) => chunks.push(chunk))

24
pnpr/crates/pnpr/src/config.rs
View File

@@ -85,6 +85,11 @@ pub struct Config {
/// installs at startup. Sourced from the YAML `log:` object
/// (Verdaccio 6+ shape). Defaults to pretty/info.
pub logs: LogConfig,
/// How long the install accelerator keeps a per-user access grant
/// before re-verifying. `None` (default) is permanent (revocation
/// relies on clear-on-discovery). YAML `installAccelerator.grantTtl`
/// (seconds).
pub install_accelerator_grant_ttl: Option<Duration>,
}
/// Auth-related runtime configuration. Built from the YAML
@@ -290,6 +295,19 @@ struct ConfigFile {
/// intentionally not accepted.
#[serde(default)]
log: Option<LogEntryFile>,
/// pnpr-only block tuning the install accelerator. Absent on a
/// stock verdaccio config (silently ignored there, like the other
/// keys verdaccio doesn't share).
#[serde(default, rename = "installAccelerator")]
install_accelerator: Option<InstallAcceleratorFile>,
}
/// The YAML `installAccelerator:` block.
#[derive(Debug, Default, Deserialize)]
struct InstallAcceleratorFile {
/// `grantTtl` in seconds. Absent ⇒ permanent grants.
#[serde(default, rename = "grantTtl")]
grant_ttl: Option<u64>,
}
/// The YAML `log:` object. Mirrors verdaccio 6's logger config.
@@ -362,6 +380,7 @@ impl Config {
policies: PackagePolicies::registry_mock_defaults(),
auth: AuthConfig::default(),
logs: LogConfig::default(),
install_accelerator_grant_ttl: None,
}
}
@@ -378,6 +397,7 @@ impl Config {
policies: PackagePolicies::registry_mock_defaults(),
auth: AuthConfig::default(),
logs: LogConfig::default(),
install_accelerator_grant_ttl: None,
}
}
@@ -497,6 +517,10 @@ impl Config {
policies,
auth,
logs,
install_accelerator_grant_ttl: file
.install_accelerator
.and_then(|block| block.grant_ttl)
.map(Duration::from_secs),
})
}

276
pnpr/crates/pnpr/src/install_accelerator.rs
View File

@@ -16,10 +16,11 @@
//! followed by the binary file frames). One round trip
//! ([pnpm/pnpm#12165](https://github.com/pnpm/pnpm/issues/12165)).
//!
//! Files are bound to access: every package whose bytes are served is
//! checked against pnpr's `packages:` policy first
//! ([`deny_unauthorized_packages`]), so a content-addressed digest is
//! never a bearer capability for a package the caller can't read.
//! Files are bound to access ([`authorize_served_packages`]): a
//! content-addressed digest is never a bearer capability. Anonymous
//! content is checked against pnpr's own `packages:` policy; content
//! fetched with the caller's forwarded credentials is gated per user
//! against the owning registry.
//!
//! The client's `registry`, `namedRegistries`, `overrides`, and the
//! verification policy (`minimumReleaseAge`, `trustPolicy`, ...) drive
@@ -29,12 +30,15 @@
//! non-frozen → reuse-and-update). A multi-project workspace is resolved
//! by reconstructing the workspace on disk (root manifest +
//! `pnpm-workspace.yaml` + member manifests) and letting pacquet's
//! install path discover and resolve every importer. **Deferred:**
//! auth/credential forwarding (so private registries resolve). Responses
//! are buffered rather than truly streamed.
//! install path discover and resolve every importer. The client also
//! forwards its per-registry credentials, so private dependencies resolve
//! and fetch as the caller. Responses are buffered rather than truly
//! streamed.
mod diff;
mod grant_table;
mod protocol;
mod public_packages;
mod resolve;
mod verdict_cache;
@@ -46,6 +50,7 @@ use std::{
io::Write as _,
path::PathBuf,
sync::{Arc, Mutex, OnceLock},
time::Duration,
};
use crate::{
@@ -64,13 +69,16 @@ use indexmap::IndexMap;
use pacquet_config::Config as PacquetConfig;
use pacquet_lockfile::Lockfile;
use pacquet_lockfile_verification::{collect_resolution_policy_violations, hash_lockfile};
use pacquet_network::ThrottledClient;
use pacquet_network::{AuthHeaders, ThrottledClient};
use pacquet_package_manager::build_resolution_verifiers;
use pacquet_resolving_npm_resolver::{InMemoryPackageMetaCache, PackageMetaCache};
use pacquet_resolving_npm_resolver::{InMemoryPackageMetaCache, PackageMetaCache, to_registry_url};
use pacquet_resolving_resolver_base::ResolutionVerifier;
use pacquet_store_dir::{StoreDir, StoreIndex};
use self::{protocol::InstallRequest, verdict_cache::VerdictCache};
use self::{
grant_table::GrantTable, protocol::InstallRequest, public_packages::PublicPackages,
verdict_cache::VerdictCache,
};
/// Per-server engine backing the pnpr install endpoints: it holds the
/// store, cache, and HTTP client used to resolve a client's project and
@@ -96,6 +104,18 @@ pub(crate) struct InstallAccelerator {
/// only if the database couldn't be opened — verification then runs
/// every time (uncached) rather than failing the server.
verdict_cache: Option<VerdictCache>,
/// Per-`(user, name@version)` access grants for externally-resolved
/// private content. `None` if the DB couldn't be opened (every such
/// package then re-verifies uncached). See [`GrantTable`].
grant_table: Option<GrantTable>,
/// Global set of anonymously-readable package names, so a public
/// package isn't gated per user. `None` if the DB couldn't be opened.
/// See [`PublicPackages`].
public_packages: Option<PublicPackages>,
/// How long a grant (or public classification) stays valid. `None`
/// (the default) is permanent, leaving revocation to
/// clear-on-discovery; a TTL lets it bite already-seen versions.
grant_ttl: Option<Duration>,
}
impl InstallAccelerator {
@@ -115,12 +135,17 @@ impl InstallAccelerator {
let _ = std::fs::create_dir_all(&store_dir);
let _ = std::fs::create_dir_all(&cache_dir);
let verdict_cache = VerdictCache::open(&cache_dir.join("lockfile-verdicts.sqlite")).ok();
let grant_table = GrantTable::open(&cache_dir.join("install-grants.sqlite")).ok();
let public_packages = PublicPackages::open(&cache_dir.join("public-packages.sqlite")).ok();
InstallAccelerator {
store_dir: StoreDir::new(store_dir),
cache_dir,
client: Arc::new(ThrottledClient::new_for_installs()),
configs: Mutex::new(HashMap::new()),
verdict_cache,
grant_table,
public_packages,
grant_ttl: config.install_accelerator_grant_ttl,
}
}
@@ -182,8 +207,8 @@ impl InstallAccelerator {
/// Handle `POST /v1/install`. `identity` is the resolved caller; the
/// store's possession of a package's bytes is not a capability to read
/// them, so every served package is checked against `policies` — see
/// [`deny_unauthorized_packages`].
/// them, so every served package is authorized first — see
/// [`authorize_served_packages`].
pub(crate) async fn handle_install(
runtime: &InstallAccelerator,
policies: &PackagePolicies,
@@ -198,6 +223,13 @@ pub(crate) async fn handle_install(
// Resolve against the client's registries, not the server's own.
let config = runtime.config_for(&request);
// The caller's forwarded upstream credentials, threaded through
// resolve/verify/fetch but kept out of the interned `config` so it
// never leaks a `&'static Config` per user.
let request_auth = Arc::new(AuthHeaders::from_map(
request.auth_headers.iter().map(|(uri, value)| (uri.clone(), value.clone())).collect(),
));
// Verify the *input* lockfile under the client's policy before
// resolving ([pnpm/pnpm#12139](https://github.com/pnpm/pnpm/issues/12139)).
// The client skips its own `verifyLockfileResolutions` whenever a
@@ -209,7 +241,8 @@ pub(crate) async fn handle_install(
// pick-time gate (the policy is wired into `config`).
if !request.trust_lockfile
&& let Some(input_lockfile) = request.lockfile.as_ref()
&& let Err(failure) = verify_input_lockfile(runtime, config, input_lockfile).await
&& let Err(failure) =
verify_input_lockfile(runtime, config, &request_auth, input_lockfile).await
{
return match failure {
VerifyFailure::Internal(response) => response,
@@ -217,13 +250,17 @@ pub(crate) async fn handle_install(
};
}
let lockfile = match resolve::resolve(config, &runtime.client, &request).await {
let lockfile = match resolve::resolve(config, &runtime.client, &request, &request_auth).await {
Ok(lockfile) => lockfile,
Err(err) => return json_error(StatusCode::INTERNAL_SERVER_ERROR, &err.to_string()),
};
let packages = resolve::collect_packages(&lockfile, &config.registry);
// `pkg_id`s fetched from upstream this request: the registry accepted
// the caller's token for each, so the gate treats them as proven.
let mut freshly_fetched: HashSet<String> = HashSet::new();
// `--lockfile-only`: pnpm resolves and writes the lockfile but
// fetches nothing and links nothing. Skip the tarball fetch + the
// file-level diff and return just the lockfile; the client writes it
@@ -236,8 +273,9 @@ pub(crate) async fn handle_install(
stats: diff::Stats { total_packages: packages.len() as u64, ..diff::Stats::default() },
}
} else {
if let Err(err) = resolve::fetch_uncached(config, &runtime.client, &packages).await {
return json_error(StatusCode::INTERNAL_SERVER_ERROR, &err.to_string());
match resolve::fetch_uncached(config, &runtime.client, &request_auth, &packages).await {
Ok(fetched) => freshly_fetched = fetched,
Err(err) => return json_error(StatusCode::INTERNAL_SERVER_ERROR, &err.to_string()),
}
let store = match StoreIndex::open_readonly_in(&config.store_dir) {
@@ -259,7 +297,17 @@ pub(crate) async fn handle_install(
}
};
if let Some(denied) = deny_unauthorized_packages(policies, &identity, &result.package_index) {
if let Some(denied) = authorize_served_packages(
runtime,
policies,
&identity,
&request,
&request_auth,
&freshly_fetched,
&result.package_index,
)
.await
{
return denied;
}
@@ -279,34 +327,63 @@ fn stats_json(stats: &diff::Stats) -> serde_json::Value {
})
}
/// Deny the install when the caller may not read a package whose files
/// are about to be served. A content-addressed digest is shared across
/// packages and reveals nothing about access, so possession of a
/// package's bytes in the store is never a capability to receive them:
/// every served package is checked against pnpr's own `packages:` policy
/// — the same decision `serve_packument` / `serve_tarball` make, in
/// process, with no network round trip. Returns the denial response (401
/// for an anonymous caller who could authenticate, 403 for an
/// authenticated caller outside the allowed set) or `None` when every
/// served package is readable.
///
/// This authorizes against pnpr's own surface, which is the authority for
/// everything the store can hold today: pnpr fetches anonymously (no
/// credential forwarding yet), so cached content is either pnpr-hosted or
/// publicly fetchable. When credential forwarding lands, packages the
/// client resolved from *external* registries under its own token become
/// reachable, and those carry no pnpr policy — their access must then be
/// re-verified per caller against the owning registry (with a TTL'd
/// verdict cache), which this local check does not cover.
fn deny_unauthorized_packages(
/// Authorize every served package before its files leave the store (a
/// shared content digest is never a read capability), dispatched by
/// whether a forwarded credential was used to fetch it: such packages are
/// gated per user against the owning registry
/// ([`authorize_upstream_package`]); the rest by pnpr's local `packages:`
/// policy ([`deny_local_policy`]). Returns the first denial, or `None`.
async fn authorize_served_packages(
runtime: &InstallAccelerator,
policies: &PackagePolicies,
identity: &Identity,
request: &InstallRequest,
request_auth: &AuthHeaders,
freshly_fetched: &HashSet<String>,
served: &[diff::PackageIndexEntry],
) -> Option<Response> {
let mut checked: HashSet<&str> = HashSet::new();
// The default registry pnpr resolved against (what `collect_packages`
// / `fetch_uncached` built every tarball URL from). Per-scope external
// registries are a future refinement.
let registry = request.registry.as_deref().unwrap_or("https://registry.npmjs.org/");
let mut local_pkg_ids: Vec<&str> = Vec::new();
for entry in served {
let Some(name) = package_name(&entry.pkg_id) else { continue };
// Access is decided per package name, so evaluate each name once.
let pkg_url = to_registry_url(registry, name);
if request_auth.for_url(&pkg_url).is_none() {
local_pkg_ids.push(entry.pkg_id.as_str());
continue;
}
if let Some(denied) = authorize_upstream_package(
runtime,
identity,
request_auth,
freshly_fetched,
registry,
name,
&entry.pkg_id,
)
.await
{
return Some(denied);
}
}
deny_local_policy(policies, identity, local_pkg_ids.into_iter())
}
/// Deny when the caller may not read a package gated by pnpr's own
/// `packages:` policy. 401 for anonymous, 403 for an authenticated caller
/// outside the allowed set; `None` when every name is readable.
fn deny_local_policy<'a>(
policies: &PackagePolicies,
identity: &Identity,
pkg_ids: impl Iterator<Item = &'a str>,
) -> Option<Response> {
let mut checked: HashSet<&str> = HashSet::new();
for pkg_id in pkg_ids {
let Some(name) = package_name(pkg_id) else { continue };
if !checked.insert(name) {
continue;
}
@@ -321,6 +398,125 @@ fn deny_unauthorized_packages(
None
}
/// Authorize one upstream-as-authority package: the owning registry, not
/// pnpr, decides. Known-public, freshly fetched, or already granted →
/// allow (recording a grant where applicable); otherwise probe the
/// registry anonymously (a `2xx` records it public globally) then
/// re-verify with the caller's token (`2xx` grants, `401`/`403` clears the
/// caller's grants and denies). Grants key on an identified user; the
/// global public set benefits anonymous callers too. See the body's
/// branches and the module tests for each path.
async fn authorize_upstream_package(
runtime: &InstallAccelerator,
identity: &Identity,
request_auth: &AuthHeaders,
freshly_fetched: &HashSet<String>,
registry: &str,
name: &str,
pkg_id: &str,
) -> Option<Response> {
// Public content needs no per-user gating, so it never reaches the
// grant table or an upstream round trip once classified.
if let Some(public) = runtime.public_packages.as_ref()
&& public.is_public(name, runtime.grant_ttl)
{
return None;
}
let user = match identity {
Identity::User { username } => Some(username.as_str()),
Identity::Anonymous => None,
};
let grants = || user.zip(runtime.grant_table.as_ref());
// The cold fetch this request already proved access: the upstream
// accepted the caller's forwarded token.
if freshly_fetched.contains(pkg_id) {
if let Some((user, table)) = grants() {
table.record(user, pkg_id);
}
return None;
}
if let Some((user, table)) = grants()
&& table.is_granted(user, pkg_id, runtime.grant_ttl)
{
return None;
}
// Classify before gating per user: a package the registry serves
// anonymously is public — record it globally so no one probes it
// again. Only a token-gated package takes the per-user path below.
if let UpstreamAccess::Authorized =
probe_upstream_access(&runtime.client, None, registry, name).await
{
if let Some(public) = runtime.public_packages.as_ref() {
public.record(name);
}
return None;
}
match probe_upstream_access(&runtime.client, Some(request_auth), registry, name).await {
UpstreamAccess::Authorized => {
if let Some((user, table)) = grants() {
table.record(user, pkg_id);
}
None
}
UpstreamAccess::Denied => {
if let Some((user, table)) = grants() {
table.clear_package(user, name);
}
Some(json_error(StatusCode::FORBIDDEN, &format!("not authorized to access {name:?}")))
}
UpstreamAccess::Unknown => Some(json_error(
StatusCode::BAD_GATEWAY,
&format!("could not verify access to {name:?}"),
)),
}
}
/// Outcome of an upstream access probe.
enum UpstreamAccess {
/// The upstream served the package's packument for the probe.
Authorized,
/// The upstream returned `401`/`403`.
Denied,
/// The upstream was unreachable or returned some other status; access
/// can't be decided.
Unknown,
}
/// Probe whether `name` is readable from `registry` by fetching its
/// (abbreviated) packument. `auth` set attaches the caller's credential
/// (a re-verify); `auth` `None` is anonymous (a public/private check).
async fn probe_upstream_access(
client: &ThrottledClient,
auth: Option<&AuthHeaders>,
registry: &str,
name: &str,
) -> UpstreamAccess {
let url = to_registry_url(registry, name);
let guard = client.acquire_for_url(&url).await;
let mut request = guard.get(&url).header("accept", "application/vnd.npm.install-v1+json");
if let Some(value) = auth.and_then(|auth| auth.for_url(&url)) {
request = request.header("authorization", value);
}
match request.send().await {
Ok(response) => {
let status = response.status().as_u16();
if (200..300).contains(&status) {
UpstreamAccess::Authorized
} else if status == 401 || status == 403 {
UpstreamAccess::Denied
} else {
UpstreamAccess::Unknown
}
}
Err(_) => UpstreamAccess::Unknown,
}
}
/// The package name from a `name@version` package id, tolerating a
/// leading scope `@` (`@scope/foo@1.0.0` → `@scope/foo`).
fn package_name(pkg_id: &str) -> Option<&str> {
@@ -421,6 +617,7 @@ enum VerifyFailure {
async fn verify_input_lockfile(
runtime: &InstallAccelerator,
config: &'static PacquetConfig,
auth_headers: &Arc<AuthHeaders>,
lockfile: &Lockfile,
) -> Result<(), VerifyFailure> {
// A fresh per-request packument cache shared with the verifier; the
@@ -431,6 +628,7 @@ async fn verify_input_lockfile(
config,
Arc::clone(&runtime.client),
Some(meta_cache as Arc<dyn PackageMetaCache>),
Some(Arc::clone(auth_headers)),
)
.map_err(|err| {
VerifyFailure::Internal(json_error(StatusCode::INTERNAL_SERVER_ERROR, &err.to_string()))

108
pnpr/crates/pnpr/src/install_accelerator/grant_table.rs Normal file
View File

@@ -0,0 +1,108 @@
//! Per-`(user, name@version)` allow-list gating externally-resolved
//! private content ([pnpm/pnpm#12184](https://github.com/pnpm/pnpm/issues/12184)):
//! the store dedups the bytes globally, but possession must not authorize
//! a user the owning registry never cleared. Backed by SQLite (WAL) like
//! [`super::verdict_cache::VerdictCache`]; every method is best-effort (a
//! DB error never fails the request, at worst one extra re-verify).
use std::{
path::Path,
sync::Mutex,
time::{Duration, SystemTime, UNIX_EPOCH},
};
use rusqlite::Connection;
/// Soft cap on stored grants; the oldest rows (by `granted_at_ms`) are
/// evicted past this.
const MAX_ROWS: i64 = 100_000;
/// Concurrency-safe store of per-`(user, name@version)` access grants.
pub(crate) struct GrantTable {
conn: Mutex<Connection>,
}
impl GrantTable {
/// Open (creating if needed) the grant database at `path`.
pub(crate) fn open(path: &Path) -> rusqlite::Result<Self> {
let conn = Connection::open(path)?;
conn.busy_timeout(Duration::from_secs(5))?;
conn.execute_batch(
"PRAGMA journal_mode=WAL;
CREATE TABLE IF NOT EXISTS grants (
user TEXT NOT NULL,
pkg TEXT NOT NULL,
granted_at_ms INTEGER NOT NULL,
PRIMARY KEY (user, pkg)
);",
)?;
Ok(Self { conn: Mutex::new(conn) })
}
/// Whether `(user, pkg)` holds a grant still within `ttl` (`None` =
/// permanent). `pkg` is the `name@version` package id.
pub(crate) fn is_granted(&self, user: &str, pkg: &str, ttl: Option<Duration>) -> bool {
let conn = self.conn.lock().expect("grant table poisoned");
let granted_at: Option<i64> = conn
.query_row(
"SELECT granted_at_ms FROM grants WHERE user = ?1 AND pkg = ?2",
rusqlite::params![user, pkg],
|row| row.get(0),
)
.ok();
let Some(granted_at) = granted_at else {
return false;
};
match ttl {
None => true,
Some(ttl) => now_ms().saturating_sub(granted_at) <= ttl.as_millis() as i64,
}
}
/// Record (or refresh) a grant for `(user, pkg)`. Best-effort.
pub(crate) fn record(&self, user: &str, pkg: &str) {
let now = now_ms();
let conn = self.conn.lock().expect("grant table poisoned");
let _ = conn.execute(
"INSERT INTO grants (user, pkg, granted_at_ms) VALUES (?1, ?2, ?3)
ON CONFLICT(user, pkg) DO UPDATE SET granted_at_ms = excluded.granted_at_ms",
rusqlite::params![user, pkg, now],
);
evict_overflow(&conn);
}
/// Clear-on-discovery: drop every grant `user` holds for `name`,
/// across all versions (matched by the `name@` prefix, since `pkg` is
/// `name@version`). Best-effort.
pub(crate) fn clear_package(&self, user: &str, name: &str) {
let with_at = format!("{name}@");
let prefix_len = with_at.chars().count() as i64;
let conn = self.conn.lock().expect("grant table poisoned");
let _ = conn.execute(
"DELETE FROM grants WHERE user = ?1 AND substr(pkg, 1, ?2) = ?3",
rusqlite::params![user, prefix_len, with_at],
);
}
}
/// Trim the oldest rows past [`MAX_ROWS`], ordered by `granted_at_ms`.
fn evict_overflow(conn: &Connection) {
let _ = conn.execute(
"DELETE FROM grants WHERE rowid IN (
SELECT rowid FROM grants
ORDER BY granted_at_ms DESC
LIMIT -1 OFFSET ?1
)",
rusqlite::params![MAX_ROWS],
);
}
fn now_ms() -> i64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|elapsed| elapsed.as_millis() as i64)
.unwrap_or(0)
}
#[cfg(test)]
mod tests;

74
pnpr/crates/pnpr/src/install_accelerator/grant_table/tests.rs Normal file
View File

@@ -0,0 +1,74 @@
use std::{thread::sleep, time::Duration};
use tempfile::TempDir;
use super::GrantTable;
fn open() -> (GrantTable, TempDir) {
let dir = TempDir::new().expect("tempdir");
let table = GrantTable::open(&dir.path().join("grants.sqlite")).expect("open grant table");
(table, dir)
}
#[test]
fn records_and_reads_a_grant() {
let (table, _dir) = open();
assert!(!table.is_granted("alice", "@acme/foo@1.0.0", None));
table.record("alice", "@acme/foo@1.0.0");
assert!(table.is_granted("alice", "@acme/foo@1.0.0", None));
// A grant is per-user and per-version.
assert!(!table.is_granted("bob", "@acme/foo@1.0.0", None));
assert!(!table.is_granted("alice", "@acme/foo@2.0.0", None));
}
#[test]
fn clear_package_drops_every_version_for_that_user_only() {
let (table, _dir) = open();
table.record("alice", "@acme/foo@1.0.0");
table.record("alice", "@acme/foo@2.0.0");
table.record("alice", "@acme/bar@1.0.0");
table.record("bob", "@acme/foo@1.0.0");
table.clear_package("alice", "@acme/foo");
assert!(!table.is_granted("alice", "@acme/foo@1.0.0", None));
assert!(!table.is_granted("alice", "@acme/foo@2.0.0", None));
// A different package the same user holds is untouched.
assert!(table.is_granted("alice", "@acme/bar@1.0.0", None));
// Another user's grant for the same package is untouched.
assert!(table.is_granted("bob", "@acme/foo@1.0.0", None));
}
#[test]
fn clear_package_does_not_prefix_match_a_sibling_name() {
let (table, _dir) = open();
// `foo` must not clear `foo-bar` — the `@`-delimited prefix guards it.
table.record("alice", "foo@1.0.0");
table.record("alice", "foo-bar@1.0.0");
table.clear_package("alice", "foo");
assert!(!table.is_granted("alice", "foo@1.0.0", None));
assert!(table.is_granted("alice", "foo-bar@1.0.0", None));
}
#[test]
fn a_ttl_expires_an_old_grant() {
let (table, _dir) = open();
table.record("alice", "foo@1.0.0");
// Still valid under a generous TTL.
assert!(table.is_granted("alice", "foo@1.0.0", Some(Duration::from_secs(60))));
// Expired under a zero TTL once any time has passed.
sleep(Duration::from_millis(5));
assert!(!table.is_granted("alice", "foo@1.0.0", Some(Duration::from_millis(1))));
}
#[test]
fn grants_persist_across_reopen() {
let dir = TempDir::new().expect("tempdir");
let path = dir.path().join("grants.sqlite");
{
let table = GrantTable::open(&path).expect("open");
table.record("alice", "foo@1.0.0");
}
let reopened = GrantTable::open(&path).expect("reopen");
assert!(reopened.is_granted("alice", "foo@1.0.0", None));
}

7
pnpr/crates/pnpr/src/install_accelerator/protocol.rs
View File

@@ -53,6 +53,13 @@ pub struct InstallRequest {
/// `namedRegistries`).
#[serde(default)]
pub named_registries: BTreeMap<String, String>,
/// The caller's forwarded upstream credentials so the server resolves
/// and fetches private content as the caller. Keyed by nerf-darted
/// registry URI with ready-to-send values, the shape
/// [`pacquet_network::AuthHeaders::from_map`] consumes. Distinct from
/// the request's HTTP `Authorization` header (pnpr identity).
#[serde(default)]
pub auth_headers: BTreeMap<String, String>,
/// The client's `overrides` (selector -> spec), applied at resolve
/// time. Kept as raw JSON; reconstructed into pacquet's override map
/// server-side.

95
pnpr/crates/pnpr/src/install_accelerator/public_packages.rs Normal file
View File

@@ -0,0 +1,95 @@
//! Global set of **anonymously-readable** package names, so the per-user
//! grant table never gates a public package
//! ([pnpm/pnpm#12184](https://github.com/pnpm/pnpm/issues/12184)). A
//! forwarded token matching a registry only means pnpr fetched a package
//! with it, not that the package is private; in a mixed proxy that would
//! gate public content per user too. Populated lazily by one anonymous
//! probe per name, so a public package costs one round trip fleet-wide.
//! SQLite (WAL) like [`super::grant_table::GrantTable`]; best-effort.
use std::{
path::Path,
sync::Mutex,
time::{Duration, SystemTime, UNIX_EPOCH},
};
use rusqlite::Connection;
/// Soft cap on classified names; the oldest rows (by `classified_at_ms`)
/// are evicted past this.
const MAX_ROWS: i64 = 100_000;
/// Concurrency-safe set of anonymously-readable package names.
pub(crate) struct PublicPackages {
conn: Mutex<Connection>,
}
impl PublicPackages {
/// Open (creating if needed) the classification database at `path`.
pub(crate) fn open(path: &Path) -> rusqlite::Result<Self> {
let conn = Connection::open(path)?;
conn.busy_timeout(Duration::from_secs(5))?;
conn.execute_batch(
"PRAGMA journal_mode=WAL;
CREATE TABLE IF NOT EXISTS public_packages (
name TEXT PRIMARY KEY,
classified_at_ms INTEGER NOT NULL
);",
)?;
Ok(Self { conn: Mutex::new(conn) })
}
/// Whether `name` was classified anonymously-readable within `ttl`
/// (`None` = permanent). Keyed by name (readability is per-name).
pub(crate) fn is_public(&self, name: &str, ttl: Option<Duration>) -> bool {
let conn = self.conn.lock().expect("public packages poisoned");
let classified_at: Option<i64> = conn
.query_row(
"SELECT classified_at_ms FROM public_packages WHERE name = ?1",
rusqlite::params![name],
|row| row.get(0),
)
.ok();
let Some(classified_at) = classified_at else {
return false;
};
match ttl {
None => true,
Some(ttl) => now_ms().saturating_sub(classified_at) <= ttl.as_millis() as i64,
}
}
/// Record (or refresh) `name` as anonymously readable. Best-effort.
pub(crate) fn record(&self, name: &str) {
let now = now_ms();
let conn = self.conn.lock().expect("public packages poisoned");
let _ = conn.execute(
"INSERT INTO public_packages (name, classified_at_ms) VALUES (?1, ?2)
ON CONFLICT(name) DO UPDATE SET classified_at_ms = excluded.classified_at_ms",
rusqlite::params![name, now],
);
evict_overflow(&conn);
}
}
/// Trim the oldest rows past [`MAX_ROWS`], ordered by `classified_at_ms`.
fn evict_overflow(conn: &Connection) {
let _ = conn.execute(
"DELETE FROM public_packages WHERE name IN (
SELECT name FROM public_packages
ORDER BY classified_at_ms DESC
LIMIT -1 OFFSET ?1
)",
rusqlite::params![MAX_ROWS],
);
}
fn now_ms() -> i64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|elapsed| elapsed.as_millis() as i64)
.unwrap_or(0)
}
#[cfg(test)]
mod tests;

42
pnpr/crates/pnpr/src/install_accelerator/public_packages/tests.rs Normal file
View File

@@ -0,0 +1,42 @@
use std::{thread::sleep, time::Duration};
use tempfile::TempDir;
use super::PublicPackages;
fn open() -> (PublicPackages, TempDir) {
let dir = TempDir::new().expect("tempdir");
let table = PublicPackages::open(&dir.path().join("public.sqlite")).expect("open");
(table, dir)
}
#[test]
fn records_and_reads_a_classification() {
let (table, _dir) = open();
assert!(!table.is_public("lodash", None));
table.record("lodash");
assert!(table.is_public("lodash", None));
// Classification is per name, not per other name.
assert!(!table.is_public("react", None));
}
#[test]
fn a_ttl_expires_an_old_classification() {
let (table, _dir) = open();
table.record("lodash");
assert!(table.is_public("lodash", Some(Duration::from_secs(60))));
sleep(Duration::from_millis(5));
assert!(!table.is_public("lodash", Some(Duration::from_millis(1))));
}
#[test]
fn classifications_persist_across_reopen() {
let dir = TempDir::new().expect("tempdir");
let path = dir.path().join("public.sqlite");
{
let table = PublicPackages::open(&path).expect("open");
table.record("lodash");
}
let reopened = PublicPackages::open(&path).expect("reopen");
assert!(reopened.is_public("lodash", None));
}

21
pnpr/crates/pnpr/src/install_accelerator/resolve.rs
View File

@@ -14,7 +14,7 @@ use std::{
use dashmap::DashMap;
use pacquet_config::{Config, NodeLinker};
use pacquet_lockfile::{Lockfile, LockfileResolution};
use pacquet_network::ThrottledClient;
use pacquet_network::{AuthHeaders, ThrottledClient};
use pacquet_package_manager::{Install, ResolvedPackages};
use pacquet_package_manifest::{DependencyGroup, PackageManifest};
use pacquet_reporter::SilentReporter;
@@ -71,6 +71,7 @@ pub async fn resolve(
config: &'static Config,
client: &Arc<ThrottledClient>,
request: &InstallRequest,
auth_headers: &Arc<AuthHeaders>,
) -> Result<Lockfile, ResolveError> {
let projects = request.projects_normalized();
@@ -191,6 +192,9 @@ pub async fn resolve(
node_linker: NodeLinker::Isolated,
lockfile_only: true,
update_seed_policy: pacquet_package_manager::UpdateSeedPolicy::KeepAll,
// Resolve as the caller (forwarded credentials) without baking
// per-user auth into the interned `&'static Config`.
auth_override: Some(Arc::clone(auth_headers)),
}
.run::<SilentReporter>()
.await
@@ -224,11 +228,16 @@ pub fn collect_packages(lockfile: &Lockfile, registry: &str) -> Vec<ResolvedPkg>
/// Fetch into the shared store every package whose store-index row is
/// absent, populating its `PackageFilesIndex` as a side effect. Cached
/// packages are skipped, matching the server hot-cache no-op.
///
/// Returns the `pkg_id`s actually fetched this call — the upstream
/// accepted the caller's credentials for each, so the gate treats a
/// freshly-fetched private package as proven (no re-verify).
pub async fn fetch_uncached(
config: &'static Config,
client: &Arc<ThrottledClient>,
auth_headers: &AuthHeaders,
packages: &[ResolvedPkg],
) -> Result<(), ResolveError> {
) -> Result<HashSet<String>, ResolveError> {
let store_dir = &config.store_dir;
let present: HashSet<String> = match StoreIndex::open_readonly_in(store_dir) {
@@ -243,9 +252,11 @@ pub async fn fetch_uncached(
.collect();
if to_fetch.is_empty() {
return Ok(());
return Ok(HashSet::new());
}
let fetched_ids: HashSet<String> = to_fetch.iter().map(|pkg| pkg.pkg_id.clone()).collect();
let integrities: Vec<Option<ssri::Integrity>> =
to_fetch.iter().map(|pkg| pkg.integrity.parse::<ssri::Integrity>().ok()).collect();
@@ -270,7 +281,7 @@ pub async fn fetch_uncached(
package_unpacked_size: None,
package_url: &pkg.tarball_url,
package_id: &pkg.pkg_id,
auth_headers: &config.auth_headers,
auth_headers,
requester: "pnpr",
prefetched_cas_paths: None,
retry_opts: RetryOpts::default(),
@@ -291,7 +302,7 @@ pub async fn fetch_uncached(
for result in results {
result?;
}
Ok(())
Ok(fetched_ids)
}
/// Derive `(integrity, tarball_url)` for a resolution, mirroring

313
pnpr/crates/pnpr/src/install_accelerator/tests.rs
View File

@@ -1,19 +1,34 @@
//! Tests for the per-caller access gate the install accelerator applies
//! before serving a package's files: a digest in the store is not a
//! bearer capability, so [`deny_unauthorized_packages`] checks every
//! served package against pnpr's own `packages:` policy.
//! Tests for the pnpr-as-authority access gate the install accelerator
//! applies before serving a package's files: a digest in the store is not
//! a bearer capability, so [`deny_local_policy`] checks every locally-
//! authoritative package against pnpr's own `packages:` policy. (The
//! upstream-as-authority regime — forwarded-credential content gated per
//! user — is exercised end to end in the pnpr-client integration tests.)
use std::collections::HashSet;
use axum::http::StatusCode;
use pacquet_network::AuthHeaders;
use tempfile::TempDir;
use super::{deny_unauthorized_packages, diff::PackageIndexEntry};
use super::{
InstallAccelerator, authorize_served_packages, authorize_upstream_package, deny_local_policy,
diff::PackageIndexEntry, protocol::InstallRequest,
};
use crate::policy::{AccessList, Identity, PackagePolicies, PackagePolicy};
fn served(name: &str) -> Vec<PackageIndexEntry> {
vec![PackageIndexEntry {
integrity: "sha512-deadbeef".to_string(),
pkg_id: format!("{name}@1.0.0"),
raw: Vec::new(),
}]
/// The `name@1.0.0` package id a served entry would carry.
fn served(name: &str) -> String {
format!("{name}@1.0.0")
}
/// Run the local-policy gate over a single served package id.
fn deny(
policies: &PackagePolicies,
identity: &Identity,
pkg_id: &str,
) -> Option<axum::response::Response> {
deny_local_policy(policies, identity, std::iter::once(pkg_id))
}
fn anonymous() -> Identity {
@@ -43,31 +58,297 @@ fn team_owned_by_alice() -> PackagePolicies {
#[test]
fn anonymous_caller_is_denied_a_private_package() {
let denied = deny_unauthorized_packages(&policies(), &anonymous(), &served("@private/foo"));
let denied = deny(&policies(), &anonymous(), &served("@private/foo"));
assert_eq!(denied.map(|response| response.status()), Some(StatusCode::UNAUTHORIZED));
}
#[test]
fn authenticated_caller_is_allowed_a_private_package() {
let denied = deny_unauthorized_packages(&policies(), &user(), &served("@private/foo"));
let denied = deny(&policies(), &user(), &served("@private/foo"));
assert!(denied.is_none());
}
#[test]
fn anonymous_caller_is_allowed_a_public_package() {
let denied = deny_unauthorized_packages(&policies(), &anonymous(), &served("is-positive"));
let denied = deny(&policies(), &anonymous(), &served("is-positive"));
assert!(denied.is_none());
}
#[test]
fn authenticated_caller_outside_the_allowed_set_is_forbidden() {
let bob = Identity::User { username: "bob".to_string() };
let denied = deny_unauthorized_packages(&team_owned_by_alice(), &bob, &served("@team/foo"));
let denied = deny(&team_owned_by_alice(), &bob, &served("@team/foo"));
assert_eq!(denied.map(|response| response.status()), Some(StatusCode::FORBIDDEN));
}
#[test]
fn authenticated_caller_in_the_allowed_set_is_allowed() {
let denied = deny_unauthorized_packages(&team_owned_by_alice(), &user(), &served("@team/foo"));
let denied = deny(&team_owned_by_alice(), &user(), &served("@team/foo"));
assert!(denied.is_none());
}
// --------------------------------------------------------------------
// Upstream-as-authority regime: forwarded-credential content gated per
// user against the owning external registry, plus the grant table.
// --------------------------------------------------------------------
/// Build a real [`InstallAccelerator`] (store/cache dirs + grant table)
/// under `storage` for the dispatch tests.
fn accelerator(storage: &std::path::Path) -> InstallAccelerator {
let addr = "127.0.0.1:4873".parse().expect("addr parses");
let config = crate::config::Config::proxy(addr, storage.to_path_buf());
InstallAccelerator::build(&config)
}
/// An [`AuthHeaders`] carrying a single default-registry credential for
/// `registry`, mirroring how a client forwards one upstream token.
fn auth_for(registry: &str, header: &str) -> AuthHeaders {
AuthHeaders::from_creds_map([(String::new(), header.to_string())], Some(registry))
}
fn entry(pkg_id: &str) -> PackageIndexEntry {
PackageIndexEntry {
integrity: "sha512-x".to_string(),
pkg_id: pkg_id.to_string(),
raw: Vec::new(),
}
}
fn is_granted(acc: &InstallAccelerator, user: &str, pkg: &str) -> bool {
acc.grant_table.as_ref().expect("grant table opened").is_granted(user, pkg, None)
}
fn is_public(acc: &InstallAccelerator, name: &str) -> bool {
acc.public_packages.as_ref().expect("public set opened").is_public(name, None)
}
fn fresh(pkg_ids: &[&str]) -> HashSet<String> {
pkg_ids.iter().map(|id| id.to_string()).collect()
}
#[tokio::test]
async fn a_fresh_upstream_fetch_is_allowed_and_records_a_grant() {
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
let auth = auth_for("https://reg.test/", "Bearer t");
let identity = Identity::User { username: "alice".to_string() };
let denied = authorize_upstream_package(
&acc,
&identity,
&auth,
&fresh(&["foo@1.0.0"]),
"https://reg.test/",
"foo",
"foo@1.0.0",
)
.await;
assert!(denied.is_none());
assert!(is_granted(&acc, "alice", "foo@1.0.0"));
}
#[tokio::test]
async fn a_granted_cache_hit_is_served_without_touching_the_upstream() {
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
acc.grant_table.as_ref().unwrap().record("alice", "foo@1.0.0");
let auth = auth_for("https://reg.test/", "Bearer t");
let identity = Identity::User { username: "alice".to_string() };
// An unreachable registry: a network probe would resolve to a 502
// denial, so a pass here proves the grant short-circuited it.
let denied = authorize_upstream_package(
&acc,
&identity,
&auth,
&fresh(&[]),
"http://127.0.0.1:1/",
"foo",
"foo@1.0.0",
)
.await;
assert!(denied.is_none());
}
#[tokio::test]
async fn an_ungranted_private_cache_hit_reverifies_then_records() {
let mut server = mockito::Server::new_async().await;
// Private: the registry withholds the packument anonymously, then
// serves it once the caller's credential is attached. The two mocks
// are mutually exclusive on the `authorization` header.
let anon = server
.mock("GET", "/foo")
.match_header("authorization", mockito::Matcher::Missing)
.with_status(401)
.create_async()
.await;
let authed = server
.mock("GET", "/foo")
.match_header("authorization", "Bearer t")
.with_status(200)
.with_body("{}")
.create_async()
.await;
let registry = format!("{}/", server.url());
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
let auth = auth_for(&registry, "Bearer t");
let identity = Identity::User { username: "alice".to_string() };
let denied = authorize_upstream_package(
&acc,
&identity,
&auth,
&fresh(&[]),
&registry,
"foo",
"foo@1.0.0",
)
.await;
assert!(denied.is_none());
anon.assert_async().await;
authed.assert_async().await;
assert!(is_granted(&acc, "alice", "foo@1.0.0"));
// A private package must never be cached as public.
assert!(!is_public(&acc, "foo"));
}
#[tokio::test]
async fn a_public_cache_hit_is_classified_once_then_served_for_free() {
let mut server = mockito::Server::new_async().await;
// Public: the registry serves the packument anonymously. Exactly one
// probe is expected across both authorize calls — the second is served
// from the global classification with no upstream contact.
let mock =
server.mock("GET", "/foo").with_status(200).with_body("{}").expect(1).create_async().await;
let registry = format!("{}/", server.url());
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
let auth = auth_for(&registry, "Bearer t");
let alice = Identity::User { username: "alice".to_string() };
let first =
authorize_upstream_package(&acc, &alice, &auth, &fresh(&[]), &registry, "foo", "foo@1.0.0")
.await;
assert!(first.is_none());
assert!(is_public(&acc, "foo"));
// Public content records no per-user grant.
assert!(!is_granted(&acc, "alice", "foo@1.0.0"));
// A different caller wanting a different cached version is served
// straight from the classification — no second probe.
let bob = Identity::User { username: "bob".to_string() };
let second =
authorize_upstream_package(&acc, &bob, &auth, &fresh(&[]), &registry, "foo", "foo@2.0.0")
.await;
assert!(second.is_none());
mock.assert_async().await;
}
#[tokio::test]
async fn a_denied_reverify_clears_the_users_grants_and_denies() {
let mut server = mockito::Server::new_async().await;
let _mock = server.mock("GET", "/foo").with_status(403).create_async().await;
let registry = format!("{}/", server.url());
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
// A standing grant for another version the caller already held: a
// discovered `403` for the package must purge it (clear-on-discovery).
acc.grant_table.as_ref().unwrap().record("alice", "foo@2.0.0");
let auth = auth_for(&registry, "Bearer t");
let identity = Identity::User { username: "alice".to_string() };
let denied = authorize_upstream_package(
&acc,
&identity,
&auth,
&fresh(&[]),
&registry,
"foo",
"foo@1.0.0",
)
.await;
assert_eq!(denied.map(|response| response.status()), Some(StatusCode::FORBIDDEN));
assert!(!is_granted(&acc, "alice", "foo@2.0.0"));
}
#[tokio::test]
async fn an_unreachable_upstream_during_reverify_is_a_bad_gateway() {
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
let auth = auth_for("http://127.0.0.1:1/", "Bearer t");
let identity = Identity::User { username: "alice".to_string() };
// Port 1 refuses the connection, so neither the anonymous classify
// probe nor the authed re-verify can decide access.
let denied = authorize_upstream_package(
&acc,
&identity,
&auth,
&fresh(&[]),
"http://127.0.0.1:1/",
"foo",
"foo@1.0.0",
)
.await;
assert_eq!(denied.map(|response| response.status()), Some(StatusCode::BAD_GATEWAY));
}
#[tokio::test]
async fn a_forwarded_credential_routes_around_the_local_policy() {
// `@private/foo` is gated to `$authenticated` by the local policy, so
// an anonymous caller would be denied under pnpr-as-authority. With a
// forwarded credential it is upstream-as-authority instead, and a
// fresh fetch proves access — so it is served.
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
let registry = "https://reg.test/";
let auth = auth_for(registry, "Bearer t");
let request = InstallRequest { registry: Some(registry.to_string()), ..Default::default() };
let denied = authorize_served_packages(
&acc,
&policies(),
&Identity::Anonymous,
&request,
&auth,
&fresh(&["@private/foo@1.0.0"]),
&[entry("@private/foo@1.0.0")],
)
.await;
assert!(denied.is_none());
}
#[tokio::test]
async fn without_a_forwarded_credential_the_local_policy_still_applies() {
let tmp = TempDir::new().unwrap();
let acc = accelerator(tmp.path());
let request =
InstallRequest { registry: Some("https://reg.test/".to_string()), ..Default::default() };
// No forwarded credential ⇒ pnpr-as-authority ⇒ `@private/foo` is
// denied to an anonymous caller, exactly as the packument/tarball
// endpoints would deny it.
let denied = authorize_served_packages(
&acc,
&policies(),
&Identity::Anonymous,
&request,
&AuthHeaders::default(),
&fresh(&[]),
&[entry("@private/foo@1.0.0")],
)
.await;
assert_eq!(denied.map(|response| response.status()), Some(StatusCode::UNAUTHORIZED));
}