mirror of
https://github.com/pnpm/pnpm.git
synced 2026-05-31 12:10:49 -04:00
perf(npm-resolver): layer abbreviated meta + attestation before full metadata in the minimumReleaseAge gate (#11704)
Follow-up to #11691 — item 2 from #11687, plus a related shortcut. ## What When the `minimumReleaseAge` lockfile verification gate needs to know when a version was published, it used to fetch a multi-MB full metadata document per package just to read one timestamp. This PR replaces that single-step path with a four-layer lookup that pays the cheapest viable source first: 1. **Abbreviated metadata's `modified` field** — the resolver already fetches this for resolution. If the package as a whole hasn't been modified within the policy cutoff, every version it contains is at least that old; return `modified` as a conservative upper-bound and skip the rest of the chain. 2. **Local `FULL_META_DIR` mirror** — exact per-version times if a previous verification populated it. 3. **npm attestation endpoint** (`/-/npm/v1/attestations/<name>@<version>`) — a tens-of-KB Sigstore bundle whose Rekor inclusion time stands in for publish time. Wins on cold cache when the package was published with provenance. 4. **Full metadata fetch** — last resort. ## Why The verification cache from #11691 made repeat installs against an unchanged lockfile effectively free. The remaining cost is the *first* verification on a fresh CI runner with no restored cache — particularly `pnpm install --frozen-lockfile`, where every locked package's publish timestamp has to be confirmed. Fetching the full metadata document for each package is wasteful when: - The resolver has usually already cached abbreviated metadata, whose `modified` field alone is enough to clear stable packages (the common case). - For recently-modified packages, the per-version attestation endpoint is orders of magnitude smaller than full metadata. ## How ### Abbreviated `modified` shortcut `fetchFullMetadataCached` is refactored to share an internal helper with a new `fetchAbbreviatedMetadataCached`. Both do conditional GETs against their respective on-disk mirrors. On a non-frozen install the abbreviated mirror is already populated by the resolver, so the shortcut hits the local cache at headers-only cost. On `--frozen-lockfile` the fetch is still cheaper than full metadata. If `Date.parse(modified) < cutoff`, return `modified` — it's an upper bound on every version's publish time in this package, and the verifier's `published < cutoff` check passes trivially. ### Attestation endpoint `fetchAttestationPublishedAt` (new module) hits `/-/npm/v1/attestations/<name>@<version>`, parses the response, and reads the earliest `bundle.verificationMaterial.tlogEntries[].integratedTime` across the attestation bundles. That's the Rekor inclusion time — a couple of seconds after publish, well within tolerance for a policy that operates in minutes/hours/days. Returns `undefined` on 404 / network error / malformed body so the caller falls back. ### Per-install dedup The lookup carries a `PublishedAtLookupContext` with four memo maps: abbreviated meta per (registry, name), local full meta per (registry, name), full meta network fetch per (registry, name), final published-at per (registry, name, version). Verifying many versions of the same package only pays the disk/network costs once. ## Trust model **No Sigstore signature verification on the attestation path.** The trust model is identical to reading the registry's `time` field on the full metadata document — we already trust the registry to serve correct publish timestamps for the gate's purpose. The win is purely bandwidth. Full Sigstore verification (Fulcio cert chain + Rekor inclusion proof) would harden the timestamp against a compromised registry. It pulls in the `sigstore` npm package and the TUF root — a separate dependency-surface discussion, parked as future work. ## Tests - **13 unit tests** in `resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts`: ISO timestamp extraction, URL construction (scoped + unscoped), 404 / 5xx / network error / malformed JSON / missing fields → undefined, earliest-of-multiple-attestations, defensive number-as-integratedTime, auth header forwarding, trailing-slash normalization. - Existing `minimumReleaseAge` + `verifyLockfileResolutions` integration suites (45 tests) still pass — the fallback chain preserves end-to-end behavior when the new shortcuts don't apply.
This commit is contained in:
8
.changeset/attestation-first-min-release-age.md
Normal file
8
.changeset/attestation-first-min-release-age.md
Normal file
@@ -0,0 +1,8 @@
|
||||
---
|
||||
"@pnpm/resolving.npm-resolver": minor
|
||||
"pnpm": patch
|
||||
---
|
||||
|
||||
Sped up the `minimumReleaseAge` lockfile verification gate on cold-cache installs by trying npm's `/-/npm/v1/attestations/<name>@<version>` endpoint before fetching the full metadata document. The attestation response is tens of KB versus the multi-MB full metadata, so `--frozen-lockfile` installs against a fleet of provenance-published packages download far less to verify timestamps.
|
||||
|
||||
The publish time comes from `bundle.verificationMaterial.tlogEntries[].integratedTime` (the Rekor inclusion time, a couple of seconds after the actual publish — close enough for a policy that operates in minutes/hours/days). When the local full-metadata mirror already has the timestamp, or the attestation endpoint 404s / errors, the verifier falls back to the existing `fetchFullMetadataCached` path. Sigstore signature verification is not performed; the trust model is unchanged versus reading the registry's `time` field on the full metadata document [#11687](https://github.com/pnpm/pnpm/issues/11687).
|
||||
@@ -279,6 +279,7 @@
|
||||
"rehoist",
|
||||
"reimagining",
|
||||
"reka",
|
||||
"Rekor",
|
||||
"relinks",
|
||||
"renderable",
|
||||
"replit",
|
||||
@@ -306,6 +307,7 @@
|
||||
"sigstore",
|
||||
"sindresorhus",
|
||||
"sirv",
|
||||
"SLSA",
|
||||
"soporan",
|
||||
"sopts",
|
||||
"spdxdocs",
|
||||
@@ -337,6 +339,7 @@
|
||||
"teambit",
|
||||
"tempy",
|
||||
"testcase",
|
||||
"tlog",
|
||||
"TLSV",
|
||||
"toctou",
|
||||
"todomvc",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { pickRegistryForPackage } from '@pnpm/config.pick-registry-for-package'
|
||||
import { createPackageVersionPolicy } from '@pnpm/config.version-policy'
|
||||
import { FULL_META_DIR } from '@pnpm/constants'
|
||||
import { PnpmError } from '@pnpm/error'
|
||||
import type {
|
||||
Resolution,
|
||||
@@ -9,8 +10,14 @@ import type { PackageVersionPolicy, Registries } from '@pnpm/types'
|
||||
import semver from 'semver'
|
||||
|
||||
import type { FetchMetadataFromFromRegistryOptions } from './fetch.js'
|
||||
import { fetchFullMetadataCached, type FetchFullMetadataCachedOptions } from './fetchFullMetadataCached.js'
|
||||
import { fetchAttestationPublishedAt } from './fetchAttestationPublishedAt.js'
|
||||
import {
|
||||
fetchAbbreviatedMetadataCached,
|
||||
fetchFullMetadataCached,
|
||||
type FetchFullMetadataCachedOptions,
|
||||
} from './fetchFullMetadataCached.js'
|
||||
import { BUILTIN_NAMED_REGISTRIES } from './parseBareSpecifier.js'
|
||||
import { getPkgMirrorPath, loadMeta } from './pickPackage.js'
|
||||
|
||||
export interface CreateNpmResolutionVerifierOptions {
|
||||
/**
|
||||
@@ -91,21 +98,19 @@ export function createNpmResolutionVerifier (
|
||||
.filter((value): value is string => value != null)
|
||||
.sort((a, b) => b.length - a.length)
|
||||
|
||||
// In-memory dedup of the time map per (registry, name) for this verifier
|
||||
// instance. The on-disk conditional-GET cache is handled inside
|
||||
// fetchFullMetadataCached via the resolver's shared mirror at opts.cacheDir.
|
||||
const inflight = new Map<string, Promise<Record<string, string | undefined> | undefined>>()
|
||||
const fetchTimeMap = async (registry: string, name: string): Promise<Record<string, string | undefined> | undefined> => {
|
||||
const cacheKey = `${registry}\x00${name}`
|
||||
const cached = inflight.get(cacheKey)
|
||||
if (cached) return cached
|
||||
const promise = fetchFullMetadataCached(opts.fetchOpts, name, {
|
||||
registry,
|
||||
authHeaderValue: opts.getAuthHeaderValueByURI(registry),
|
||||
cacheDir: opts.cacheDir,
|
||||
}).then((meta) => meta.time)
|
||||
inflight.set(cacheKey, promise)
|
||||
return promise
|
||||
// Per-install dedup of every network/disk fetch the verifier issues
|
||||
// (see fetchPublishedAt for the lookup order). The on-disk
|
||||
// conditional-GET cache is handled inside fetch{Abbreviated,Full}MetadataCached
|
||||
// via the resolver's shared mirrors at opts.cacheDir.
|
||||
const lookupContext: PublishedAtLookupContext = {
|
||||
fetchOpts: opts.fetchOpts,
|
||||
getAuthHeaderValueByURI: opts.getAuthHeaderValueByURI,
|
||||
cacheDir: opts.cacheDir,
|
||||
cutoffMs: cutoff,
|
||||
abbreviatedMetaCache: new Map(),
|
||||
publishedAtCache: new Map(),
|
||||
localMetaCache: new Map(),
|
||||
fullMetaCache: new Map(),
|
||||
}
|
||||
|
||||
const minimumReleaseAge = opts.minimumReleaseAge
|
||||
@@ -119,9 +124,9 @@ export function createNpmResolutionVerifier (
|
||||
|
||||
const tarballUrl = (resolution as { tarball?: string }).tarball
|
||||
const registry = pickRegistryForVersion(opts.registries, namedRegistryPrefixes, name, tarballUrl)
|
||||
let time: Record<string, string | undefined> | undefined
|
||||
let published: string | undefined
|
||||
try {
|
||||
time = await fetchTimeMap(registry, name)
|
||||
published = await fetchPublishedAt(lookupContext, registry, name, version)
|
||||
} catch (err) {
|
||||
return {
|
||||
ok: false,
|
||||
@@ -129,12 +134,11 @@ export function createNpmResolutionVerifier (
|
||||
reason: uncheckable(err instanceof Error ? err.message : String(err)),
|
||||
}
|
||||
}
|
||||
const published = time?.[version]
|
||||
if (!published) {
|
||||
// Full metadata is missing this version — either an unpublish or the
|
||||
// registry doesn't expose per-version timestamps for it. Either way
|
||||
// the release-age can't be verified, so report a violation rather
|
||||
// than silently passing.
|
||||
// No source — attestation, local mirror, or full metadata —
|
||||
// surfaced a publish timestamp for this version. Either it's
|
||||
// unpublished or the registry doesn't expose per-version
|
||||
// timestamps. Report a violation rather than silently passing.
|
||||
return {
|
||||
ok: false,
|
||||
code: 'MINIMUM_RELEASE_AGE_VIOLATION',
|
||||
@@ -175,6 +179,192 @@ export function createNpmResolutionVerifier (
|
||||
}
|
||||
}
|
||||
|
||||
type PublishedAtTimeMap = Record<string, string | undefined>
|
||||
|
||||
interface PublishedAtLookupContext {
|
||||
fetchOpts: FetchMetadataFromFromRegistryOptions
|
||||
getAuthHeaderValueByURI: (registry: string) => string | undefined
|
||||
cacheDir?: string
|
||||
/**
|
||||
* The `minimumReleaseAge` cutoff converted to a unix-ms epoch. A
|
||||
* version with a publish time strictly less than this passes the
|
||||
* policy. Used by the abbreviated-metadata shortcut: if the
|
||||
* package's last-modified time is older than the cutoff, every
|
||||
* version it contains is too.
|
||||
*/
|
||||
cutoffMs: number
|
||||
/**
|
||||
* Per-(registry+name) memo of the abbreviated metadata fetch.
|
||||
* Abbreviated is what the resolver populates by default, so on a
|
||||
* non-frozen install the conditional GET hits the disk mirror at
|
||||
* ~zero cost. Resolves to the parsed metadata or `undefined` on
|
||||
* failure.
|
||||
*/
|
||||
abbreviatedMetaCache: Map<string, Promise<{ modified?: string } | undefined>>
|
||||
/**
|
||||
* Per-(registry+name+version) memo of the final published-at answer
|
||||
* the verifier hands to the policy check. One install verifies each
|
||||
* (name, version) pair at most once.
|
||||
*/
|
||||
publishedAtCache: Map<string, Promise<string | undefined>>
|
||||
/**
|
||||
* Per-(registry+name) memo of the on-disk full-metadata mirror read.
|
||||
* One disk read per package regardless of how many versions we
|
||||
* verify of it.
|
||||
*/
|
||||
localMetaCache: Map<string, Promise<PublishedAtTimeMap | undefined>>
|
||||
/**
|
||||
* Per-(registry+name) memo of the full-metadata network fetch — only
|
||||
* issued when both the abbreviated-modified shortcut and the
|
||||
* attestation endpoint fail to yield a timestamp.
|
||||
*/
|
||||
fullMetaCache: Map<string, Promise<PublishedAtTimeMap | undefined>>
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-(registry, name, version) lookup with a layered fallback:
|
||||
*
|
||||
* 1. **Abbreviated metadata `modified` shortcut.** This is what the
|
||||
* resolver already fetches by default; it's a small document with
|
||||
* a package-level last-modified time but no per-version timestamps.
|
||||
* If `modified` is older than the policy cutoff, every version in
|
||||
* this package was published at least that long ago — return the
|
||||
* `modified` timestamp as a conservative upper bound and skip the
|
||||
* rest of the chain. Costs one conditional GET that the resolver
|
||||
* has usually already paid for.
|
||||
* 2. **On-disk full-metadata mirror.** If a previous verification
|
||||
* populated `FULL_META_DIR`, take the per-version timestamp from
|
||||
* there.
|
||||
* 3. **npm attestation endpoint.** Small payload, just this version's
|
||||
* Sigstore-anchored timestamp. Wins on cold cache when the package
|
||||
* was published with provenance.
|
||||
* 4. **Full metadata fetch.** Last resort — only paid when the
|
||||
* abbreviated shortcut can't decide, the local full mirror is
|
||||
* cold, and there's no attestation.
|
||||
*/
|
||||
async function fetchPublishedAt (
|
||||
context: PublishedAtLookupContext,
|
||||
registry: string,
|
||||
name: string,
|
||||
version: string
|
||||
): Promise<string | undefined> {
|
||||
const cacheKey = `${registry}\x00${name}\x00${version}`
|
||||
let cachedPromise = context.publishedAtCache.get(cacheKey)
|
||||
if (cachedPromise == null) {
|
||||
cachedPromise = resolvePublishedAt(context, registry, name, version)
|
||||
context.publishedAtCache.set(cacheKey, cachedPromise)
|
||||
}
|
||||
return cachedPromise
|
||||
}
|
||||
|
||||
async function resolvePublishedAt (
|
||||
context: PublishedAtLookupContext,
|
||||
registry: string,
|
||||
name: string,
|
||||
version: string
|
||||
): Promise<string | undefined> {
|
||||
const abbreviatedShortcut = await tryAbbreviatedModifiedShortcut(context, registry, name)
|
||||
if (abbreviatedShortcut != null) return abbreviatedShortcut
|
||||
|
||||
const localTime = await readLocalMetaTime(context, registry, name)
|
||||
if (localTime?.[version]) return localTime[version]
|
||||
|
||||
const attestationTime = await fetchAttestationPublishedAt(context.fetchOpts, name, version, {
|
||||
registry,
|
||||
authHeaderValue: context.getAuthHeaderValueByURI(registry),
|
||||
})
|
||||
if (attestationTime != null) return attestationTime
|
||||
|
||||
const fullMetaTime = await fetchFullMetaTime(context, registry, name)
|
||||
return fullMetaTime?.[version]
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the abbreviated metadata's `modified` timestamp **iff** it
|
||||
* proves the gate would pass — i.e. modified is strictly older than
|
||||
* the policy cutoff. In that case every version this package contains
|
||||
* predates the cutoff, so the caller can short-circuit with `modified`
|
||||
* as a conservative upper-bound publish time.
|
||||
*
|
||||
* Returns `undefined` otherwise (modified is too recent, the metadata
|
||||
* lacks a parseable modified field, or the fetch failed) and the
|
||||
* caller proceeds with per-version lookups.
|
||||
*/
|
||||
async function tryAbbreviatedModifiedShortcut (
|
||||
context: PublishedAtLookupContext,
|
||||
registry: string,
|
||||
name: string
|
||||
): Promise<string | undefined> {
|
||||
const meta = await fetchAbbreviatedMeta(context, registry, name)
|
||||
const modified = meta?.modified
|
||||
if (typeof modified !== 'string') return undefined
|
||||
const modifiedMs = Date.parse(modified)
|
||||
if (Number.isNaN(modifiedMs)) return undefined
|
||||
if (modifiedMs >= context.cutoffMs) return undefined
|
||||
return modified
|
||||
}
|
||||
|
||||
function fetchAbbreviatedMeta (
|
||||
context: PublishedAtLookupContext,
|
||||
registry: string,
|
||||
name: string
|
||||
): Promise<{ modified?: string } | undefined> {
|
||||
const cacheKey = `${registry}\x00${name}`
|
||||
let cachedPromise = context.abbreviatedMetaCache.get(cacheKey)
|
||||
if (cachedPromise == null) {
|
||||
cachedPromise = fetchAbbreviatedMetadataCached(context.fetchOpts, name, {
|
||||
registry,
|
||||
authHeaderValue: context.getAuthHeaderValueByURI(registry),
|
||||
cacheDir: context.cacheDir,
|
||||
}).catch(() => undefined)
|
||||
context.abbreviatedMetaCache.set(cacheKey, cachedPromise)
|
||||
}
|
||||
return cachedPromise
|
||||
}
|
||||
|
||||
function readLocalMetaTime (
|
||||
context: PublishedAtLookupContext,
|
||||
registry: string,
|
||||
name: string
|
||||
): Promise<PublishedAtTimeMap | undefined> {
|
||||
if (!context.cacheDir) return Promise.resolve(undefined)
|
||||
const cacheKey = `${registry}\x00${name}`
|
||||
let cachedPromise = context.localMetaCache.get(cacheKey)
|
||||
if (cachedPromise == null) {
|
||||
cachedPromise = loadLocalMetaTime(context.cacheDir, registry, name)
|
||||
context.localMetaCache.set(cacheKey, cachedPromise)
|
||||
}
|
||||
return cachedPromise
|
||||
}
|
||||
|
||||
async function loadLocalMetaTime (
|
||||
cacheDir: string,
|
||||
registry: string,
|
||||
name: string
|
||||
): Promise<PublishedAtTimeMap | undefined> {
|
||||
const pkgMirror = getPkgMirrorPath(cacheDir, FULL_META_DIR, registry, name)
|
||||
const cached = await loadMeta(pkgMirror)
|
||||
return cached?.time
|
||||
}
|
||||
|
||||
function fetchFullMetaTime (
|
||||
context: PublishedAtLookupContext,
|
||||
registry: string,
|
||||
name: string
|
||||
): Promise<PublishedAtTimeMap | undefined> {
|
||||
const cacheKey = `${registry}\x00${name}`
|
||||
let cachedPromise = context.fullMetaCache.get(cacheKey)
|
||||
if (cachedPromise == null) {
|
||||
cachedPromise = fetchFullMetadataCached(context.fetchOpts, name, {
|
||||
registry,
|
||||
authHeaderValue: context.getAuthHeaderValueByURI(registry),
|
||||
cacheDir: context.cacheDir,
|
||||
}).then((meta) => meta.time)
|
||||
context.fullMetaCache.set(cacheKey, cachedPromise)
|
||||
}
|
||||
return cachedPromise
|
||||
}
|
||||
|
||||
function pickRegistryForVersion (
|
||||
registries: Registries,
|
||||
namedRegistryPrefixes: string[],
|
||||
|
||||
127
resolving/npm-resolver/src/fetchAttestationPublishedAt.ts
Normal file
127
resolving/npm-resolver/src/fetchAttestationPublishedAt.ts
Normal file
@@ -0,0 +1,127 @@
|
||||
import * as retry from '@zkochan/retry'
|
||||
|
||||
import type { FetchMetadataFromFromRegistryOptions } from './fetch.js'
|
||||
|
||||
/**
|
||||
* Per-version publish timestamp from npm's attestation endpoint —
|
||||
* `/-/npm/v1/attestations/<name>@<version>`.
|
||||
*
|
||||
* The response is a small JSON document containing one or more Sigstore
|
||||
* bundles. We read `bundle.verificationMaterial.tlogEntries[].integratedTime`
|
||||
* (the Rekor inclusion time) and surface it as an ISO date. This is a
|
||||
* couple of seconds after the actual publish — close enough for a
|
||||
* release-age policy that operates in minutes/hours/days.
|
||||
*
|
||||
* We deliberately do **not** verify the Sigstore signature here: the
|
||||
* trust model is identical to reading the registry's `time` field on
|
||||
* the full metadata document. The win is bandwidth — the attestation
|
||||
* payload is tens of KB versus the multi-MB full metadata document, so
|
||||
* cold-cache + `--frozen-lockfile` installs against a fleet of
|
||||
* provenance-published packages pay far less to verify timestamps.
|
||||
*
|
||||
* Returns `undefined` when:
|
||||
*
|
||||
* - The package has no published attestations (`404`).
|
||||
* - The response is malformed or missing the timestamp.
|
||||
* - The request itself fails (network error, registry 5xx).
|
||||
*
|
||||
* In all of those cases the caller falls back to fetching full metadata.
|
||||
*/
|
||||
export interface FetchAttestationOptions {
|
||||
registry: string
|
||||
authHeaderValue?: string
|
||||
}
|
||||
|
||||
export async function fetchAttestationPublishedAt (
|
||||
fetchOpts: FetchMetadataFromFromRegistryOptions,
|
||||
pkgName: string,
|
||||
version: string,
|
||||
opts: FetchAttestationOptions
|
||||
): Promise<string | undefined> {
|
||||
const url = `${opts.registry.replace(/\/$/, '')}/-/npm/v1/attestations/${pkgName}@${version}`
|
||||
const retryOperation = retry.operation(fetchOpts.retry)
|
||||
return new Promise<string | undefined>((resolve) => {
|
||||
retryOperation.attempt(async () => {
|
||||
let response: Response
|
||||
try {
|
||||
response = await fetchOpts.fetch(url, {
|
||||
authHeaderValue: opts.authHeaderValue,
|
||||
retry: fetchOpts.retry,
|
||||
timeout: fetchOpts.timeout,
|
||||
})
|
||||
} catch {
|
||||
// Network errors fall through to the full-metadata path; the
|
||||
// caller's `fetchFullMetadataCached` has its own retry policy.
|
||||
resolve(undefined)
|
||||
return
|
||||
}
|
||||
// 404 = package never published attestations. Other 4xx/5xx also
|
||||
// mean "can't get an answer from this endpoint, fall back."
|
||||
if (response.status >= 400) {
|
||||
resolve(undefined)
|
||||
return
|
||||
}
|
||||
let body: unknown
|
||||
try {
|
||||
body = await response.json()
|
||||
} catch {
|
||||
resolve(undefined)
|
||||
return
|
||||
}
|
||||
resolve(extractPublishedAt(body))
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Pull the earliest `integratedTime` across every attestation bundle in
|
||||
* the response and convert it to an ISO timestamp. Earliest is the
|
||||
* conservative choice: if two attestations disagree (e.g. publish
|
||||
* v0.1 vs SLSA provenance v1), we attribute the publish to the older
|
||||
* Rekor entry. The Rekor timestamp is what tells us when the artifact
|
||||
* existed in a transparency log — that's the floor on publish time.
|
||||
*/
|
||||
function extractPublishedAt (body: unknown): string | undefined {
|
||||
if (!body || typeof body !== 'object') return undefined
|
||||
const attestations = (body as { attestations?: unknown }).attestations
|
||||
if (!Array.isArray(attestations)) return undefined
|
||||
|
||||
let earliestSeconds: number | undefined
|
||||
for (const attestation of attestations) {
|
||||
const seconds = readEarliestIntegratedTime(attestation)
|
||||
if (seconds == null) continue
|
||||
if (earliestSeconds == null || seconds < earliestSeconds) {
|
||||
earliestSeconds = seconds
|
||||
}
|
||||
}
|
||||
if (earliestSeconds == null) return undefined
|
||||
return new Date(earliestSeconds * 1000).toISOString()
|
||||
}
|
||||
|
||||
function readEarliestIntegratedTime (attestation: unknown): number | undefined {
|
||||
if (!attestation || typeof attestation !== 'object') return undefined
|
||||
const bundle = (attestation as { bundle?: unknown }).bundle
|
||||
if (!bundle || typeof bundle !== 'object') return undefined
|
||||
const verificationMaterial = (bundle as { verificationMaterial?: unknown }).verificationMaterial
|
||||
if (!verificationMaterial || typeof verificationMaterial !== 'object') return undefined
|
||||
const tlogEntries = (verificationMaterial as { tlogEntries?: unknown }).tlogEntries
|
||||
if (!Array.isArray(tlogEntries)) return undefined
|
||||
|
||||
let earliest: number | undefined
|
||||
for (const entry of tlogEntries) {
|
||||
if (!entry || typeof entry !== 'object') continue
|
||||
const rawIntegratedTime = (entry as { integratedTime?: unknown }).integratedTime
|
||||
// npm serializes integratedTime as a string ("1778583836") to avoid
|
||||
// JSON precision loss; accept either string or number defensively.
|
||||
const seconds = parseIntegratedTimeSeconds(rawIntegratedTime)
|
||||
if (seconds == null) continue
|
||||
if (earliest == null || seconds < earliest) earliest = seconds
|
||||
}
|
||||
return earliest
|
||||
}
|
||||
|
||||
function parseIntegratedTimeSeconds (raw: unknown): number | undefined {
|
||||
const seconds = typeof raw === 'string' ? Number(raw) : typeof raw === 'number' ? raw : NaN
|
||||
if (!Number.isFinite(seconds) || seconds <= 0) return undefined
|
||||
return seconds
|
||||
}
|
||||
@@ -1,23 +1,24 @@
|
||||
import { FULL_META_DIR } from '@pnpm/constants'
|
||||
import { ABBREVIATED_META_DIR, FULL_META_DIR } from '@pnpm/constants'
|
||||
import { PnpmError } from '@pnpm/error'
|
||||
import type { PackageMeta } from '@pnpm/resolving.registry.types'
|
||||
|
||||
import { fetchMetadataFromFromRegistry, type FetchMetadataFromFromRegistryOptions } from './fetch.js'
|
||||
import { getPkgMirrorPath, loadMeta, loadMetaHeaders, prepareJsonForDisk, saveMeta } from './pickPackage.js'
|
||||
|
||||
export interface FetchFullMetadataCachedOptions {
|
||||
export interface FetchMetadataCachedOptions {
|
||||
registry: string
|
||||
authHeaderValue?: string
|
||||
/**
|
||||
* pnpm's on-disk cache directory. When set, the call issues a conditional
|
||||
* GET against the same `FULL_META_DIR` mirror the resolver populates: a
|
||||
* 304 Not Modified response serves the body from disk, a 200 writes the
|
||||
* new body back. Omit to disable caching — every call re-fetches the
|
||||
* full manifest.
|
||||
* GET against the matching mirror the resolver populates: a 304 Not
|
||||
* Modified response serves the body from disk, a 200 writes the new body
|
||||
* back. Omit to disable caching — every call re-fetches.
|
||||
*/
|
||||
cacheDir?: string
|
||||
}
|
||||
|
||||
export type FetchFullMetadataCachedOptions = FetchMetadataCachedOptions
|
||||
|
||||
/**
|
||||
* Fetch a full registry metadata document for `pkgName`, reusing pnpm's
|
||||
* shared on-disk metadata mirror when `cacheDir` is supplied. Built for the
|
||||
@@ -30,15 +31,40 @@ export async function fetchFullMetadataCached (
|
||||
fetchOpts: FetchMetadataFromFromRegistryOptions,
|
||||
pkgName: string,
|
||||
opts: FetchFullMetadataCachedOptions
|
||||
): Promise<PackageMeta> {
|
||||
return fetchMetadataCached(fetchOpts, pkgName, { ...opts, fullMetadata: true, metaDir: FULL_META_DIR })
|
||||
}
|
||||
|
||||
/**
|
||||
* Sibling of {@link fetchFullMetadataCached} that hits the abbreviated
|
||||
* metadata endpoint (`Accept: application/vnd.npm.install-v1+json`) and
|
||||
* caches under `ABBREVIATED_META_DIR` — the same mirror the resolver
|
||||
* populates by default. Used by the lockfile verification gate as a
|
||||
* cheap upper-bound check: if the package's `modified` field is older
|
||||
* than the policy cutoff, every version in it predates the cutoff and
|
||||
* no per-version timestamp lookup is needed.
|
||||
*/
|
||||
export async function fetchAbbreviatedMetadataCached (
|
||||
fetchOpts: FetchMetadataFromFromRegistryOptions,
|
||||
pkgName: string,
|
||||
opts: FetchMetadataCachedOptions
|
||||
): Promise<PackageMeta> {
|
||||
return fetchMetadataCached(fetchOpts, pkgName, { ...opts, fullMetadata: false, metaDir: ABBREVIATED_META_DIR })
|
||||
}
|
||||
|
||||
async function fetchMetadataCached (
|
||||
fetchOpts: FetchMetadataFromFromRegistryOptions,
|
||||
pkgName: string,
|
||||
opts: FetchMetadataCachedOptions & { fullMetadata: boolean, metaDir: string }
|
||||
): Promise<PackageMeta> {
|
||||
const pkgMirror = opts.cacheDir != null
|
||||
? getPkgMirrorPath(opts.cacheDir, FULL_META_DIR, opts.registry, pkgName)
|
||||
? getPkgMirrorPath(opts.cacheDir, opts.metaDir, opts.registry, pkgName)
|
||||
: null
|
||||
const cacheHeaders = pkgMirror != null ? await loadMetaHeaders(pkgMirror) : null
|
||||
const result = await fetchMetadataFromFromRegistry(fetchOpts, pkgName, {
|
||||
registry: opts.registry,
|
||||
authHeaderValue: opts.authHeaderValue,
|
||||
fullMetadata: true,
|
||||
fullMetadata: opts.fullMetadata,
|
||||
etag: cacheHeaders?.etag,
|
||||
modified: cacheHeaders?.modified,
|
||||
})
|
||||
|
||||
212
resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts
Normal file
212
resolving/npm-resolver/test/fetchAttestationPublishedAt.test.ts
Normal file
@@ -0,0 +1,212 @@
|
||||
import { describe, expect, test } from '@jest/globals'
|
||||
|
||||
import { fetchAttestationPublishedAt } from '../src/fetchAttestationPublishedAt.js'
|
||||
|
||||
type StubFetch = (url: string, opts?: unknown) => Promise<{
|
||||
status: number
|
||||
json: () => Promise<unknown>
|
||||
}>
|
||||
|
||||
function makeFetchOpts (fetch: StubFetch) {
|
||||
return {
|
||||
fetch: fetch as never,
|
||||
retry: { retries: 0 },
|
||||
timeout: 10_000,
|
||||
fetchWarnTimeoutMs: 10_000,
|
||||
}
|
||||
}
|
||||
|
||||
const REGISTRY = 'https://registry.npmjs.org/'
|
||||
|
||||
function attestationResponse (...integratedTimes: Array<string | number>): unknown {
|
||||
return {
|
||||
attestations: integratedTimes.map((t) => ({
|
||||
predicateType: 'https://github.com/npm/attestation/tree/main/specs/publish/v0.1',
|
||||
bundle: {
|
||||
verificationMaterial: {
|
||||
tlogEntries: [{ integratedTime: t }],
|
||||
},
|
||||
},
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
describe('fetchAttestationPublishedAt', () => {
|
||||
test('returns an ISO timestamp built from tlogEntries[].integratedTime', async () => {
|
||||
// 1778583836 (Unix seconds) → 2026-05-12T05:43:56.000Z
|
||||
const fetch: StubFetch = async () => ({
|
||||
status: 200,
|
||||
json: async () => attestationResponse('1778583836'),
|
||||
})
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBe(new Date(1778583836 * 1000).toISOString())
|
||||
})
|
||||
|
||||
test('hits /-/npm/v1/attestations/<name>@<version> with a literal name@version spec', async () => {
|
||||
const seenUrls: string[] = []
|
||||
const fetch: StubFetch = async (url) => {
|
||||
seenUrls.push(url)
|
||||
return { status: 404, json: async () => null }
|
||||
}
|
||||
await fetchAttestationPublishedAt(makeFetchOpts(fetch), 'pnpm', '11.1.1', { registry: REGISTRY })
|
||||
expect(seenUrls).toEqual(['https://registry.npmjs.org/-/npm/v1/attestations/pnpm@11.1.1'])
|
||||
})
|
||||
|
||||
test('scoped package name passes through unencoded (slashes are path separators)', async () => {
|
||||
const seenUrls: string[] = []
|
||||
const fetch: StubFetch = async (url) => {
|
||||
seenUrls.push(url)
|
||||
return { status: 404, json: async () => null }
|
||||
}
|
||||
await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'@pnpm/exe',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(seenUrls).toEqual(['https://registry.npmjs.org/-/npm/v1/attestations/@pnpm/exe@11.1.1'])
|
||||
})
|
||||
|
||||
test('returns undefined when the registry has no attestations for the package (404)', async () => {
|
||||
const fetch: StubFetch = async () => ({ status: 404, json: async () => null })
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBeUndefined()
|
||||
})
|
||||
|
||||
test('returns undefined on 5xx — caller falls back to full metadata', async () => {
|
||||
const fetch: StubFetch = async () => ({ status: 503, json: async () => null })
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBeUndefined()
|
||||
})
|
||||
|
||||
test('returns undefined when the fetch itself throws (network error)', async () => {
|
||||
const fetch: StubFetch = async () => {
|
||||
throw new Error('ECONNREFUSED')
|
||||
}
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBeUndefined()
|
||||
})
|
||||
|
||||
test('returns undefined when the body is malformed JSON', async () => {
|
||||
const fetch: StubFetch = async () => ({
|
||||
status: 200,
|
||||
json: async () => {
|
||||
throw new SyntaxError('bad')
|
||||
},
|
||||
})
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBeUndefined()
|
||||
})
|
||||
|
||||
test('returns undefined when the response has no attestations array', async () => {
|
||||
const fetch: StubFetch = async () => ({ status: 200, json: async () => ({}) })
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBeUndefined()
|
||||
})
|
||||
|
||||
test('returns undefined when no tlogEntry carries a usable integratedTime', async () => {
|
||||
const fetch: StubFetch = async () => ({
|
||||
status: 200,
|
||||
json: async () => ({
|
||||
attestations: [{ bundle: { verificationMaterial: { tlogEntries: [{}] } } }],
|
||||
}),
|
||||
})
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBeUndefined()
|
||||
})
|
||||
|
||||
test('picks the earliest integratedTime across multiple attestations', async () => {
|
||||
// SLSA provenance is signed slightly before npm publish attestation;
|
||||
// earlier integratedTime is the conservative pick.
|
||||
const fetch: StubFetch = async () => ({
|
||||
status: 200,
|
||||
json: async () => attestationResponse('1778583836', '1778583833'),
|
||||
})
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBe(new Date(1778583833 * 1000).toISOString())
|
||||
})
|
||||
|
||||
test('accepts integratedTime as a number too (defensive against schema drift)', async () => {
|
||||
const fetch: StubFetch = async () => ({
|
||||
status: 200,
|
||||
json: async () => attestationResponse(1778583836),
|
||||
})
|
||||
const result = await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY }
|
||||
)
|
||||
expect(result).toBe(new Date(1778583836 * 1000).toISOString())
|
||||
})
|
||||
|
||||
test('forwards the auth header to the fetch call', async () => {
|
||||
let seenAuth: string | undefined
|
||||
const fetch: StubFetch = async (_url, opts) => {
|
||||
seenAuth = (opts as { authHeaderValue?: string } | undefined)?.authHeaderValue
|
||||
return { status: 404, json: async () => null }
|
||||
}
|
||||
await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: REGISTRY, authHeaderValue: 'Bearer secret' }
|
||||
)
|
||||
expect(seenAuth).toBe('Bearer secret')
|
||||
})
|
||||
|
||||
test('strips a trailing slash on the registry URL', async () => {
|
||||
const seenUrls: string[] = []
|
||||
const fetch: StubFetch = async (url) => {
|
||||
seenUrls.push(url)
|
||||
return { status: 404, json: async () => null }
|
||||
}
|
||||
await fetchAttestationPublishedAt(
|
||||
makeFetchOpts(fetch),
|
||||
'pnpm',
|
||||
'11.1.1',
|
||||
{ registry: 'https://registry.npmjs.org/' }
|
||||
)
|
||||
expect(seenUrls[0]).toBe('https://registry.npmjs.org/-/npm/v1/attestations/pnpm@11.1.1')
|
||||
})
|
||||
})
|
||||
Reference in New Issue
Block a user