Files
pnpm/justfile
Zoltan Kochan 3610b85e30 feat(pnpr): serve container images (#14629)
Adds OCI as a fourth ecosystem beside npm, Cargo, and Python: a hosted
image registry that docker, podman, and skopeo push to and pull from.

The distribution API cannot be moved under a path prefix. A client
derives the API root from the image reference's host, so
`pnpr.example.com/acme/app:1.0` always requests
`/v2/acme/app/manifests/1.0`. `/v2/` therefore mounts at the host root
whatever else is served, and the repository name alone selects the
registry through the declared-provenance rules the other surfaces use.
Artifactory has to burn the first path segment as a repository key for
want of that invariant; pnpr does not, so the image name stays the image
name. `/oci/v2/` and `/oci/~<name>/v2/` are served too, for podman and
containerd, whose registry configuration does accept a path.

Repository names are `/`-joined, which the pattern language had no shape
for. `PackagePattern::parse` now takes the ecosystem and offers each one
only the wildcard its names can carry: npm keeps `@scope/*` and `@*/*`,
images get `<namespace>/*` over a single leading component, and Cargo
and PyPI get neither, since a flat name could never match one. A
`packages:` key is normalized through that language rather than through
the name rules, so a wildcard key parses as itself.

Blob uploads stream to a local file across requests instead of buffering
in memory, and are verified against the promised digest before anything
is stored. The manifest write is the commit point and rides the existing
publish journal; blobs land outside it, content-addressed and invisible
until a manifest names them, which is what leaves unreferenced blobs for
a collector rather than half-publishing a release. A tag carries the
time it last moved, so a transaction recovered after a crash cannot drag
one back to an older manifest.

`Basic` credentials now carry a token as the password under any
username, which is how `docker login` sends them, and `GET /v2/`
challenges an anonymous caller even where reads are open: a client
settles its authentication scheme on that one response, so a 200 would
leave it no way to authenticate a push.

Verified end to end against docker 29.7.2 (login, push, pull, run),
podman 5.8.4, and skopeo 1.22.2, plus an anonymous pull and a push
refused after logout. The two client stacks take different upload paths
and between them cover both: moby sends a blob monolithically, while
containers/image chunks it.

Not yet served: proxying an upstream image registry, blob collection,
the referrers API, cross-repository blob mounts, and ranged blob
downloads.
2026-09-07 03:49:35 +02:00

146 lines
5.8 KiB
Makefile

#!/usr/bin/env -S just --justfile
_default:
just --list -u
alias r := ready
alias c := codecov
alias t := test
# Initialize the project by installing all the necessary tools.
# Make sure you have cargo-binstall installed.
# You can download the pre-compiled binary from <https://github.com/cargo-bins/cargo-binstall#installation>
# or install via `cargo install cargo-binstall`
init:
cargo binstall cargo-nextest cargo-watch cargo-insta typos-cli taplo-cli wasm-pack cargo-llvm-cov sccache@0.17.0 -y
# `cargo-fixit` has no prebuilt binaries, so install it from source
# with `cargo install` (pinned) instead of `cargo binstall`.
cargo install cargo-fixit@0.1.15 --locked
# When ready, run the same CI commands
ready:
typos pnpm pnpr
cargo fmt
just check
just test
just lint
git status
# Update our local branch with the remote branch (this is for you to sync the submodules)
update:
git pull
git submodule update --init
# Install necessary dependencies.
# `pnpm/tasks/registry-mock` is a member of the root pnpm workspace,
# so the root install populates its node_modules.
install:
pnpm install --frozen-lockfile --prefer-offline
# Run `cargo watch`
# --no-vcs-ignores: cargo-watch has a bug loading all .gitignores, including the ones listed in .gitignore
# use .ignore file getting the ignore list
watch command:
cargo watch --no-vcs-ignores -x '{{command}}'
# Format all files
fmt:
cargo fmt
taplo format
# Run cargo check
check:
cargo check --locked --workspace --all-targets
# Run all the tests.
test:
node pnpm/scripts/run-rust-tests.mjs
# A test process that is killed cannot run `TempDir`'s cleanup, so a
# fail-fast or interrupted run abandons whole fixture trees — each holding a
# per-test store for the mocked-registry tests, which is what actually adds
# up. Only `pacquet-test-*` is swept: that prefix comes from
# `CommandTempCwd`, so a match is known to be ours. `-mindepth 1` keeps the
# root itself out of the match, and the age floor leaves a concurrent run
# alone.
# Remove fixture trees that earlier test runs abandoned.
sweep-test-temp:
find "${TMPDIR:-/tmp}" -mindepth 1 -maxdepth 1 -name 'pacquet-test-*' -mmin +60 -exec rm -rf {} + 2>/dev/null || true
# Run pacquet package tests only.
test-pacquet:
node pnpm/scripts/run-rust-tests.mjs --workspace --exclude pnpr --exclude pnpr-auth --exclude pnpr-cargo --exclude pnpr-config --exclude pnpr-error --exclude pnpr-fixtures --exclude pnpr-oci --exclude pnpr-osv --exclude pnpr-package-name --exclude pnpr-pipeline-runs --exclude pnpr-policy --exclude pnpr-pypi --exclude pnpr-registry --exclude pnpr-route --exclude pnpr-search --exclude pnpr-shared-artifacts --exclude pnpr-storage --exclude pnpr-upstream
# Run pnpr package tests only.
test-pnpr:
# Every `pnpr-*` crate, selected together so cargo's feature unification
# gives them the same backend features `pnpr` itself defaults to — selecting
# one alone would build it bare and silently skip its backend tests.
cargo nextest run -p pnpr -p pnpr-auth -p pnpr-cargo -p pnpr-config -p pnpr-error -p pnpr-fixtures -p pnpr-oci -p pnpr-osv -p pnpr-package-name -p pnpr-pipeline-runs -p pnpr-policy -p pnpr-pypi -p pnpr-registry -p pnpr-route -p pnpr-search -p pnpr-shared-artifacts -p pnpr-storage -p pnpr-upstream
# List expected-failing test ports
[unix]
known-failures:
@cargo test --workspace known_failures -- --list 2>/dev/null | rg '^known_failures::'
[windows]
known-failures:
@cargo test --workspace known_failures -- --list 2>nul | rg '^known_failures::'
# Lint the whole project
lint:
cargo clippy --locked --workspace --all-targets -- --deny warnings
# Apply clippy's autofix suggestions across the workspace.
# Uses `cargo fixit --clippy` (installed by `just init`, pinned to
# `cargo-fixit@0.1.15`) instead of `cargo clippy --fix`. `cargo fixit`
# is faster than `cargo clippy --fix` on repeated runs because it skips
# the full re-check compile between fix rounds, so iterating on a lint
# cleanup doesn't rebuild the workspace each pass.
fix:
cargo fixit --clippy --workspace --all-targets --allow-dirty --allow-staged
# Run perfectionist dylint rules. Requires `cargo-dylint` and `dylint-link`
# (install from source with `cargo install cargo-dylint dylint-link`; the
# prebuilt binstall binaries fail to build the driver locally). The lint
# library is pinned in `dylint.toml`.
dylint:
env RUSTFLAGS="-D warnings" cargo dylint --all -- --all-targets --workspace
# Get code coverage
codecov:
cargo codecov --html
# Run the benchmarks. See `tasks/benchmark`
micro-benchmark:
cargo run --bin=micro-benchmark --release
# Compare Rust artifact reuse between two disposable worktrees.
bench-rust-cache *args:
node pnpm/scripts/bench-rust-cache.mjs {{args}}
# Manage registry-mock. The launcher spawns `pnpr`; on
# Windows you can't overwrite a running .exe, so we pre-build all
# the test artifacts a subsequent `just test` will need with the
# exact same invocation. A `-p pnpr`-scoped pre-build is
# not enough — workspace-wide feature unification gives a
# different fingerprint and nextest would still try to re-link the
# running binary, failing with `os error 5` on Windows MSVC.
registry-mock +args:
cargo nextest run --no-run
cargo run --bin=pnpm-registry-mock -- {{args}}
# The benchmark may auto-spawn the registry mock (via
# `AutoMockInstance::load_or_init()`), so make sure `pnpr`
# is built before the executor runs — otherwise the spawn step
# aborts with "binary not found". Built with `--release` so the
# mock serves at optimized perf; a debug build would put the
# Rust mock at a multi-second handicap vs verdaccio, which V8
# always JITs, polluting the install-perf signal.
integrated-benchmark +args:
cargo build --release --bin=pnpr
cargo run --bin=integrated-benchmark -- {{args}}
cli +args:
cargo run --bin pnpm -- {{args}}