Adds OCI as a fourth ecosystem beside npm, Cargo, and Python: a hosted image registry that docker, podman, and skopeo push to and pull from. The distribution API cannot be moved under a path prefix. A client derives the API root from the image reference's host, so `pnpr.example.com/acme/app:1.0` always requests `/v2/acme/app/manifests/1.0`. `/v2/` therefore mounts at the host root whatever else is served, and the repository name alone selects the registry through the declared-provenance rules the other surfaces use. Artifactory has to burn the first path segment as a repository key for want of that invariant; pnpr does not, so the image name stays the image name. `/oci/v2/` and `/oci/~<name>/v2/` are served too, for podman and containerd, whose registry configuration does accept a path. Repository names are `/`-joined, which the pattern language had no shape for. `PackagePattern::parse` now takes the ecosystem and offers each one only the wildcard its names can carry: npm keeps `@scope/*` and `@*/*`, images get `<namespace>/*` over a single leading component, and Cargo and PyPI get neither, since a flat name could never match one. A `packages:` key is normalized through that language rather than through the name rules, so a wildcard key parses as itself. Blob uploads stream to a local file across requests instead of buffering in memory, and are verified against the promised digest before anything is stored. The manifest write is the commit point and rides the existing publish journal; blobs land outside it, content-addressed and invisible until a manifest names them, which is what leaves unreferenced blobs for a collector rather than half-publishing a release. A tag carries the time it last moved, so a transaction recovered after a crash cannot drag one back to an older manifest. `Basic` credentials now carry a token as the password under any username, which is how `docker login` sends them, and `GET /v2/` challenges an anonymous caller even where reads are open: a client settles its authentication scheme on that one response, so a 200 would leave it no way to authenticate a push. Verified end to end against docker 29.7.2 (login, push, pull, run), podman 5.8.4, and skopeo 1.22.2, plus an anonymous pull and a push refused after logout. The two client stacks take different upload paths and between them cover both: moby sends a blob monolithically, while containers/image chunks it. Not yet served: proxying an upstream image registry, blob collection, the referrers API, cross-repository blob mounts, and ranged blob downloads.
146 lines
5.8 KiB
Makefile
146 lines
5.8 KiB
Makefile
#!/usr/bin/env -S just --justfile
|
|
|
|
_default:
|
|
just --list -u
|
|
|
|
alias r := ready
|
|
alias c := codecov
|
|
alias t := test
|
|
|
|
# Initialize the project by installing all the necessary tools.
|
|
# Make sure you have cargo-binstall installed.
|
|
# You can download the pre-compiled binary from <https://github.com/cargo-bins/cargo-binstall#installation>
|
|
# or install via `cargo install cargo-binstall`
|
|
init:
|
|
cargo binstall cargo-nextest cargo-watch cargo-insta typos-cli taplo-cli wasm-pack cargo-llvm-cov sccache@0.17.0 -y
|
|
# `cargo-fixit` has no prebuilt binaries, so install it from source
|
|
# with `cargo install` (pinned) instead of `cargo binstall`.
|
|
cargo install cargo-fixit@0.1.15 --locked
|
|
|
|
# When ready, run the same CI commands
|
|
ready:
|
|
typos pnpm pnpr
|
|
cargo fmt
|
|
just check
|
|
just test
|
|
just lint
|
|
git status
|
|
|
|
# Update our local branch with the remote branch (this is for you to sync the submodules)
|
|
update:
|
|
git pull
|
|
git submodule update --init
|
|
|
|
# Install necessary dependencies.
|
|
# `pnpm/tasks/registry-mock` is a member of the root pnpm workspace,
|
|
# so the root install populates its node_modules.
|
|
install:
|
|
pnpm install --frozen-lockfile --prefer-offline
|
|
|
|
# Run `cargo watch`
|
|
# --no-vcs-ignores: cargo-watch has a bug loading all .gitignores, including the ones listed in .gitignore
|
|
# use .ignore file getting the ignore list
|
|
watch command:
|
|
cargo watch --no-vcs-ignores -x '{{command}}'
|
|
|
|
# Format all files
|
|
fmt:
|
|
cargo fmt
|
|
taplo format
|
|
|
|
# Run cargo check
|
|
check:
|
|
cargo check --locked --workspace --all-targets
|
|
|
|
# Run all the tests.
|
|
test:
|
|
node pnpm/scripts/run-rust-tests.mjs
|
|
|
|
# A test process that is killed cannot run `TempDir`'s cleanup, so a
|
|
# fail-fast or interrupted run abandons whole fixture trees — each holding a
|
|
# per-test store for the mocked-registry tests, which is what actually adds
|
|
# up. Only `pacquet-test-*` is swept: that prefix comes from
|
|
# `CommandTempCwd`, so a match is known to be ours. `-mindepth 1` keeps the
|
|
# root itself out of the match, and the age floor leaves a concurrent run
|
|
# alone.
|
|
|
|
# Remove fixture trees that earlier test runs abandoned.
|
|
sweep-test-temp:
|
|
find "${TMPDIR:-/tmp}" -mindepth 1 -maxdepth 1 -name 'pacquet-test-*' -mmin +60 -exec rm -rf {} + 2>/dev/null || true
|
|
|
|
# Run pacquet package tests only.
|
|
test-pacquet:
|
|
node pnpm/scripts/run-rust-tests.mjs --workspace --exclude pnpr --exclude pnpr-auth --exclude pnpr-cargo --exclude pnpr-config --exclude pnpr-error --exclude pnpr-fixtures --exclude pnpr-oci --exclude pnpr-osv --exclude pnpr-package-name --exclude pnpr-pipeline-runs --exclude pnpr-policy --exclude pnpr-pypi --exclude pnpr-registry --exclude pnpr-route --exclude pnpr-search --exclude pnpr-shared-artifacts --exclude pnpr-storage --exclude pnpr-upstream
|
|
|
|
# Run pnpr package tests only.
|
|
test-pnpr:
|
|
# Every `pnpr-*` crate, selected together so cargo's feature unification
|
|
# gives them the same backend features `pnpr` itself defaults to — selecting
|
|
# one alone would build it bare and silently skip its backend tests.
|
|
cargo nextest run -p pnpr -p pnpr-auth -p pnpr-cargo -p pnpr-config -p pnpr-error -p pnpr-fixtures -p pnpr-oci -p pnpr-osv -p pnpr-package-name -p pnpr-pipeline-runs -p pnpr-policy -p pnpr-pypi -p pnpr-registry -p pnpr-route -p pnpr-search -p pnpr-shared-artifacts -p pnpr-storage -p pnpr-upstream
|
|
|
|
# List expected-failing test ports
|
|
[unix]
|
|
known-failures:
|
|
@cargo test --workspace known_failures -- --list 2>/dev/null | rg '^known_failures::'
|
|
|
|
[windows]
|
|
known-failures:
|
|
@cargo test --workspace known_failures -- --list 2>nul | rg '^known_failures::'
|
|
# Lint the whole project
|
|
lint:
|
|
cargo clippy --locked --workspace --all-targets -- --deny warnings
|
|
|
|
# Apply clippy's autofix suggestions across the workspace.
|
|
# Uses `cargo fixit --clippy` (installed by `just init`, pinned to
|
|
# `cargo-fixit@0.1.15`) instead of `cargo clippy --fix`. `cargo fixit`
|
|
# is faster than `cargo clippy --fix` on repeated runs because it skips
|
|
# the full re-check compile between fix rounds, so iterating on a lint
|
|
# cleanup doesn't rebuild the workspace each pass.
|
|
fix:
|
|
cargo fixit --clippy --workspace --all-targets --allow-dirty --allow-staged
|
|
|
|
# Run perfectionist dylint rules. Requires `cargo-dylint` and `dylint-link`
|
|
# (install from source with `cargo install cargo-dylint dylint-link`; the
|
|
# prebuilt binstall binaries fail to build the driver locally). The lint
|
|
# library is pinned in `dylint.toml`.
|
|
dylint:
|
|
env RUSTFLAGS="-D warnings" cargo dylint --all -- --all-targets --workspace
|
|
|
|
# Get code coverage
|
|
codecov:
|
|
cargo codecov --html
|
|
|
|
# Run the benchmarks. See `tasks/benchmark`
|
|
micro-benchmark:
|
|
cargo run --bin=micro-benchmark --release
|
|
|
|
# Compare Rust artifact reuse between two disposable worktrees.
|
|
bench-rust-cache *args:
|
|
node pnpm/scripts/bench-rust-cache.mjs {{args}}
|
|
|
|
# Manage registry-mock. The launcher spawns `pnpr`; on
|
|
# Windows you can't overwrite a running .exe, so we pre-build all
|
|
# the test artifacts a subsequent `just test` will need with the
|
|
# exact same invocation. A `-p pnpr`-scoped pre-build is
|
|
# not enough — workspace-wide feature unification gives a
|
|
# different fingerprint and nextest would still try to re-link the
|
|
# running binary, failing with `os error 5` on Windows MSVC.
|
|
registry-mock +args:
|
|
cargo nextest run --no-run
|
|
cargo run --bin=pnpm-registry-mock -- {{args}}
|
|
|
|
# The benchmark may auto-spawn the registry mock (via
|
|
# `AutoMockInstance::load_or_init()`), so make sure `pnpr`
|
|
# is built before the executor runs — otherwise the spawn step
|
|
# aborts with "binary not found". Built with `--release` so the
|
|
# mock serves at optimized perf; a debug build would put the
|
|
# Rust mock at a multi-second handicap vs verdaccio, which V8
|
|
# always JITs, polluting the install-perf signal.
|
|
integrated-benchmark +args:
|
|
cargo build --release --bin=pnpr
|
|
cargo run --bin=integrated-benchmark -- {{args}}
|
|
|
|
cli +args:
|
|
cargo run --bin pnpm -- {{args}}
|