Now that base-images.yml's master-push trigger includes the install
script and apt-mirror script (commit 7fff8584), enumerate them in
the workflow-surfaces table instead of the vaguer "base Dockerfile/
workflow" phrasing.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The migration shipped over a sequence of PRs (#9726 → #9727 → #9730 →
#9731 → #9737 → #9738 plus a handful of direct-to-master fixes) and
left the .agents/ docs significantly out of date.
Updated:
- .agents/ci-caching.md (significant rewrite)
- Cache key shape: now includes per-arch suffix (cache<suffix>-<arch>).
- New "Workflow surfaces" overview table.
- New "Pre-built base images (base-grpc-*)" section covering the 10
quay.io/go-skynet/ci-cache:base-grpc-* tags, the multi-target
Dockerfile pattern (builder-fromsource / builder-prebuilt /
aliasing FROM), the BUILDER_BASE_IMAGE → BUILDER_TARGET derivation,
the bootstrap-on-branch order for new variants.
- New "Per-arch native builds + manifest merge" section: split
matrix entries, push-by-digest, backend_merge.yml, why
provenance: false matters.
- New "Path filter on master push" section: changed-backends.js
handles push events via the Compare API; weekly Sunday cron is
the safety net for unpinned Python deps.
- New "ccache for C++ backend builds" section.
- New "Composite actions" section: free-disk-space and
setup-build-disk.
- New "Concurrency" section documenting the per-PR-per-commit group
fix.
- Darwin section gains the brew link --overwrite note (after-
cache-restore symlinks weren't restored) and the llama-cpp-darwin
consolidation context.
- "Self-hosted runners" section confirming the matrix is free of
arc-runner-set / bigger-runner references except the residual
test-extra.yml vibevoice case.
- "Touching the cache pipeline" rule list extended (provenance,
install-base-deps.sh single-source-of-truth, base-images bootstrap
order).
- .agents/adding-backends.md
- Section 2 title: backend.yml -> backend-matrix.yml (path moved).
- New paragraph on per-arch entries (platform-tag + paired matrix
rows + auto-firing merge job).
- New paragraph on builder-base-image for llama-cpp / ik-llama-cpp /
turboquant.
- Final checklist line updated accordingly.
- .agents/building-and-testing.md
- Reference: backend.yml -> backend-matrix.yml.
- Note about builder-base-image and BUILDER_TARGET defaulting to
builder-fromsource for local builds.
- AGENTS.md
- One-line description update for ci-caching.md to mention the new
infrastructure (per-arch keys, base-grpc-*, manifest-merge,
setup-build-disk, path filter).
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
macOS runners can't use the registry-backed BuildKit cache (no Docker
daemon), so every darwin matrix run was paying full cost for brew
installs, Go module downloads, llama.cpp recompiles and Python wheel
resolution.
Wires actions/cache@v4 into the reusable workflow for four caches:
- Go modules + build cache (setup-go cache: true), shared across matrix
- Homebrew downloads + selected /opt/homebrew/Cellar entries, with
HOMEBREW_NO_AUTO_UPDATE so restored Cellar paths stay stable
- ccache for the llama-cpp CMake variants, keyed on the pinned
LLAMA_VERSION; CMAKE_*_COMPILER_LAUNCHER is exported via GITHUB_ENV
so backend/cpp/llama-cpp/Makefile picks it up without script changes
- Python uv + pip wheel cache, keyed by backend + ISO week — same
one-cold-rebuild-per-week cadence as the Linux DEPS_REFRESH
Read/write semantics match the existing BuildKit policy: every run
restores, only master/tag pushes save, so PRs can't pollute master's
warm cache.
Documents the new caches and the macOS-specific constraints in
.agents/ci-caching.md.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-7[1m] [Claude Code]
The shared backend/Dockerfile.python ends in:
RUN cd /${BACKEND} && PORTABLE_PYTHON=true make
which `pip install`s each backend's requirements*.txt. A scan of all 34
Python backends shows every single one ships at least some unpinned deps
(torch, transformers, vllm, diffusers, ...). With the registry cache now
enabled, that `make` layer's BuildKit hash depends only on Dockerfile
instructions + COPYed source — not on what pip resolves at runtime — so
a warm cache would freeze upstream versions indefinitely.
DEPS_REFRESH is an ARG declared right before that RUN. backend_build.yml
computes `date -u +%Y-W%V` (ISO week, e.g. `2026-W17`) and passes it as
a build-arg, so the install layer invalidates at most once per week and
re-resolves PyPI / nightly indexes. Within a week, builds stay warm.
Only Dockerfile.python is affected: Go (go.sum) and Rust (Cargo.lock)
already lock their deps, and the C++ backends pull gRPC at a pinned tag
and llama.cpp at a pinned commit.
Add .agents/ci-caching.md documenting the cache layout
(quay.io/go-skynet/ci-cache:cache<tag-suffix>), read/write semantics
(master writes, PRs read-only), DEPS_REFRESH semantics, and how to
manually evict tags. Index it from AGENTS.md (CLAUDE.md is a symlink).
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:claude-opus-4-7-1m
