ci(merge): source per-arch digests from ci-cache, not local-ai-backends

Follow-up to PR #9781. v4.2.2 (run 25745181433) showed the keepalive
anchor in ci-cache wasn't enough on its own: 19 of 37 multiarch merges
still failed with "manifest not found" for the same digests we'd just
anchored.

Quay's manifest GC is per-repository. The anchor tag in ci-cache
protects the manifest copy that lives in ci-cache, but the same digest
in local-ai-backends is independently tracked and gets reaped because
nothing in local-ai-backends references it (push-by-digest=true leaves
it untagged). The merge then asks
`local-ai-backends@sha256:<digest>` and quay correctly says "not found"
in that repo, even though `ci-cache@sha256:<digest>` is alive and well.

Empirical confirmation against a live failed digest from v4.2.2:

  $ docker buildx imagetools inspect quay.io/go-skynet/ci-cache@sha256:05377fe6...
  Name: quay.io/go-skynet/ci-cache@sha256:05377fe6...
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  $ docker buildx imagetools inspect quay.io/go-skynet/local-ai-backends@sha256:05377fe6...
  ERROR: ... not found

Switch the source of the quay merge step to ci-cache. The blobs the
manifest references are already accessible from local-ai-backends
(verified via direct registry HEAD: HTTP 200 from both repos — the
original push cross-mounted blobs at content-addressable storage time
and they outlive the per-repo manifest GC). buildx imagetools create
republishes the manifest into local-ai-backends, then writes the
user-facing manifest list pointing at it. End state is self-contained:
the published manifest list references child manifests by digest only,
no embedded reference to ci-cache.

Dockerhub merge step is unchanged. Dockerhub's GC isn't aggressive
enough to reap untagged manifests at the timescales we operate on
(verified: localai/localai-backends@<same digest> still resolves cleanly
after >24h).

Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
2026-05-12 22:35:55 +00:00
parent bc4cd3dd85
commit 8521af145f

View File

@@ -88,6 +88,25 @@ jobs:
latest=${{ inputs.tag-latest }}
suffix=${{ inputs.tag-suffix }},onlatest=true
# Source from ci-cache, not local-ai-backends.
#
# The build job pushes per-arch manifests to local-ai-backends with
# push-by-digest=true (no tag), then anchors a tagged copy into
# ci-cache so the manifest can be retrieved hours later when this
# merge runs. Quay's manifest GC, however, is per-repository: the
# anchor tag in ci-cache protects the manifest there, but the same
# digest in local-ai-backends has no tag in *that* repo and gets
# reaped independently. Sourcing local-ai-backends@<digest> here
# then fails with "manifest not found" — exactly the regression
# we hit on v4.2.2 (19/37 multiarch merges failed).
#
# ci-cache@<digest> resolves because we anchored it there. buildx
# imagetools create copies the manifest into local-ai-backends
# (cross-repo within the same registry, blobs already cross-mounted
# from the original push so no transfer needed) and publishes the
# manifest list with the user-facing tags. The resulting manifest
# list is fully self-contained in local-ai-backends — child digests
# only, no embedded references to ci-cache.
- name: Create manifest list and push (quay)
if: github.event_name != 'pull_request'
working-directory: /tmp/digests
@@ -104,7 +123,7 @@ jobs:
else
# shellcheck disable=SC2086
docker buildx imagetools create $tags \
$(printf 'quay.io/go-skynet/local-ai-backends@sha256:%s ' *)
$(printf 'quay.io/go-skynet/ci-cache@sha256:%s ' *)
fi
- name: Create manifest list and push (dockerhub)