fix(pii): wrap reversible redaction tokens

Use configurable token delimiters to avoid restoring ordinary model text that happens to match an internal identifier. Rename the option and document the confidentiality tradeoff.

Assisted-by: Codex:gpt-5

.agents/adding-backends.md

@@ -34,7 +34,7 @@ The build matrix is data-only YAML at `.github/backend-matrix.yml` (not inside `
 
 **Without an entry here no image is ever built or pushed, and the gallery entry in `backend/index.yaml` will point at a tag that does not exist.** The `dockerfile:` field must point at `./backend/Dockerfile.<lang>` matching the language bucket from step 1 (e.g. `Dockerfile.python`, `Dockerfile.golang`, `Dockerfile.rust`). The `tag-suffix` must match the `uri:` in the corresponding `backend/index.yaml` image entry exactly.
 
-**`scripts/changed-backends.js` registration — REQUIRED for any new dockerfile suffix.** This is the single most common omission, because it has no effect on the PR that adds the backend (when no prior path filter could catch it anyway) — it only breaks the *next* PR that touches your backend's directory, which then gets zero CI jobs and looks broken for unrelated reasons. Edit `scripts/changed-backends.js:inferBackendPath` and add a branch BEFORE the more-generic suffixes:
+**Path-filter registration — REQUIRED for any new dockerfile suffix.** This is the single most common omission, because it has no effect on the PR that adds the backend (when no prior path filter could catch it anyway) — it only breaks the *next* PR that touches your backend's directory, which then gets zero CI jobs and looks broken for unrelated reasons. Edit `scripts/lib/backend-filter.mjs:inferBackendPath` and add a branch BEFORE the more-generic suffixes:
 
 ```js
 if (item.dockerfile.endsWith("<your-dockerfile-suffix>")) {
@@ -54,7 +54,9 @@ for (const e of m.include.filter(e => e.backend === '<your-backend>')) {
 }"
 ```
 
-A quick way to find the right insertion point: `grep -n 'item.dockerfile.endsWith' scripts/changed-backends.js`.
+A quick way to find the right insertion point: `grep -n 'item.dockerfile.endsWith' scripts/lib/backend-filter.mjs`.
+
+If your backend consumes a *shared* build input that lives outside its own directory (a new script under `scripts/build/`, a new file copied into every image), add a rule to `SHARED_BUILD_INPUTS` in the same file — the per-backend prefix match cannot see those, and a miss ships your change to no image at all. See `scripts/lib/backend-filter_test.mjs` for the pattern; `make test-ci-scripts` runs it.
 
 **`bump_deps.yaml` registration — REQUIRED for any backend pinning an upstream commit.** If your backend's Makefile has a `*_VERSION?=<sha>` pin to a third-party repo, the daily auto-bump bot at `.github/workflows/bump_deps.yaml` won't notice it unless you register the backend in its matrix. The bot runs `.github/bump_deps.sh` which `grep`s for `^$VAR?=` in the Makefile you list — so the pin MUST live in the Makefile (not in a separate shell script). The bump for ds4 (#9761) had to walk this back because the original landed the pin in `prepare.sh`, which the bot can't see. Pattern (for `antirez/ds4`):
 
@@ -115,7 +117,7 @@ Wiring a backend into `includeDarwin:` is more than the matrix entry:
 
 1. **`includeDarwin:` entry** — `tag-suffix: "-metal-darwin-arm64-<backend>"`, `build-type: "metal"`, `lang: "go"` for go+ggml backends; omit `build-type` for the bespoke C++ ones (llama-cpp / ds4 / privacy-filter). Match an existing entry of the same shape.
 2. **`backend/index.yaml`** — add `metal:` to the backend's `capabilities` map (main and `-development`) and concrete `metal-<backend>` / `metal-<backend>-development` image entries pointing at the `-metal-darwin-arm64-<backend>` images.
-3. **C/C++ backends only** — add an `inferBackendPathDarwin` case in `scripts/changed-backends.js` returning `backend/cpp/<backend>/` (the generic fallthrough assumes `backend/<lang>/`, which is wrong for a C++ source tree driven with `lang: go`), and give `run.sh` a Darwin branch that exports `DYLD_LIBRARY_PATH` instead of `LD_LIBRARY_PATH`. If the build is bespoke (single `grpc-server` + dylib bundling), model it on `scripts/build/ds4-darwin.sh` and add a `backends/<backend>-darwin` make target plus a gated step in `.github/workflows/backend_build_darwin.yml`.
+3. **C/C++ backends only** — add an `inferBackendPathDarwin` case in `scripts/lib/backend-filter.mjs` returning `backend/cpp/<backend>/` (the generic fallthrough assumes `backend/<lang>/`, which is wrong for a C++ source tree driven with `lang: go`), and give `run.sh` a Darwin branch that exports `DYLD_LIBRARY_PATH` instead of `LD_LIBRARY_PATH`. If the build is bespoke (single `grpc-server` + dylib bundling), model it on `scripts/build/ds4-darwin.sh` and add a `backends/<backend>-darwin` make target plus a gated step in `.github/workflows/backend_build_darwin.yml`.
 4. **C++ proto gotcha** — if the backend compiles the generated gRPC/protobuf in a separate CMake target (e.g. `hw_grpc_proto`), that target must link `protobuf::libprotobuf` + `gRPC::grpc++` so the Homebrew include dirs propagate; otherwise macOS fails with `google/protobuf/runtime_version.h not found` (Linux hides this because apt headers sit in `/usr/include`).
 
 The CI path filter only builds a backend on a PR when a file under its directory changes, so a darwin-only YAML edit builds nothing — touch a file under `backend/<lang>/<backend>/` (a one-line comment is enough) in the same PR.
@@ -216,6 +218,69 @@ docker-build-backends: ... docker-build-<backend-name>
 - If the backend is in `backend/python/<backend-name>/` but uses `.` as context in the workflow file, use `.` context
 - Check similar backends to determine the correct context
 
+## Engine preference for gallery model variants
+
+A gallery entry can declare `variants`, alternative builds of the same weights,
+and LocalAI picks one per host: it drops builds whose backend cannot run here or
+that do not fit memory, then ranks the survivors by **engine preference
+first, serving feature second, size third** (`SelectVariant` in
+`core/gallery/resolve_variant.go`).
+
+Ask whether your backend should outrank another one on some hardware. If it
+should, add it to `engineNamePreferenceRules` in `pkg/system/capabilities.go`,
+best engine first for that capability:
+
+```go
+ 	{Nvidia, []string{engineVLLM, engineSGLang, engineLlamaCpp}},
++	{Nvidia, []string{engineVLLM, engineSGLang, engineMyEngine, engineLlamaCpp}},
+```
+
+That is the ENGINE NAME table, matched as a substring of a gallery entry's
+`backend:` value. Two sibling tables in the same file speak different
+vocabularies and are matched against different things:
+
+| Table | Vocabulary | Matched against | Consumer |
+|-------|-----------|-----------------|----------|
+| `backendBuildTagPreferenceRules` | build tags (`cuda`, `rocm`, `metal`) | installed build directory names, as a substring | alias resolution in `ListSystemBackends` |
+| `engineNamePreferenceRules` | engine names (`vllm`, `llama-cpp`, `mlx`) | a gallery entry's `backend:`, as a substring | gallery variant ranking |
+| `servingFeaturePreferenceTokens` | serving features (`dflash`, `mtp`) | a gallery entry's `tags:`, compared whole and case-insensitively, and nothing else | gallery variant ranking, one rank below the engine |
+
+**Putting a token in the wrong table matches nothing and does not error**: every
+candidate scores equal and the next sort key decides, so the preference silently
+stops existing. The block comment above all three tables spells the contract out.
+
+The serving feature table is the odd one: it is not keyed by capability, because
+no hardware prefers a plain build over an equivalent faster build of the same
+weights. It reads a declared tag and nothing else. The entry name was the
+original signal and is gone: a naming convention is not a contract, and names
+are author-supplied free text where a short marker like `mtp` turns up inside
+unrelated words or on weights whose entry enables nothing.
+`overrides.options` was rejected for the mirror-image reason: `spec_type:` is
+llama.cpp's config vocabulary, whereas a cross-backend ranking decision must
+work the same for `ds4`'s `mtp_path:` and `sglang`'s `speculative_algorithm:`.
+
+**If your backend can serve the same weights faster** (speculative decoding,
+multi-token prediction), say so in the docs for its gallery entries so curators
+tag them: the tagging rule and the per-backend evidence table live in
+[adding-gallery-models.md](adding-gallery-models.md). A backend never needs to
+appear in the token table itself; it ranks builds, not engines.
+
+Leaving your backend out is a valid choice when no ordering can be justified for
+it. It then ranks below every known engine and selection falls back to size,
+which is the behaviour that predates preference.
+
+**Leaving a whole capability out is not.** A missing row gives that host an
+empty preference list, so size alone decides among everything that survives the
+filters, and the filter will not save you: `IsBackendCompatible` derives hardware
+support from the engine NAME, so `vllm` and `sglang` carry no darwin, cuda, rocm
+or sycl token and are never dropped on a host with no GPU. That is why `default`
+(no usable accelerator, including a GPU under the 4 GiB VRAM floor) and
+`darwin-x86` both have rows putting `llama-cpp` first. Every capability
+`getSystemCapabilities()` can return needs a row unless every engine really is
+equally at home there. When you add one, enumerate the engines you are demoting
+rather than relying on them falling through unmatched: unmatched engines all tie
+with each other, so size decides among them.
+
 ## Documenting the backend (README + docs)
 
 A backend is not "added" until it is discoverable. Update the user-facing docs:
@@ -243,7 +308,7 @@ After adding a new backend, verify:
 
 - [ ] Backend directory structure is complete with all necessary files
 - [ ] Build configurations added to `.github/backend-matrix.yml` for all desired platforms (per-arch entries with `platform-tag` for multi-arch; `builder-base-image` for llama-cpp / ik-llama-cpp / turboquant)
-- [ ] **OS coverage considered**: added to `includeDarwin:` (macOS/Apple Silicon) if the backend can build there — with the `backend/index.yaml` `metal:` capability + `metal-<backend>` image entries, a `run.sh` Darwin/DYLD branch and `inferBackendPathDarwin` case for C++ backends — or the PR explains why an OS is unsupported. Do not ship Linux-only by default.
+- [ ] **OS coverage considered**: added to `includeDarwin:` (macOS/Apple Silicon) if the backend can build there — with the `backend/index.yaml` `metal:` capability + `metal-<backend>` image entries, a `run.sh` Darwin/DYLD branch and `inferBackendPathDarwin` case (in `scripts/lib/backend-filter.mjs`) for C++ backends — or the PR explains why an OS is unsupported. Do not ship Linux-only by default.
 - [ ] Meta definition added to `backend/index.yaml` in the `## metas` section
 - [ ] Image entries added to `backend/index.yaml` for all build variants (latest + development)
 - [ ] Tag suffixes match between workflow file and index.yaml
@@ -251,6 +316,8 @@ After adding a new backend, verify:
 - [ ] No YAML syntax errors (check with linter)
 - [ ] No Makefile syntax errors (check with linter)
 - [ ] Follows the same pattern as similar backends (e.g., if it's a transcription backend, follow `faster-whisper` pattern)
+- [ ] **`Load` validates its input and refuses models it can't serve.** When a model config has no explicit `backend:`, the model loader greedily probes *every* installed backend with the model's name and binds to the first `Load` that succeeds — an accept-anything `Load` will capture arbitrary LLMs (issue #9287). Backends that load a real artefact get this for free (the load fails); backends with no artefact must gate on the name: `opus` accepts only its own name (or none), `local-store` requires the `store.NamespacePrefix` namespace marker sent by `core/backend/stores.go`.
+- [ ] **Gallery variant ranking considered**: if this backend should be preferred over another on some hardware, it is listed in `engineNamePreferenceRules` (NOT `backendBuildTagPreferenceRules`, NOT `servingFeaturePreferenceTokens`) in `pkg/system/capabilities.go`. A missing entry silently ranks it last and lets the next sort key decide.
 - [ ] Documented: added to the category list in `docs/content/features/backends.md` (and any new endpoint/realtime capability documented under `docs/content/`)
 - [ ] If it is an in-house native C/C++/GGML engine, added to the maintained-engines table in the top-level `README.md`
 

.agents/adding-gallery-models.md

@@ -91,6 +91,108 @@ To add a variant (e.g., different quantization), use YAML merge:
       uri: huggingface://<gguf-org>/<gguf-repo>/<filename>-Q8_0.gguf
 ```
 
+## Offering several builds of one model (`variants`)
+
+When the same model is published in more than one quantization, or is also
+servable by another engine, add each build as its own ordinary gallery entry and
+then point one of them at the others with `variants`:
+
+```yaml
+- !!merge <<: *chatml
+  name: "nanbeige4.1-3b-q4"
+  # ... the usual urls / overrides / files for the Q4 build ...
+  variants:
+    - model: nanbeige4.1-3b-q8
+```
+
+Rules:
+
+- The declaring entry is a **complete, normal entry**. It keeps its own
+  `files`/`overrides` and stays installable on every host and by every older
+  LocalAI release, which simply ignore `variants`.
+- A variant references another gallery entry **by name**. That entry must exist
+  and must not declare `variants` of its own.
+- **A referenced entry keeps its own gallery row by default.** It is hidden only
+  in the collapsed listing (`collapse_variants=true`, which the web UI requests
+  by default), where the declaring entry stands in for it. Searching there still
+  matches the referenced entry and answers with the entry declaring it, so
+  referencing an entry never makes it unfindable; turning the collapse off
+  returns it under its own name.
+- **Order carries no meaning.** Do not try to encode a preference; write the
+  list in whatever order reads best.
+- **A variant may be smaller than the declaring entry.** Offering a downgrade
+  for small hosts is a normal shape: the declaring entry's own build competes
+  like every other candidate, so a large host keeps the large build.
+- **Do not describe hardware.** At install time LocalAI drops variants whose
+  backend cannot run on the host, then drops those that do not fit available
+  memory. The declaring entry's own build is exempt from both filters, so
+  selection always terminates on something installable. Sizes are measured live
+  from the weights and cached, so nothing has to be written down.
+- **Engine preference outranks size.** Among the builds that survive the
+  filters, the host's preferred engine wins first and only then does the larger
+  footprint win. On NVIDIA a vLLM build beats a larger llama.cpp one; on Apple
+  silicon an MLX build beats a larger GGUF one; on a host with no preference for
+  either engine the larger build wins, since a bigger footprint is a higher
+  quality quantization of the same weights. Predict what a user gets by asking
+  which engine the host prefers before asking which build is biggest. The
+  per-capability order lives in `engineNamePreferenceRules`
+  (`pkg/system/capabilities.go`); see
+  [adding-backends.md](adding-backends.md) for how a backend gets into it.
+- **Serving feature preference sits between engine and size.** Among builds on
+  an equally preferred engine, one that speculates or predicts several tokens
+  per step beats the plain build of the same weights, because it answers faster
+  for the same output: a `dflash` build beats an `mtp` one, and either beats a
+  plain build. The order lives in `servingFeaturePreferenceTokens`
+  (`pkg/system/capabilities.go`) and is matched against the entry's `tags:` and
+  **nothing else**: not the entry name, not `overrides.options`. See
+  [the tagging rule](#the-dflash--mtp-tagging-rule) below. Engine deliberately
+  outranks it: a serving feature makes the right engine faster, it does not make
+  a wrong engine right. Fit still outranks both, so a drafter pairing (strictly
+  larger than the plain build, since it ships a drafter alongside it) is dropped
+  on a host too small for it before this order is ever consulted.
+- A variant is nothing but a name; there is no per-variant memory field. When
+  the measured size for a build is wrong, correct it on the referenced entry by
+  setting that entry's own `size:` (e.g. `size: "20GiB"`). The estimator prefers
+  a declared size over its own guesswork, so the fix applies everywhere the size
+  is shown or compared rather than only to variant selection.
+
+Users can override the automatic choice with `variant` on `POST /models/apply`,
+`local-ai models install --variant`, or the `install_model` MCP tool. See
+`docs/content/features/model-gallery.md`.
+
+The gallery lint specs live in `core/gallery`, so run that suite after adding a
+`variants` list.
+
+### The `dflash` / `mtp` tagging rule
+
+**Tag an entry `dflash` or `mtp` when the entry actually configures that
+feature. Variant ranking reads the tag and nothing else.**
+
+Decide by looking at what the entry configures, in whatever vocabulary its
+backend uses:
+
+| Backend | Configures the feature when it declares |
+|---------|------------------------------------------|
+| `llama-cpp` | `overrides.options` contains `spec_type:draft-dflash` or `spec_type:draft-mtp` |
+| `ds4` | `overrides.options` contains `mtp_path:` / `mtp_draft:` |
+| `sglang` | the referenced `gallery/*.yaml` sets `speculative_algorithm:` |
+
+That check is curation-time only. `spec_type` is llama.cpp's config vocabulary,
+and a cross-backend ranking decision must not depend on one backend's option
+syntax, which is precisely why the ranker reads the tag instead of the options.
+
+Two mistakes the rule exists to prevent:
+
+- **Weights that carry the heads are not an entry that enables them.** The
+  NVFP4 GGUF entries ship MTP-bearing weights but set only `use_jinja:true`, so
+  they enable no speculative decoding and must NOT be tagged. Tagging them wins
+  them the feature axis without being any faster.
+- **A name is not a declaration.** An entry whose name spells `-mtp` while
+  configuring nothing gets no tag, and an entry that configures the feature is
+  tagged even when its name says nothing (`hy3`, `glm-5.2`). Ranking never reads
+  the name, so an untagged build that does enable the feature is simply ranked
+  as plain rather than promoted on a marker nobody meant.
+
 ## Available template configs
 
 Look at existing `.yaml` files in `gallery/` to find the right prompt template for your model architecture:

.agents/building-and-testing.md

@@ -28,7 +28,6 @@ The core Go suites (`./pkg`, `./core`, plus the in-process integration suite `./
   - **Build tags (`COVERAGE_TAGS`, passed via `GINKGO_TAGS`):** defaults to `debug auth`. The `auth` tag is required to compile the real (sqlite-backed) auth implementation and its ~150 `//go:build auth` tests — without it those files aren't built, the tests don't run, and the gate scores auth against a stub (~3.7% instead of ~38%). If you add new tag-gated tests, extend `COVERAGE_TAGS` or they won't count (and likely won't run in CI at all).
 - `make test-coverage-check` — runs `test-coverage`, then `scripts/coverage-check.sh` fails the build if total coverage is **below** the committed baseline in `coverage-baseline.txt`. The Linux job in `.github/workflows/test.yml` runs this instead of `make test`.
 - `make test-coverage-baseline` — regenerates and overwrites `coverage-baseline.txt` from the current run.
-- `make install-hooks` — sets `core.hooksPath` to the versioned `.githooks/`, whose `pre-commit` runs checks scoped to what's staged: Go changes → `make lint` + `make test-coverage-check`; `core/http/react-ui/` changes → `make test-ui-coverage-check` (Playwright e2e + UI coverage gate). A commit touching neither is skipped; bypass with `git commit --no-verify`. The hook resolves golangci-lint's new-from base to `upstream/master` → `origin/master` → `master`, so it works from a fork clone where `origin/master` is stale (passed to `make lint` via `LINT_NEW_FROM`).
 
 ### React UI coverage
 
@@ -38,12 +37,11 @@ The React UI (`core/http/react-ui/`) has **no component/unit tests** — its onl
 - **Browser:** the flake dev shell ships `chromium` and exports `PLAYWRIGHT_CHROMIUM_PATH`; `playwright.config.js` uses it via `launchOptions.executablePath`, and the Makefile skips `playwright install` when it's set. This avoids Playwright's downloaded browser, which can't resolve system libs (`libglib-2.0`, …) on NixOS. In CI (no `PLAYWRIGHT_CHROMIUM_PATH`) the Makefile falls back to `playwright install --with-deps chromium`.
 - The app is a React SPA, so coverage accumulates across in-app navigation within a test; a full `page.goto`/reload resets it.
 - `.nycrc.json` uses `all: true`, so **every `src/**` file is in the report**, including 0%-coverage ones — that's how you spot features with no test at all (sort the HTML report or `coverage-summary.json` by line% ascending). 
-- **UI coverage gate:** `make test-ui-coverage-check` runs the suite then `scripts/ui-coverage-check.sh`, failing if total line coverage drops more than `UI_COVERAGE_TOLERANCE` below `core/http/react-ui/coverage-baseline.txt`. `make test-ui-coverage-baseline` regenerates the baseline. Runs in CI (`tests-ui-e2e.yml`) and pre-commit on `core/http/react-ui/` changes.
+- **UI coverage gate:** `make test-ui-coverage-check` runs the suite then `scripts/ui-coverage-check.sh`, failing if total line coverage drops more than `UI_COVERAGE_TOLERANCE` below `core/http/react-ui/coverage-baseline.txt`. `make test-ui-coverage-baseline` regenerates the baseline. Runs in CI (`tests-ui-e2e.yml`).
 - **Why it has a tolerance (unlike the strict Go gate):** UI e2e coverage is *non-deterministic*. Specs that assert on state and end while async/lazy render work is still in flight collect those lines only when the render beats the coverage teardown — so the total drifts with machine speed/load (a fast local box reads higher than a slow CI runner), diffusely across many specs. The tolerance absorbs that drift, so set the baseline *below* the slow-CI floor, never to a fast-local `make test-ui-coverage-baseline` number, or CI flaps.
 - **Raising coverage is cheap:** a *render-smoke* spec (navigate to a route, assert its header renders) mounts a lazy page and runs its full render + initial effects, capturing most of its lines in a few lines of test — see `e2e/page-render-smoke.spec.js`. Auth is disabled in the test server (`isAdmin=true`), so `RequireAdmin`/`RequireFeature` routes render without a mock. The most *deterministic* win is removing a race: make a spec `await` a rendered element before ending (see `e2e/agents.spec.js` → AgentCreate) so its lines count every run.
 
 Rules (both gates):
-- **Install the hooks:** `make install-hooks` once per clone so lint + coverage run pre-commit. Don't lean on CI for what the hook catches.
-- **Don't work around the gate:** never `git commit --no-verify`, and never hand-lower a baseline or widen a tolerance to turn a red gate green. The ratchet only moves up.
+- **Don't weaken the gate:** never hand-lower a baseline or widen a tolerance to turn a red gate green. The ratchet only moves up.
 - If a change drops coverage, **add tests** (sort `coverage-summary.json` by line% ascending to find untested code) rather than editing the baseline. When coverage legitimately rises, commit the regenerated baseline (`make test-coverage-baseline` / `test-ui-coverage-baseline`).
 - The Go gate is **strict — no tolerance**; `covermode=atomic` keeps it deterministic. The UI gate keeps a small tolerance only because its e2e coverage isn't.

.agents/ci-caching.md

@@ -114,8 +114,97 @@ Both `backend.yml` (push) and `backend_pr.yml` (PR) generate their matrix dynami
 - **Tag pushes**: `FORCE_ALL=true` is set from the workflow side (`startsWith(github.ref, 'refs/tags/')`) — releases rebuild every backend regardless of diff.
 - **Schedule / `workflow_dispatch`**: no `event.before`, falls through to "run everything" automatically.
 
+### Shared build inputs
+
+The per-backend prefix match only sees files under a backend's own directory, so a change to shared build infrastructure would rebuild *nothing* — an empty matrix, every job green, and the change reaching no image. That silently un-shipped PR #10946 (a partial-cuDNN packaging fix in `scripts/build/package-gpu-libs.sh`), which merged 1h48m after the weekly cron and so sat unbuilt for a week.
+
+`SHARED_BUILD_INPUTS` in `scripts/lib/backend-filter.mjs` closes that hole. Each rule maps a shared path to the narrowest set of matrix entries it can honestly invalidate, since a full matrix is 417 Linux + 56 Darwin builds:
+
+| Changed path | Rebuilds |
+|---|---|
+| `backend/backend.proto` | nothing if the edit is additive-only, otherwise everything (see below) |
+| `backend/Dockerfile.<x>` | the Linux entries whose `dockerfile:` names it |
+| `backend/python/common/` | Python, Linux + Darwin |
+| `scripts/build/package-gpu-libs.sh` | every Linux entry (Python, Go and C++ all run it) |
+| `scripts/build/<lang>-darwin.sh` | the Darwin entries that build target routes to |
+| `.github/workflows/backend_build[_darwin].yml` | everything on that OS |
+| anything else under `scripts/build/` (except `*_test.sh`) | everything — conservative default for unclassified packaging inputs |
+
+Deliberately excluded: `backend/index.yaml` (gallery metadata, never enters an image), `.github/backend-matrix.yml` (adding a backend would rebuild all of them), `backend/Dockerfile.base-grpc-builder` (owned by `base-images.yml`), and the root `Makefile` (touched in ~11% of commits, and its backend-relevant edits arrive alongside the backend directory anyway). `make test-ci-scripts` pins all of this.
+
+#### `backend/backend.proto` is content-filtered, not path-filtered
+
+Every language consumes the proto, so a path rule for it can only ever say "rebuild all 473 images". It changes in ~1.3% of commits, and that was enough to make it the single largest CI cost driver in the repo: on 2026-07-29 four runs totalling 935 queued jobs traced to nothing but a proto edit, one of which (#11158) was a six-line diff adding `bool cache_prompt = 8;`.
+
+An additive proto edit cannot change how a backend that never references the new symbol behaves, so `filterMatrix()` suppresses the rule for one. `changed-backends.js` fetches `backend/backend.proto` at the base revision (same contents-API pattern as `.github/backend-matrix.yml`) and hands both texts to `protoChangeIsAdditive()`, which compares them structurally rather than textually:
+
+- **Additive, rebuilds nothing**: a new field with an unused number, a new message, a new enum value, a new RPC. Comment, whitespace and ordering changes also land here.
+- **Breaking, rebuilds everything**: a removed, renumbered, retyped or renamed field, a dropped RPC, a changed `option` or `package`. So does an unresolvable base revision, matching the run-all posture used for a truncated diff.
+
+Checked against every proto commit in the preceding six months, all nine resolvable ones classify as additive. Note the tradeoff this accepts: generated stubs do change for an additive edit, so image bytes would differ on a rebuild even though behavior does not. That is the same standard already applied when the filter declines to rebuild on unrelated `pkg/` changes, and the weekly cron remains the backstop.
+
 The Sunday 06:00 UTC cron on `backend.yml` exists specifically because path filtering can leave Python backends frozen on stale wheels. `DEPS_REFRESH` (below) only fires when the build actually runs, so an untouched Python backend would never re-resolve its unpinned deps. The weekly cron is the safety net.
 
+## Content-blind PRs skip the workflows that cannot see them
+
+`backend_pr.yml` and `test-extra.yml` filter themselves (matrix generation and a `detect-changes` job), so a gallery-only or docs-only PR costs them about one job each. The Go and image workflows had no filter of any kind, so a one-line `gallery/index.yaml` edit queued 20 jobs, and a docs-only PR queued the same.
+
+This is worth more than it looks. Measured over the week to 2026-07-30, **97% of CI wall-clock is queueing, 3% is execution** (median queue ~5h against a 4-20min median job). Cutting job count is therefore the only lever that shortens feedback time; making individual jobs faster moves 3%.
+
+The volume is real: 13 gallery-only PRs merged that week with 10 open at once, and 78 of the 137 PRs opened were bot-generated.
+
+`paths-ignore` on the PR trigger of `image-pr.yml` (7 jobs), `build-test.yaml` (3), `lint.yml` (2) and `tests-e2e.yml` (1) drops 13 of those 20. The excluded set:
+
+| Path | Why no image or Go build can see it |
+|---|---|
+| `gallery/**` | Model-gallery metadata, parsed at runtime, never copied into an image |
+| `docs/**`, `examples/**`, `**/*.md` | Never enter an image or a binary. `lint.yml` already excluded these before gallery was added |
+
+### `backend/{cpp,go,python}/**` on `image-pr.yml` and `build-test.yaml` only
+
+Version-pin bumps dominate PR volume: 48 `update/*` PRs in the week to 2026-07-30, from 16 pins, each a two-line diff. Most edit nothing but one `backend/*/<name>/Makefile`.
+
+Neither of those two workflows can observe such a change. `make build` is `go build ./cmd/local-ai`, GoReleaser builds the same plus `./cmd/launcher`, and the core image's final stage ships only `entrypoint.sh`, `healthcheck.sh` and that binary. The per-backend trees are copied into the builder but nothing in them reaches the output.
+
+What still triggers a full run, because none of it lives under those prefixes:
+
+- `backend/backend.proto` — feeds `protogen-go`, so it does change the binary.
+- `go.mod` / `go.sum` — the `go mod tidy` before-hook.
+- `backend/Dockerfile.*` and anything else directly under `backend/`.
+
+Deliberately **not** applied to:
+
+| Workflow | Why it must keep seeing `backend/**` |
+|---|---|
+| `test.yml` | `TEST_PATHS` explicitly includes `./backend/go/cloud-proxy/...`, `./backend/go/local-store/...` and `./backend/go/valkey-store/...` |
+| `lint.yml` | `.golangci.yml` carries `backend/`-scoped rules, so golangci-lint covers that tree |
+| `tests-e2e.yml` | The e2e suite drives real backends over gRPC |
+| `backend_pr.yml` | This is the workflow whose entire job is to rebuild the changed backend |
+
+What still runs, and why it has to:
+
+| Workflow | Why it keeps running |
+|---|---|
+| `test.yml` (`tests`) | `core/gallery/variants_lint_test.go` reads the real `gallery/index.yaml` and asserts the index invariants (no duplicate entry names, no build claimed by two parents). This is the only schema-level check the gallery has. |
+| `yaml-check.yml` (`Yamllint`) | Lints `gallery/` for syntax. |
+| `backend_pr.yml`, `test-extra.yml` | Already self-filtering; they stop after the detect step. |
+
+Two properties this relies on:
+
+- `paths-ignore` skips a run only when **every** changed file matches, so a PR touching the gallery *and* Go code still runs everything. That is what makes the exclusion safe rather than a hole.
+- `master` carries no branch protection and no rulesets, so a skipped workflow reports no status and nothing waits on it. If required status checks are ever introduced, these four entries must be excluded from the required set or PRs will hang on "Expected — Waiting for status to be reported".
+
+### `image.yml` on master push is gated too, by a job rather than a path filter
+
+The same reasoning applies to master pushes, and the volume is larger there: on 2026-07-30, **12 of the 23 queued `image.yml` runs** were commits like "add 1 new model to gallery" or a docs fix, each rebuilding all 18 container images.
+
+`image.yml` now has a `changes` job that decides once whether the push can affect any image; the other 11 jobs carry `needs: changes` plus an `if:` on its output. Verified against the shipped `Dockerfile`: the final stage copies only `entrypoint.sh`, `healthcheck.sh` and the `local-ai` binary, there is no `go:embed` of `gallery/` or `docs/`, and the gallery is fetched at runtime from `github:mudler/LocalAI/gallery/index.yaml@master`. A gallery-only commit therefore produces byte-identical images, and the gallery change reaches users through GitHub immediately whether or not an image is rebuilt.
+
+Two properties to preserve if you touch it:
+
+- **It is a job gate, not `paths-ignore`.** `paths-ignore` on `push` also applies to tag pushes, and a tag created on an existing commit carries an empty commits list, which would silently skip the release image build. The gate short-circuits to "build" for `refs/tags/*`, and for any push whose base commit is missing, zero, or unresolvable.
+- **The merge jobs must name the gate explicitly.** They use `if: ${{ !cancelled() && ... }}`, and `!cancelled()` is true when a dependency is *skipped*, so without the extra condition they would run and try to merge manifest lists for images that were never built.
+
 ## The `DEPS_REFRESH` cache-buster (Python backends)
 
 Every Python backend goes through the shared `backend/Dockerfile.python`, which ends with:
@@ -151,15 +240,38 @@ RUN --mount=type=cache,target=/root/.ccache,id=<backend>-ccache-${TARGETARCH}-${
     bash /usr/local/sbin/compile.sh
 ```
 
-The compile script exports `CMAKE_C/CXX/CUDA_COMPILER_LAUNCHER=ccache` so CMake threads ccache through gcc/g++/nvcc. `cache-to: type=registry,mode=max` exports the cache mount data into the registry cache, so subsequent builds restore it.
+The compile script exports `CMAKE_C/CXX/CUDA_COMPILER_LAUNCHER=ccache` so CMake threads ccache through gcc/g++/nvcc. Cache scope is per `(TARGETARCH, BUILD_TYPE)` so e.g. cublas-12 doesn't share with cublas-13 (their CUDA headers differ; cross-pollination would just be cache misses anyway).
 
-On a `LLAMA_VERSION` bump, most translation units are byte-identical to the previous version's preprocessed source — ccache returns the previous `.o` and skips the real compile. Same for LocalAI source changes that don't actually touch llama.cpp's CMake inputs. Cache scope is per `(TARGETARCH, BUILD_TYPE)` so e.g. cublas-12 doesn't share with cublas-13 (their CUDA headers differ; cross-pollination would just be cache misses anyway).
+### ⚠️ This ccache does nothing in CI today
+
+This section previously claimed that `cache-to: type=registry,mode=max` "exports the cache mount data into the registry cache, so subsequent builds restore it". **That is not true.** BuildKit does not export the contents of a `--mount=type=cache` to a registry cache export. A cache mount lives in the builder's local state, and every CI job gets a fresh runner with a fresh builder, so `/root/.ccache` starts empty on every single build.
+
+Measured on 2026-07-30 from the `ccache -s` output the compile script already prints (it runs `ccache -z` first, so the numbers are per-build):
+
+| Job | Commit touched | Build time | ccache |
+|---|---|---|---|
+| 89766266951 (llama-cpp, cublas 13) | `backend/go/magpie-tts-cpp/Makefile` only | 6369s | **0 / 889 hits**, and 0 / 1778 |
+| 89766267281 (llama-cpp, hipblas) | same commit | 8160s | **0 / 537 hits** |
+| 90210828110 (llama-cpp, cublas 12.8) | `LLAMA_VERSION` bump | 5673s | **0 / 813 hits** |
+
+The first two are the decisive control: commit `90355cd44` changed exactly one file, `backend/go/magpie-tts-cpp/Makefile`, nowhere near llama.cpp. The engine source was byte-identical to the previous build, which is precisely the case this section says ccache should serve, and the hit rate was still **0.00%**. A cache that was being restored but merely matching poorly would show partial hits; 0-of-N is the signature of an empty cache.
+
+So the paragraph above about `LLAMA_VERSION` bumps reusing previous `.o` files describes an intended design that is not in effect. `Dockerfile.{llama-cpp,ik-llama-cpp,turboquant,bonsai,ds4,privacy-filter}` pay the ccache wrapper overhead and get nothing back. Multi-hour C++ rebuilds are recompiling identical translation units from scratch.
+
+**Do not "fix" this by adding cache mounts to more Dockerfiles.** Wiring the same mount into `Dockerfile.golang` (215 of the 434 matrix entries) was measured locally at 18% faster on a rebuild after a source edit, with a 71.5% ccache hit rate — but only because the local test reused one builder across both builds. In CI it would be a no-op for exactly the reason above.
+
+Making this actually work needs the cache to live outside the builder. The options, none of them free:
+
+- **ccache `remote_storage`** (ccache ≥ 4.4, HTTP or Redis backend) or **sccache** with an S3/GCS/Redis backend. Genuinely works across runners; needs a cache service to point at. quay.io is a registry, not a blob store, so the existing infra does not cover it.
+- **Round-trip the cache dir through `actions/cache` on the runner**: restore it, pass it in, and export it back out via a build stage output. No external infra, but clunky, and the repo already sits at GitHub's 10 GB cache ceiling while the llama-cpp ccache alone is capped at 5 GB.
+
+Until one of those lands, treat C++ backend builds as always-cold and spend the effort on not running them instead (path filtering, see above).
 
 ## Composite actions
 
 Two composite actions handle runner-side prep:
 
-- **`.github/actions/free-disk-space/action.yml`** — wraps `jlumbroso/free-disk-space@main` plus an explicit apt purge of dotnet/android/ghc/mono/etc. Reclaims ~6–10 GB on `ubuntu-latest`. No-op on self-hosted runners. Used by `backend_build.yml`, `image_build.yml`, `test.yml`, `tests-aio.yml`, etc.
+- **`.github/actions/free-disk-space/action.yml`** — wraps `jlumbroso/free-disk-space@main` plus an explicit apt purge of dotnet/android/ghc/mono/etc. Reclaims ~6–10 GB on `ubuntu-latest`. No-op on self-hosted runners. Used by `backend_build.yml`, `image_build.yml` and `base-images.yml` — the jobs that actually build images. Deliberately **not** used by `test.yml`, which runs no buildx step.
 - **`.github/actions/setup-build-disk/action.yml`** — relocates Docker's data-root to `/mnt` on hosted X64 runners. GHA hosted `ubuntu-latest` ships ~75 GB of unused space at `/mnt`; combined with the free-disk-space cleanup this gives ~100 GB working space — enough for ROCm dev image + vLLM torch install + flash-attn intermediate layers. No-op on self-hosted and on non-X64 hosted runners. Used by `backend_build.yml`, `image_build.yml`, `base-images.yml`.
 
 Both actions run before any docker buildx step.
@@ -200,10 +312,20 @@ Eviction is rarely needed in normal operation — `DEPS_REFRESH` handles weekly
 
 ## What the cache does **not** cover
 
-- The `free-disk-space` and `setup-build-disk` composite actions run on every job — these reclaim runner-state, not Docker layers, so BuildKit caches don't apply.
+- The `free-disk-space` and `setup-build-disk` composite actions run on every job — these reclaim runner-state, not Docker layers, so BuildKit caches don't apply. `test.yml` deliberately does **not** use `free-disk-space`: it runs no buildx step, and the multi-GB fixture downloads that once justified it left `make test` in the test-suite reorg.
 - Intermediate artifacts of `Build (PR)` are not pushed anywhere — PRs only build for verification.
 - Darwin builds (see below) — macOS runners have no Docker daemon, so the registry-backed BuildKit cache cannot apply.
 
+### The Linux Go workflows set `cache: false` on purpose
+
+`test.yml`, `lint.yml`, `tests-e2e.yml` and friends pass `cache: false` to `actions/setup-go@v5`, unlike the darwin jobs. This looks like an oversight and is not.
+
+Measured over the week to 2026-07-30, the `Set up Go` step has a **median of 11 seconds** on these runners. There is essentially nothing to win: the module download is not where the time goes. The expensive steps are compilation and test execution (`Test (with coverage gate)` at ~18.6min, `Test Backend E2E` at ~14.5min), and Go's build cache would have to survive across runners to touch those.
+
+Enabling it also has a real cost. GitHub caps Actions cache at **10 GB per repo and the repo already sits at that ceiling** (31 entries), so every `setup-go` entry written by a branch with a distinct `go.sum` (222-375 MB on Linux, up to 1.4 GB on macOS) evicts something else. See the darwin cache budget below.
+
+Before re-enabling this, measure `Set up Go` again and confirm it has actually become slow. If room is needed in the 10 GB budget, the cheapest evictions are the `docker.io--tonistiigi--binfmt` entries (~30 MB each, trivially re-fetched).
+
 ## Darwin native caches
 
 `backend_build_darwin.yml` runs natively on `macOS-14` GitHub-hosted runners — there is no Docker, no BuildKit, no cross-job registry cache. Instead, the reusable workflow uses `actions/cache@v4` for four native caches that mirror the spirit of the Linux cache (warm by default, weekly refresh for unpinned Python deps, PRs read-only).
@@ -237,6 +359,26 @@ GitHub Actions caches are limited to 10 GB per repo. Steady-state worst case: ~8
 
 One residual self-hosted reference remains in `test-extra.yml` (`tests-vibevoice-cpp-grpc-transcription` uses `bigger-runner` for the 30s JFK-decode timeout headroom). That's a separate concern.
 
+### Small always-on jobs routed to `arc-runner-set`
+
+The hosted pool is shared across the whole *account*, not per repo, so a burst in one repo starves the others. On 2026-07-31 it went to **zero scheduled jobs for 35 consecutive minutes** with 39 jobs queued, while `arc-runner-set` completed 12 jobs without interruption over the same window. Actions was healthy globally at the time (other public repos were scheduling normally), so this is an account-level throttle, not an outage.
+
+`gh-pages.yml` (`build` + `deploy`) is therefore routed to `arc-runner-set` when `github.repository == 'mudler/LocalAI'`. It needs no fork-safety clause because it only triggers on push-to-master and `workflow_dispatch`, so it never executes pull-request code. The repository guard keeps forks (which have no such runner label) from queueing forever. It fetches its own toolchains via `setup-go` / `actions-hugo` and uses no `sudo`/`apt`.
+
+#### What the `arc-runner-set` image actually contains
+
+Measured 2026-07-31 on run `30637392862` by a preflight step, not assumed:
+
+| present | **absent** |
+|---|---|
+| `git`, `curl`, `unzip`, `tar`, `ldd`, `python3` | **`make`**, **`gcc`** |
+
+That is why `lint.yml` is **not** on the self-hosted pool. Both of its jobs were routed there and both failed in one second: `golangci-lint` needs `make` (for `make protogen-go`, itself needing `curl`+`unzip` to fetch protoc, and for `make lint`), and `build-scripts` additionally needs a C toolchain because the packaging-script tests compile a throwaway binary and inspect it with `ldd`. Both jobs are back on `ubuntu-latest`.
+
+The preflight steps were deliberately left in place. They cost about a second on the hosted pool and mean that whenever the runner image gains `make` + `gcc`, re-routing is one `runs-on:` line per job and any remaining gap reports itself by name rather than as an opaque mid-build failure.
+
+Note for any future re-route: `lint.yml` also triggers on `pull_request`, and a fork PR runs untrusted contributor code. That must never reach a persistent self-hosted runner, so any re-route has to stay push-only, e.g. `${{ (github.event_name == 'push' && github.repository == 'mudler/LocalAI') && 'arc-runner-set' || 'ubuntu-latest' }}`.
+
 ## Touching the cache pipeline
 
 When changing `image_build.yml`, `backend_build.yml`, any of the `backend/Dockerfile.*` files, `Dockerfile.base-grpc-builder`, `.docker/install-base-deps.sh`, `.docker/<backend>-compile.sh`, or `scripts/changed-backends.js`:

.agents/coding-style.md

@@ -65,7 +65,42 @@ This is enforced by `forbidigo` (see `.golangci.yml`): `http.DefaultClient` and
 
 The project documentation is located in `docs/content`. When adding new features or changing existing functionality, it is crucial to update the documentation to reflect these changes. This helps users understand how to use the new capabilities and ensures the documentation stays relevant.
 
+- **Docs-with-code rule**: When you change user-facing behavior (API endpoints, CLI flags, config keys, or features), update the corresponding page under `docs/content/` in the SAME change, not as a follow-up. A user-facing change without a matching docs update is incomplete. The PR template carries a checklist item for this.
 - **Feature Documentation**: If you add a new feature (like a new backend or API endpoint), create a new markdown file in `docs/content/features/` explaining what it is, how to configure it, and how to use it.
 - **Configuration**: If you modify configuration options, update the relevant sections in `docs/content/`.
 - **Examples**: providing concrete examples (like YAML configuration blocks) is highly encouraged to help users get started quickly.
 - **Shortcodes**: Use `{{% notice note %}}`, `{{% notice tip %}}`, or `{{% notice warning %}}` for callout boxes. Do **not** use `{{% alert %}}` — that shortcode does not exist in this project's Hugo theme and will break the docs build.
+
+## React UI styling
+
+The React UI ships a design system in `core/http/react-ui/src/App.css`: design
+tokens, form grids, data tables, stat cards, callouts, plus a small semantic
+primitive layer (`.stack`, `.hstack`, `.text-note`, `.text-meta`, `.tone-*`,
+`.icon-chip`). **Use it instead of `style={{ ... }}`.** Inline styles are a
+spacing or colour decision made in one file, so no two pages end up sharing a
+rhythm, which is the main reason the app reads as unfinished.
+
+Inline styles are still correct for values that are genuinely computed at
+runtime: `width: ${pct}%`, a data-driven `background`, a tooltip's coordinates.
+Everything else belongs in a class.
+
+A ratchet enforces this:
+
+```sh
+cd core/http/react-ui
+npm run lint:inline-styles          # fails if the count went UP
+npm run lint:inline-styles:report   # per-file counts, worst first
+npm run lint:inline-styles:write    # refresh the baseline after converting
+```
+
+The gate also fails on **duplicate `className` attributes on one element**. JSX
+keeps the last and silently drops the first, so `<i className={icon}
+className="text-xs" />` loses its icon while passing lint, the build and the e2e
+suite. Converting a style to a class on an element that already has a
+`className` is the usual way to introduce one; merge them into a single
+attribute instead.
+
+When converting a page, prefer naming the shapes it actually has
+(`.p2p-diagram`, `.usage-tile`) over adding more utilities, and check whether an
+existing block already covers it: the Nodes page reuses the P2P setup shapes,
+and Model Editor reuses the Settings section rail.

.agents/preparing-a-release.md

@@ -0,0 +1,26 @@
+# Preparing a Release
+
+A release is not finished when the tag is pushed. The GitHub release, the blog post and the demo clips ship together, because the changelog says what moved and the post and the clips are what make anyone care.
+
+## What a release must include
+
+1. **Labels on the merged PRs.** GitHub generates the raw notes from PR labels, so label first, generate second. Wrong labels mean a miscategorised changelog that has to be edited by hand.
+2. **`RELEASE_NOTES_vX.Y.Z.md`** at the repository root, in the house style: what changed, why it matters, PR numbers so people can read the diffs.
+3. **A blog post under `website/content/blog/`.** One post per release, front matter with `title`, `date`, `author`, `category: "Release"`, `tags`, `summary` and `extracss: ["blog.css"]`. Cover the two or three changes that alter what a user does day to day, not the whole changelog, and link the PR numbers. See `website/content/blog/what-landed-in-localai-4-8.md` for the shape.
+4. **Demo clips for the notable features.** Anything visible (a new backend, a UI change, a new endpoint, a measured speedup) gets a short screen recording. Put the file in `website/static/media/`, reference it from the blog post, and reuse it on the marketing pages where it fits.
+
+A release without a post and without clips is incomplete, in the same way a user-facing code change without a docs update is incomplete.
+
+## Clip conventions
+
+- MP4, H.264, no audio track unless the feature is about audio. Keep them short (10 to 30 seconds) and loopable.
+- Record the real thing. A clip from the engine's own benchmark suite or a real session, never a mockup.
+- Where the change is a speedup, record both sides on the same machine on the same input, so the comparison is honest.
+- Name the file after the feature, not the release (`vllm-race.mp4`, not `v4-8-demo.mp4`), so it stays reusable once the release is old.
+- The marketing site plays clips with `muted loop playsinline preload="none"` and a `data-lazy` attribute, which the site's IntersectionObserver uses to play and pause them on scroll. Follow that pattern for anything you add.
+
+## Order of work
+
+Label the PRs, generate and edit the release notes, cut the draft release, record the clips while the branch is still fresh in your head, then write the post against the notes and the clips. Publishing the release and merging the post should happen on the same day.
+
+The `creating-localai-releases` skill drives steps 1 to 3 and captures the React UI screenshots that go into the notes.

.agents/vllm-backend.md

@@ -21,6 +21,17 @@ options:
   - reasoning_parser:qwen3
 ```
 
+## `Options[]` doubles as CLI-style engine flags
+
+Beyond the parser names above, `Options[]` carries `--` prefixed engine flags (`--enable-prefix-caching`, `--kv-cache-dtype:fp8_e5m2`). `apply_options_to_engine_args` in `backend/python/common/vllm_utils.py` maps them onto `AsyncEngineArgs` fields, and it must run **before** `AsyncLLMEngine.from_engine_args()` - applying them afterwards is a silent no-op, which is exactly what issue #11130 was.
+
+Things to keep straight when touching this:
+
+- Precedence is typed proto fields → `options:` → `engine_args:`. `applyEngineArgDefaults` in `core/config/hooks_vllm.go` therefore skips seeding a production default whose key the user already set as an option, otherwise the later `engine_args:` pass would silently override them.
+- Only `--` prefixed entries are engine flags; `tool_parser:`/`reasoning_parser:` and friends keep their meaning. Parser lookups accept both spellings via `normalize_option_key`.
+- Unknown or uncoercible flags warn and are skipped, unlike `engine_args:` which is strict - `Options[]` is a shared bag and knows entries this mapping doesn't.
+- Field types come from the annotation's *base* (`Literal["auto","float16"]` is not a float). The helper's tests are stdlib-only: `make test-python-helpers`.
+
 Auto-defaults for known model families live in `core/config/parser_defaults.json` and are applied:
 - at gallery import time by `core/gallery/importers/vllm.go`
 - at model load time by the `vllm` / `vllm-omni` backend hook in `core/config/hooks_vllm.go`

.docker/bonsai-compile.sh

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+# Shared compile logic for backend/Dockerfile.bonsai.
+# Sourced (via bind mount) from both builder-fromsource and builder-prebuilt stages.
+
+set -euxo pipefail
+
+export CCACHE_DIR=/root/.ccache
+ccache --max-size=5G || true
+ccache -z || true
+
+export CMAKE_ARGS="${CMAKE_ARGS:-} -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_CUDA_COMPILER_LAUNCHER=ccache"
+
+if [[ -n "${CUDA_DOCKER_ARCH:-}" ]]; then
+  CUDA_ARCH_ESC="${CUDA_DOCKER_ARCH//;/\\;}"
+  export CMAKE_ARGS="${CMAKE_ARGS} -DCMAKE_CUDA_ARCHITECTURES=${CUDA_ARCH_ESC}"
+  echo "CMAKE_ARGS(env) = ${CMAKE_ARGS}"
+  rm -rf /LocalAI/backend/cpp/bonsai-*-build
+fi
+
+cd /LocalAI/backend/cpp/bonsai
+
+if [ -z "${BUILD_TYPE:-}" ]; then
+  # Pure CPU image: one ggml CPU_ALL_VARIANTS build replaces the per-microarch binaries.
+  # arm64: the armv9.2 SME variants need gcc-14 (gcc-13 rejects +sme).
+  if [ "${TARGETARCH}" = "arm64" ]; then
+    apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
+    export CC=gcc-14 CXX=g++-14
+  fi
+  make bonsai-cpu-all
+else
+  # GPU build (cublas/hipblas/sycl/vulkan/...): single fallback CPU build, the accelerator
+  # does the compute. Keeps the GPU compile from also building the CPU variant matrix and
+  # avoids the gcc-14 apt step on GPU base images such as nvidia l4t.
+  make bonsai-fallback
+fi
+make bonsai-grpc
+make bonsai-rpc-server
+
+ccache -s || true

.docker/install-base-deps.sh

@@ -113,6 +113,54 @@ if [ "${BUILD_TYPE:-}" = "vulkan" ] && [ "${SKIP_DRIVERS:-false}" = "false" ]; t
     rm -rf /var/lib/apt/lists/*
 fi
 
+# --- 2b. Intel graphics driver (BUILD_TYPE=sycl*) ---
+# The Intel oneAPI base image brings the compilers and the oneAPI libraries, but
+# not the driver that talks to the graphics card. The packaging step copies that
+# driver into the backend, so that the backend works on a machine which has no
+# Intel graphics packages of its own, for the same reason the Vulkan section
+# above installs the Mesa drivers. Install it here so there is something to copy.
+#
+# Only the sycl builds are covered, because those are the ones whose packaging
+# copies the driver. See package_intel_libs in scripts/build/package-gpu-libs.sh.
+#
+# The driver comes from Intel's own package repository, not from the Ubuntu
+# archive. The archive has 23.43 from late 2023, which does not know any card
+# released since, so a machine with a recent Intel GPU would end up carrying a
+# driver that cannot drive it. Intel's repository has 25.18 for the same Ubuntu
+# release.
+#
+# Anything that goes wrong here fails the build, on purpose. An unreachable
+# repository is a passing problem that a retry fixes, whereas carrying a
+# different driver than intended, or none, is a difference nobody would notice
+# until a user reports an idle GPU.
+if case "${BUILD_TYPE:-}" in sycl*) true;; *) false;; esac \
+    && [ "${SKIP_DRIVERS:-false}" = "false" ]; then
+    # Ubuntu release name, which is what the repository is indexed by.
+    ubuntu_codename=$(. /etc/os-release && echo "${VERSION_CODENAME:-}")
+    if [ -z "$ubuntu_codename" ]; then
+        echo "ERROR: cannot tell which Ubuntu release this image is, so cannot pick the Intel driver repository" >&2
+        exit 1
+    fi
+
+    # The key is armored text, which apt reads directly from a .asc file, so
+    # there is no need for gnupg here. "unified" is the component Intel ships
+    # its current driver in.
+    mkdir -p /usr/share/keyrings
+    curl -fsSL https://repositories.intel.com/gpu/intel-graphics.key \
+        -o /usr/share/keyrings/intel-graphics.asc
+    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.asc] https://repositories.intel.com/gpu/ubuntu ${ubuntu_codename} unified" \
+        > /etc/apt/sources.list.d/intel-graphics.list
+    apt-get update
+    # The first package holds the driver OpenCL talks to, the second the driver
+    # Level Zero talks to. Between them they pull in the compiler and the memory
+    # manager that both need.
+    apt-get install -y --no-install-recommends \
+        intel-opencl-icd \
+        libze-intel-gpu1
+    apt-get clean
+    rm -rf /var/lib/apt/lists/*
+fi
+
 # --- 3. CUDA toolkit (BUILD_TYPE=cublas|l4t) ---
 if { [ "${BUILD_TYPE:-}" = "cublas" ] || [ "${BUILD_TYPE:-}" = "l4t" ]; } && [ "${SKIP_DRIVERS:-false}" = "false" ]; then
     apt-get update

.docker/llama-cpp-build-target.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+arch=${1:?target architecture is required}
+build_type=${2-}
+
+# GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
+# to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
+# builder images can supply that compiler.
+if [ "$arch" = "arm64" ] && [ -n "$build_type" ]; then
+  echo llama-cpp-fallback
+else
+  echo llama-cpp-cpu-all
+fi

.docker/llama-cpp-compile.sh

@@ -18,27 +18,27 @@ if [[ -n "${CUDA_DOCKER_ARCH:-}" ]]; then
 fi
 
 cd /LocalAI/backend/cpp/llama-cpp
-if [ -z "${BUILD_TYPE:-}" ]; then
-  # Pure CPU image (BUILD_TYPE empty): one build with ggml CPU_ALL_VARIANTS replaces the
-  # per-microarch binaries (x86: avx/avx2/avx512/fallback; arm64: armv8.x/armv9.x). ggml
-  # dlopens the best libggml-cpu-*.so at runtime by probing host CPU features.
+BUILD_TARGET=$(/LocalAI/.docker/llama-cpp-build-target.sh "${TARGETARCH}" "${BUILD_TYPE:-}")
+if [ "$BUILD_TARGET" = "llama-cpp-cpu-all" ]; then
+  # One build with ggml CPU_ALL_VARIANTS replaces the per-microarch binaries (x86:
+  # avx/avx2/avx512/fallback; arm64: armv8.x/armv9.x). BUILD_TYPE remains in the
+  # environment, so GPU builds retain their accelerator backend while ggml dlopens the
+  # best CPU library when work is offloaded to the host.
   #
   # arm64: the CPU_ALL_VARIANTS table includes armv9.2 SME variants whose -march=...+sme is
   # rejected by the Ubuntu 24.04 default gcc-13. gcc-14 accepts it, so build the arm64
   # variants with it (the host never *selects* SME unless it has it, but every variant must
   # still compile).
   if [ "${TARGETARCH}" = "arm64" ]; then
+    # The prebuilt base inherits default ports.ubuntu.com sources; honor the
+    # APT_*_MIRROR build args here like the from-source path does, so this
+    # apt step survives a mirror outage.
+    sh /LocalAI/.docker/apt-mirror.sh || true
     apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
     export CC=gcc-14 CXX=g++-14
   fi
-  make llama-cpp-cpu-all
-else
-  # GPU build (cublas/hipblas/sycl/vulkan/...): the accelerator does the compute, so a
-  # single fallback CPU build is enough - no per-microarch CPU variants needed. (This also
-  # keeps the heavy GPU backend compile from also building the whole CPU variant matrix,
-  # and avoids the gcc-14 apt step on GPU base images such as nvidia l4t.)
-  make llama-cpp-fallback
 fi
+make "$BUILD_TARGET"
 make llama-cpp-grpc
 make llama-cpp-rpc-server
 

.docker/turboquant-build-target.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+arch=${1:?target architecture is required}
+build_type=${2-}
+
+# GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
+# to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
+# builder images can supply that compiler.
+if [ "$arch" = "arm64" ] && [ -n "$build_type" ]; then
+  echo turboquant-fallback
+else
+  echo turboquant-cpu-all
+fi

.docker/turboquant-compile.sh

@@ -19,20 +19,18 @@ fi
 
 cd /LocalAI/backend/cpp/turboquant
 
-if [ -z "${BUILD_TYPE:-}" ]; then
-  # Pure CPU image: one ggml CPU_ALL_VARIANTS build replaces the per-microarch binaries.
+BUILD_TARGET=$(/LocalAI/.docker/turboquant-build-target.sh "${TARGETARCH}" "${BUILD_TYPE:-}")
+if [ "$BUILD_TARGET" = "turboquant-cpu-all" ]; then
+  # BUILD_TYPE remains in the environment, so GPU builds retain their accelerator while
+  # ggml selects the best CPU library when model work is offloaded to the host.
   # arm64: the armv9.2 SME variants need gcc-14 (gcc-13 rejects +sme).
   if [ "${TARGETARCH}" = "arm64" ]; then
+    sh /LocalAI/.docker/apt-mirror.sh || true
     apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
     export CC=gcc-14 CXX=g++-14
   fi
-  make turboquant-cpu-all
-else
-  # GPU build (cublas/hipblas/sycl/vulkan/...): single fallback CPU build, the accelerator
-  # does the compute. Keeps the GPU compile from also building the CPU variant matrix and
-  # avoids the gcc-14 apt step on GPU base images such as nvidia l4t.
-  make turboquant-fallback
 fi
+make "$BUILD_TARGET"
 make turboquant-grpc
 make turboquant-rpc-server
 

.dockerignore

@@ -40,6 +40,16 @@ backend/cpp/privacy-filter/build
 backend/cpp/privacy-filter/grpc-server
 backend/cpp/privacy-filter/package
 
+# audio-cpp: same in-place pattern. The Makefile clones audio.cpp at the pinned
+# AUDIO_CPP_VERSION and the `audio.cpp:` target is the directory itself, so a
+# stale host checkout COPY'd in makes the build compile against whatever commit
+# the host had. build/ is worse than stale: its CMakeCache.txt records the host
+# source, prefix and compiler paths, and cmake refuses to reconfigure from it.
+backend/cpp/audio-cpp/audio.cpp
+backend/cpp/audio-cpp/build
+backend/cpp/audio-cpp/grpc-server
+backend/cpp/audio-cpp/package
+
 # Rust backend build output (sources are tracked; target/ is generated)
 backend/rust/*/target
 

.githooks/pre-commit

@@ -1,60 +0,0 @@
-#!/usr/bin/env sh
-#
-# LocalAI pre-commit hook. Install it (once per clone) with:
-#
-#     make install-hooks
-#
-# Runs only the checks relevant to what's staged:
-#   - Go files          -> make lint + make test-coverage-check
-#   - core/http/react-ui -> make test-ui-coverage-check (Playwright e2e + gate)
-# A commit touching neither is skipped entirely (docs/YAML/etc. can't change
-# lint findings, Go coverage, or the UI).
-#
-# To bypass for a single commit (e.g. a WIP checkpoint): git commit --no-verify
-set -eu
-
-repo_root="$(git rev-parse --show-toplevel)"
-cd "$repo_root"
-
-staged="$(git diff --cached --name-only --diff-filter=ACMRD)"
-
-go_changed=0
-ui_changed=0
-if echo "$staged" | grep -qE '\.go$'; then go_changed=1; fi
-if echo "$staged" | grep -qE '^core/http/react-ui/'; then ui_changed=1; fi
-
-if [ "$go_changed" -eq 0 ] && [ "$ui_changed" -eq 0 ]; then
-	echo "pre-commit: no Go or React UI changes staged — skipping."
-	exit 0
-fi
-
-if [ "$go_changed" -eq 1 ]; then
-	# Resolve the ref golangci-lint's new-from-merge-base should compare
-	# against. .golangci.yml pins origin/master, which is correct in CI
-	# (origin == the canonical repo) but wrong from a fork clone, where
-	# origin/master lags behind and lint would report the whole upstream
-	# backlog. Prefer upstream/master, then origin/master, then master.
-	lint_base=""
-	for ref in upstream/master origin/master master; do
-		if git rev-parse --verify --quiet "${ref}^{commit}" >/dev/null 2>&1; then
-			lint_base="$ref"
-			break
-		fi
-	done
-
-	echo "pre-commit ▶ golangci-lint (make lint${lint_base:+, new-from $lint_base})"
-	make lint LINT_NEW_FROM="$lint_base"
-
-	echo "pre-commit ▶ coverage gate (make test-coverage-check) — builds and runs the"
-	echo "             pkg/core suites plus tests/e2e; can take a few minutes."
-	make test-coverage-check
-fi
-
-if [ "$ui_changed" -eq 1 ]; then
-	echo "pre-commit ▶ React UI e2e + coverage gate (make test-ui-coverage-check) —"
-	echo "             rebuilds the UI + ui-test-server, runs the Playwright specs, and"
-	echo "             fails if line coverage regressed; can take a couple of minutes."
-	make test-ui-coverage-check
-fi
-
-echo "pre-commit ✓ all relevant checks passed"

.github/PULL_REQUEST_TEMPLATE.md

@@ -7,6 +7,7 @@ This PR fixes #
 
 **[Signed commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)**
 - [ ] Yes, I signed my commits.
+- [ ] Documentation updated (docs/content/) for user-facing changes, or not applicable
  
 <!--
 Thank you for contributing to LocalAI! 

.github/bump_deps.sh

@@ -1,5 +1,8 @@
 #!/bin/bash
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1
 BRANCH=$2
 VAR=$3
@@ -9,7 +12,20 @@ if [ -z "$FILE" ]; then
     FILE="Makefile"
 fi
 
-LAST_COMMIT=$(curl -s -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH")
+# gh_curl follows redirects so a renamed/transferred upstream repo (GitHub
+# answers 301) still resolves, and fails on HTTP errors rather than letting an
+# error page reach sed below. `|| true` keeps a failed lookup from aborting the
+# script at exit 22 with no context — the SHA guard below reports it instead.
+LAST_COMMIT=$(gh_curl -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH" || true)
+
+# Guard the sed input: anything that is not a bare 40-hex SHA (an API error
+# body, an empty response) would otherwise be spliced into the Makefile pin —
+# either corrupting it silently or blowing up sed with an unterminated
+# expression, which is how this job failed for a renamed repo.
+if ! [[ "$LAST_COMMIT" =~ ^[0-9a-f]{40}$ ]]; then
+    echo "Refusing to bump $VAR: expected a 40-char commit SHA for $REPO@$BRANCH, got: $LAST_COMMIT" >&2
+    exit 1
+fi
 
 # Read $VAR from Makefile (only first match)
 set +e

.github/bump_docs.sh

@@ -1,7 +1,18 @@
 #!/bin/bash
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1
 
-LATEST_TAG=$(curl -s "https://api.github.com/repos/$REPO/releases/latest" | jq -r '.tag_name')
+LATEST_TAG=$(gh_curl -H "Accept: application/vnd.github+json" \
+    "https://api.github.com/repos/$REPO/releases/latest" | jq -r '.tag_name')
+
+# jq prints the string "null" for a missing key, so a throttled or otherwise
+# unexpected API response would otherwise be published as the docs version.
+if [ -z "$LATEST_TAG" ] || [ "$LATEST_TAG" = "null" ]; then
+    echo "Refusing to bump docs version: could not resolve the latest release tag for $REPO." >&2
+    exit 1
+fi
 
 cat <<< $(jq ".version = \"$LATEST_TAG\"" docs/data/version.json) > docs/data/version.json

.github/bump_vllm_metal.sh

@@ -11,6 +11,9 @@
 # darwin build can only use the exact vLLM version vllm-metal supports, so it may
 # lag the Linux pin (requirements-cublas13-after.txt) until vllm-metal catches up.
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1   # vllm-project/vllm-metal
 FILE=$2   # backend/python/vllm/install.sh
 VAR=$3    # VLLM_METAL_VERSION (used for the workflow's output file names)
@@ -22,12 +25,12 @@ fi
 
 # vllm-metal ships frequent dev releases, all flagged as non-prerelease, so
 # /releases/latest returns the newest one (with its cp312 wheel asset).
-LATEST_TAG=$(curl -sS -H "Accept: application/vnd.github+json" \
+LATEST_TAG=$(gh_curl -H "Accept: application/vnd.github+json" \
     "https://api.github.com/repos/$REPO/releases/latest" \
     | python3 -c "import json,sys; print(json.load(sys.stdin)['tag_name'])")
 
 # The coupled vLLM source version lives in vllm-metal's installer at that tag.
-NEW_VLLM_VERSION=$(curl -fsSL \
+NEW_VLLM_VERSION=$(gh_curl \
     "https://raw.githubusercontent.com/$REPO/$LATEST_TAG/install.sh" \
     | grep -oE 'vllm_v="[0-9]+\.[0-9]+\.[0-9]+"' | head -1 | cut -d'"' -f2)
 

.github/bump_vllm_wheel.sh

@@ -9,6 +9,9 @@
 # vars in Makefiles; this script handles the two-value rewrite specific to the
 # vLLM requirements file.
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1   # vllm-project/vllm
 FILE=$2   # backend/python/vllm/requirements-cublas13-after.txt
 VAR=$3    # VLLM_VERSION (used for output file names so the workflow can read them)
@@ -19,7 +22,7 @@ if [ -z "$FILE" ] || [ -z "$REPO" ] || [ -z "$VAR" ]; then
 fi
 
 # /releases/latest returns the most recent non-prerelease tag.
-LATEST_TAG=$(curl -sS -H "Accept: application/vnd.github+json" \
+LATEST_TAG=$(gh_curl -H "Accept: application/vnd.github+json" \
     "https://api.github.com/repos/$REPO/releases/latest" \
     | python3 -c "import json,sys; print(json.load(sys.stdin)['tag_name'])")
 

.github/ci/apexentries/README.md

@@ -0,0 +1,194 @@
+# apexentries
+
+Generates gallery entries for the `mudler/*-APEX-GGUF` HuggingFace repositories.
+
+Each APEX repo becomes one **family**: one entry per quality rung the repo
+publishes and one per quantization rung its unsloth counterpart publishes, all
+gathered under the **base model's** entry. LocalAI's variant selector then picks
+the build that fits the hardware in front of it.
+
+## The hub is the base model entry, never a generated `*-apex` parent
+
+Somebody looking for `qwen3.6-35b-a3b` must find every build of those weights
+under that one name: the APEX imatrix rungs, the unsloth quant rungs and any
+speculative build. A separate `qwen3.6-35b-a3b-apex` hub competing with the base
+entry would split the family in two and leave whichever half the user did not
+search for effectively invisible.
+
+So the generator resolves the hub by stripping the `-APEX`, `-MTP` and `-TQ`
+markers and looking the result up in the index, trying both the repo-derived and
+the stem-derived candidate the same way `CounterpartCandidates` does. Then:
+
+- **The hub exists** (14 of the 45 repos, resolving to 10 distinct entries).
+  Nothing new is emitted for the family root. A `variants:` block is spliced into
+  the entry that is already there, textually, leaving its description, icon,
+  tags, overrides and files untouched. The line editing is shared with the
+  `variantproposals` job via `.github/ci/galleryedit`.
+- **The hub is absent** (the other 31). A new hub is emitted, named for the base
+  model and never for the APEX repo. It carries one of the discovered builds as
+  its own payload so it is a complete installable entry rather than a bare index,
+  and that payload is what gives it an `overrides.backend`. Without a declared
+  backend the verifier would skip it, so a hub carrying feature tags would escape
+  the tagging check in silence.
+
+Several APEX repos routinely resolve to one base model, so both paths accumulate
+by hub name rather than assuming one family per hub.
+
+Two references are always filtered out of a hub's list: anything the entry
+already declares, and the hub's own name. The self reference is not merely
+redundant. An unsloth rung whose weights the gallery already ships under the base
+name resolves, through the merge, straight back to the hub, and the verifier
+reads a self reference as a variant that declares variants of its own.
+
+The four hand-written `*-apex` entries (`qwen3.6-35b-a3b-apex`,
+`gemma-4-26b-a4b-it-apex`, `qwen3.5-35b-a3b-apex`,
+`nemotron-3-nano-omni-30b-a3b-reasoning-apex`) are **ordinary builds**, not hubs.
+They are referenced from their hub's variants list like any other rung, and are
+never deleted or renamed.
+
+## Flags
+
+| Flag | Default | Meaning |
+|------|---------|---------|
+| `-index <path>` | `gallery/index.yaml` | Gallery index to dedup against. Read only, unless `-apply` is passed. |
+| `-only <a,b,c>` | (all) | Comma-separated full repo names (`mudler/Foo-APEX-GGUF`) to restrict generation to. A name that matches nothing is reported as a warning, since it is a typo rather than an empty result. |
+| `-out <path>` | (none) | Write the entries to add to this file. Nothing is written to the gallery. |
+| `-apply` | `false` | Splice the variants into `-index` and append the new entries to it. |
+| `-verify <path>` | (none) | Verify a gallery index and exit. Ignores every other flag. |
+
+Either `-out` or `-apply` is required, otherwise the run has nothing to do.
+
+`-apply` splices variant lines into existing entries and **appends** new ones. It
+never re-serialises the index: it is roughly 40,000 lines, and a YAML round trip
+would reflow the whole file, drop the anchors and merge keys the gallery relies
+on, and produce a diff nobody can review. On the three-family sample the splice
+is 24 added lines across 3 hunks with zero deletions.
+
+## Discovery is by filename suffix, never by repo name
+
+Builds come from the files a repo actually publishes. A filename is never
+constructed from a repo name, because the two disagree:
+`mudler/gemma-4-26B-A4B-it-APEX-GGUF` ships `gemma-4-26B-A4B-APEX-*.gguf`, and
+five other repos likewise drop a suffix (`-it`, `-2603`) or a vendor prefix
+(`NVIDIA-`) that the repo name carries. Composing a URL from the repo name would
+produce a 404 for every one of them, and the 404 would only surface after the
+entry shipped.
+
+The quality ladder is matched on the trailing tier marker, `-(I-)?(Quality|
+Balanced|Compact|Mini|Nano).gguf`. The `I-` prefix marks the imatrix ladder. The
+imatrix ladder is emitted when it is non-empty and the plain ladder is used only
+as a fallback, because two of the 45 repos publish no imatrix tiers at all and
+must still contribute. Eleven repos carry a fifth `I-Nano` rung, so nothing
+assumes a fixed number of rungs.
+
+Every run prints, per repo, the counts that discovery accounted for. If the
+number of classified files is short of the number of `.gguf` files the repo
+publishes, the shortfall is printed as `UNCLASSIFIED`. That check is a set
+difference on counts rather than a second pass over filenames: a second matcher
+would duplicate the tier regex and the two copies would drift. The failure it
+catches is quiet. A publishing-script typo that breaks every imatrix filename in
+a repo does not produce a short ladder; it makes the imatrix ladder empty, and
+the fallback then downgrades the whole family to the plain ladder with nothing
+said. A downstream HTTP check cannot catch it either, because it validates the
+URLs that were emitted, and an undiscovered tier emits none.
+
+The same reasoning applies to `UNACCOUNTED QUANT`, printed when the unsloth
+counterpart demonstrably publishes a wanted quant that produced no build. It is
+reported at discovery time because a dropped quant leaves no trace at all in the
+finished gallery file.
+
+## sha256 always comes from the API
+
+Every file stanza takes its `sha256` from the HuggingFace models API
+(`lfs.sha256`). A GGUF the API describes without one is a fatal error for that
+family: the repo is reported by name and the run ends non-zero. It is never
+substituted from another field, because that is exactly how a Xet hash ends up
+masquerading as a content hash.
+
+## The dflash / mtp tagging rule
+
+An entry is tagged `dflash` or `mtp` **if and only if** it configures the
+matching `spec_type:draft-<feature>`. Variant ranking reads tags and nothing
+else, so a tag that does not match the configuration either promotes a build
+that is no faster or hides one that genuinely is.
+
+A repo name is not configuration. `mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF` ships
+weights that carry MTP heads; an entry that does not enable them is not an MTP
+entry and is not tagged as one.
+
+A generated hub inherits the tags of the build it carries as its payload, rather
+than rebuilding them from the base set, so a hub whose payload configures a
+`spec_type` stays tagged consistently with the overrides copied alongside it.
+
+## Reuse reporting: two categories, not one
+
+Generated entries are deduped against the gallery and against the batch itself.
+The run prints the result under two separate headings, because the two cases are
+not equivalent:
+
+- **URI MATCHES** mean the gallery, or an earlier entry in this batch, already
+  ships exactly these weights. Pointing the hub at the existing entry is correct
+  and needs no thought.
+- **NAME COLLISIONS** mean an entry already owns the name but holds different
+  weights. Referencing it would point the hub at a build other than the one
+  generated. Every one of these must be inspected by hand.
+
+The run then prints `HUBS SPLICED`, listing every reference that will be added to
+an entry the gallery already ships along with the line it will be added at, and
+`HUBS CREATED` for the families that get a new hub. The splices are the part a
+review has to read closely, because they modify entries somebody else wrote.
+
+Hubs are deliberately kept out of the merge. A new hub carries the family's top
+rung as its own payload, so URI dedup would fold the hub into that rung and the
+family would lose the very entry point this command exists to create.
+
+## Workflow: sample first, then the full set
+
+Never run the full generation straight into the gallery. Generate a small,
+deliberately awkward sample, have it reviewed, then run the rest.
+
+```bash
+# 1. Sample three families that between them cover the awkward shapes:
+#    a standard four-rung repo, one with the extra I-Nano rung AND a file stem
+#    that differs from its repo name, and one whose unsloth counterpart shards
+#    its quants across subdirectories.
+go run ./.github/ci/apexentries \
+  -index gallery/index.yaml \
+  -only mudler/Qwen3.6-35B-A3B-APEX-GGUF,mudler/gemma-4-26B-A4B-it-APEX-GGUF,mudler/Step-3.7-Flash-APEX-GGUF \
+  -out /tmp/sample.yaml
+
+# 2. Verify the sample against the gallery it would join, splices included. Apply
+#    to a COPY, never to the real index, and check that the diff is only the
+#    intended variant lines. Compare the verifier output to the gallery's own
+#    baseline: what matters is that the sample adds no new problem, not that the
+#    total is zero.
+cp gallery/index.yaml /tmp/index-copy.yaml
+go run ./.github/ci/apexentries -index /tmp/index-copy.yaml -only <same list> -apply
+diff -u gallery/index.yaml /tmp/index-copy.yaml   # expect zero deletions
+
+go run ./.github/ci/apexentries -verify gallery/index.yaml > /tmp/baseline.log 2>&1
+go run ./.github/ci/apexentries -verify /tmp/index-copy.yaml > /tmp/spliced.log 2>&1
+diff /tmp/baseline.log /tmp/spliced.log
+
+# 3. Have a human review /tmp/sample.yaml and every reported name collision.
+
+# 4. Only then, the full set.
+go run ./.github/ci/apexentries -index gallery/index.yaml -apply
+```
+
+## Tests
+
+```bash
+go test ./.github/ci/apexentries/
+```
+
+The shared line editor has its own package:
+
+```bash
+go test ./.github/ci/galleryedit/
+```
+
+`.github/ci/` is invisible to `go list ./...`, so these specs are not covered by
+`make lint` or the repository test run. `.github/workflows/ci-tools-tests.yaml`
+names the package explicitly; keep that workflow in step with any package added
+under `.github/ci/`.

.github/ci/apexentries/discover.go

@@ -0,0 +1,70 @@
+package main
+
+import (
+	"regexp"
+	"strings"
+)
+
+// tierRE matches the tier marker APEX repos put at the end of a weight
+// filename. Discovery is by suffix because the stem is not predictable from
+// the repo name: six of the 45 repos drop a suffix ("-it", "-2603") or a
+// vendor prefix ("NVIDIA-") that the repo name carries.
+var tierRE = regexp.MustCompile(`-(I-)?(Quality|Balanced|Compact|Mini|Nano)\.gguf$`)
+
+// fullPrecisionRE matches the unquantized source weights an APEX repo publishes
+// alongside its ladder, flat (-F16.gguf) or sharded across a numbered set
+// (-F16-00001-of-00010.gguf). bf16 is accepted because some repos publish that
+// instead, and the match is case-insensitive because the casing varies between
+// publishing scripts.
+//
+// These are deliberately not tiers: they are the weights the ladder is quantized
+// FROM, and generation is scoped to the ladder itself.
+var fullPrecisionRE = regexp.MustCompile(`(?i)-b?f16(-\d{5}-of-\d{5})?\.gguf$`)
+
+// IsFullPrecision reports whether a weight filename is an unquantized source.
+func IsFullPrecision(name string) bool {
+	return fullPrecisionRE.MatchString(name)
+}
+
+// Tier is one discovered build of an APEX repo.
+type Tier struct {
+	Label string
+	File  GGUFFile
+}
+
+// DiscoverAPEXTiers splits a repo's weight files into the imatrix ladder and
+// the plain ladder. mmproj files are never tiers.
+func DiscoverAPEXTiers(files []GGUFFile) (imatrix, plain []Tier) {
+	for _, f := range files {
+		if strings.HasPrefix(f.Name, "mmproj") {
+			continue
+		}
+		m := tierRE.FindStringSubmatch(f.Name)
+		if m == nil {
+			continue
+		}
+		if m[1] != "" {
+			imatrix = append(imatrix, Tier{Label: "I-" + m[2], File: f})
+			continue
+		}
+		plain = append(plain, Tier{Label: m[2], File: f})
+	}
+	return imatrix, plain
+}
+
+// DiscoverMMProj returns the repo's projector file, if it publishes one. The
+// name varies across repos (mmproj.gguf, mmproj-F16.gguf,
+// mmproj-step3.7-flash-f16.gguf), so match the prefix rather than a fixed name.
+func DiscoverMMProj(files []GGUFFile) (GGUFFile, bool) {
+	for _, f := range files {
+		if strings.HasPrefix(f.Name, "mmproj") {
+			return f, true
+		}
+	}
+	return GGUFFile{}, false
+}
+
+// FileStem returns a tier's filename with its tier suffix removed.
+func FileStem(t Tier) string {
+	return tierRE.ReplaceAllString(t.File.Name, "")
+}

.github/ci/apexentries/discover_test.go

@@ -0,0 +1,68 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("DiscoverAPEXTiers", func() {
+	It("finds tiers regardless of how the stem relates to the repo name", func() {
+		// This repo is mudler/gemma-4-26B-A4B-it-APEX-GGUF but its files drop "-it".
+		files := []GGUFFile{
+			{Name: "gemma-4-26B-A4B-APEX-I-Quality.gguf", SHA256: "a"},
+			{Name: "gemma-4-26B-A4B-APEX-I-Nano.gguf", SHA256: "b"},
+			{Name: "gemma-4-26B-A4B-APEX-Quality.gguf", SHA256: "c"},
+			{Name: "mmproj-F16.gguf", SHA256: "d"},
+		}
+
+		imatrix, plain := DiscoverAPEXTiers(files)
+
+		Expect(labels(imatrix)).To(ConsistOf("I-Quality", "I-Nano"))
+		Expect(labels(plain)).To(ConsistOf("Quality"))
+	})
+
+	It("excludes mmproj from the tier list", func() {
+		files := []GGUFFile{{Name: "mmproj.gguf", SHA256: "d"}}
+
+		imatrix, plain := DiscoverAPEXTiers(files)
+
+		Expect(imatrix).To(BeEmpty())
+		Expect(plain).To(BeEmpty())
+	})
+})
+
+var _ = Describe("DiscoverMMProj", func() {
+	It("finds an mmproj whatever its suffix", func() {
+		files := []GGUFFile{
+			{Name: "Model-APEX-I-Mini.gguf", SHA256: "a"},
+			{Name: "mmproj-step3.7-flash-f16.gguf", SHA256: "b"},
+		}
+
+		got, ok := DiscoverMMProj(files)
+
+		Expect(ok).To(BeTrue())
+		Expect(got.Name).To(Equal("mmproj-step3.7-flash-f16.gguf"))
+	})
+
+	It("reports absence when the repo ships none", func() {
+		_, ok := DiscoverMMProj([]GGUFFile{{Name: "Model-APEX-Quality.gguf", SHA256: "a"}})
+
+		Expect(ok).To(BeFalse())
+	})
+})
+
+var _ = Describe("FileStem", func() {
+	It("strips the tier suffix", func() {
+		t := Tier{Label: "I-Quality", File: GGUFFile{Name: "gemma-4-26B-A4B-APEX-I-Quality.gguf"}}
+
+		Expect(FileStem(t)).To(Equal("gemma-4-26B-A4B-APEX"))
+	})
+})
+
+func labels(ts []Tier) []string {
+	out := make([]string, 0, len(ts))
+	for _, t := range ts {
+		out = append(out, t.Label)
+	}
+	return out
+}

.github/ci/apexentries/hf.go

@@ -0,0 +1,130 @@
+package main
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"net/http"
+	"strings"
+	"time"
+
+	"github.com/mudler/LocalAI/pkg/httpclient"
+)
+
+// ErrNoSHA256 marks a GGUF the HuggingFace API describes without an
+// lfs.sha256. Emitting an entry without a hash would ship an unverifiable
+// download, and guessing one from another field is how a Xet hash ends up
+// masquerading as a content hash, so this is fatal rather than skippable.
+var ErrNoSHA256 = errors.New("gguf file has no lfs.sha256")
+
+// GGUFFile is one .gguf sibling of a HuggingFace repo.
+type GGUFFile struct {
+	Name   string
+	Size   int64
+	SHA256 string
+}
+
+type apiSibling struct {
+	RFilename string `json:"rfilename"`
+	Size      int64  `json:"size"`
+	LFS       *struct {
+		SHA256 string `json:"sha256"`
+	} `json:"lfs"`
+}
+
+type apiModel struct {
+	Siblings []apiSibling `json:"siblings"`
+}
+
+// ParseRepoFiles returns every .gguf sibling described by a models API body.
+func ParseRepoFiles(body []byte) ([]GGUFFile, error) {
+	var m apiModel
+	if err := json.Unmarshal(body, &m); err != nil {
+		return nil, fmt.Errorf("decoding model response: %w", err)
+	}
+
+	var out []GGUFFile
+	for _, s := range m.Siblings {
+		if !strings.HasSuffix(s.RFilename, ".gguf") {
+			continue
+		}
+		if s.LFS == nil || s.LFS.SHA256 == "" {
+			return nil, fmt.Errorf("%s: %w", s.RFilename, ErrNoSHA256)
+		}
+		out = append(out, GGUFFile{Name: s.RFilename, Size: s.Size, SHA256: s.LFS.SHA256})
+	}
+	return out, nil
+}
+
+// FetchOptionalRepoFiles asks the models API for a repo the caller can do
+// without, and reports separately whether the repo was merely unreadable.
+//
+// HuggingFace answers 401 Unauthorized, not 404, for a repo that does not exist
+// when the request carries no credentials. Without a token there is therefore no
+// way to tell "this repo was never published" from "this repo is private", so an
+// optional probe has to treat 401 and 403 exactly like 404: whatever the reason,
+// there is nothing here for us to read, so there is no counterpart.
+//
+// The second return value exists because that collapse is lossy in one
+// direction: 401/403 can also mean a real, gated repo whose quants we would
+// genuinely want. The caller reports those repos so a silently dropped
+// counterpart is visible to a human rather than invisible.
+func FetchOptionalRepoFiles(client *http.Client, repo string) ([]GGUFFile, bool, error) {
+	files, status, err := fetchRepoFiles(client, repo)
+	if err != nil && (status == http.StatusUnauthorized || status == http.StatusForbidden) {
+		return nil, true, nil
+	}
+	return files, false, err
+}
+
+// FetchRepoFiles asks the models API for one repo. A 404 yields (nil, nil) so
+// that probing for an optional counterpart repo is not an error. Every other
+// non-200, 401 and 403 included, is an error: for a repo the run REQUIRES there
+// is no benign reading of "we cannot see it".
+func FetchRepoFiles(client *http.Client, repo string) ([]GGUFFile, error) {
+	files, _, err := fetchRepoFiles(client, repo)
+	return files, err
+}
+
+// fetchRepoFiles does the request and returns the HTTP status alongside the
+// result, so the optional and required callers can apply different policies to
+// the same response without duplicating the request.
+func fetchRepoFiles(client *http.Client, repo string) ([]GGUFFile, int, error) {
+	url := fmt.Sprintf("https://huggingface.co/api/models/%s?blobs=true", repo)
+	req, err := http.NewRequest(http.MethodGet, url, nil)
+	if err != nil {
+		return nil, 0, err
+	}
+	req.Header.Set("User-Agent", "localai-apexentries/1.0")
+
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, 0, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode == http.StatusNotFound {
+		return nil, resp.StatusCode, nil
+	}
+	if resp.StatusCode != http.StatusOK {
+		return nil, resp.StatusCode, fmt.Errorf("%s: unexpected status %d", repo, resp.StatusCode)
+	}
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return nil, resp.StatusCode, err
+	}
+	files, err := ParseRepoFiles(body)
+	return files, resp.StatusCode, err
+}
+
+// newHTTPClient builds the client used against the HuggingFace API. It goes
+// through pkg/httpclient rather than a bare &http.Client{} because the std
+// client follows redirects and forwards custom credential headers to the
+// redirect target on a cross-host hop (GHSA-3mj3-57v2-4636). This caller sends
+// only a User-Agent today, but it talks to an external API that could start
+// redirecting, and an HF_TOKEN header here later would then leak.
+func newHTTPClient() *http.Client {
+	return httpclient.NewWithTimeout(60 * time.Second)
+}

.github/ci/apexentries/hf_test.go

@@ -0,0 +1,142 @@
+package main
+
+import (
+	"bytes"
+	"io"
+	"net/http"
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestApexEntries(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "apexentries")
+}
+
+// stubTransport answers every request with one canned status and body, so the
+// status handling of the fetchers can be exercised without reaching the real
+// HuggingFace API.
+type stubTransport struct {
+	status int
+	body   string
+}
+
+func (t stubTransport) RoundTrip(req *http.Request) (*http.Response, error) {
+	return &http.Response{
+		StatusCode: t.status,
+		Body:       io.NopCloser(bytes.NewBufferString(t.body)),
+		Header:     make(http.Header),
+		Request:    req,
+	}, nil
+}
+
+func stubClient(status int, body string) *http.Client {
+	return &http.Client{Transport: stubTransport{status: status, body: body}}
+}
+
+const oneGGUFBody = `{"siblings":[{"rfilename":"Model-APEX-I-Quality.gguf","size":10,"lfs":{"sha256":"aa","size":10}}]}`
+
+var _ = Describe("FetchOptionalRepoFiles", func() {
+	// HuggingFace answers 401 rather than 404 for a repo that does not exist
+	// when the client carries no credentials, so an optional probe cannot tell
+	// "absent" from "unauthorized" and must treat both as "no counterpart".
+	It("treats a 401 as an absent repo and flags it as unavailable", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusUnauthorized, ""), "unsloth/Nope-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+		Expect(unavailable).To(BeTrue())
+	})
+
+	It("treats a 403 as an absent repo and flags it as unavailable", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusForbidden, ""), "unsloth/Gated-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+		Expect(unavailable).To(BeTrue())
+	})
+
+	// A clean 404 is an unambiguous absence, so it must NOT be reported as
+	// unavailable: the whole point of the flag is to separate the ambiguous
+	// case a human may need to look at from the settled one.
+	It("treats a 404 as an absent repo without flagging it as unavailable", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusNotFound, ""), "unsloth/Nope-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+		Expect(unavailable).To(BeFalse())
+	})
+
+	It("parses a 200 body as usual", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusOK, oneGGUFBody), "unsloth/Real-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(unavailable).To(BeFalse())
+		Expect(files).To(HaveLen(1))
+		Expect(files[0].Name).To(Equal("Model-APEX-I-Quality.gguf"))
+		Expect(files[0].SHA256).To(Equal("aa"))
+	})
+
+	// Tolerating 401/403 must not widen into tolerating everything: a 500 is a
+	// broken API, not evidence about whether the repo exists.
+	It("still errors on a 500", func() {
+		_, _, err := FetchOptionalRepoFiles(stubClient(http.StatusInternalServerError, ""), "unsloth/Real-GGUF")
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unexpected status 500"))
+	})
+})
+
+var _ = Describe("FetchRepoFiles", func() {
+	// The APEX repo itself is not optional. A 401 there means the repo the run
+	// was asked to publish cannot be read, which is a real failure and must not
+	// be quietly downgraded to "no files".
+	It("errors on a 401 for a required repo", func() {
+		_, err := FetchRepoFiles(stubClient(http.StatusUnauthorized, ""), "mudler/Model-APEX-GGUF")
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unexpected status 401"))
+	})
+
+	It("errors on a 403 for a required repo", func() {
+		_, err := FetchRepoFiles(stubClient(http.StatusForbidden, ""), "mudler/Model-APEX-GGUF")
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unexpected status 403"))
+	})
+
+	It("still treats a 404 as an absent repo", func() {
+		files, err := FetchRepoFiles(stubClient(http.StatusNotFound, ""), "mudler/Model-APEX-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+	})
+})
+
+var _ = Describe("ParseRepoFiles", func() {
+	It("returns gguf siblings with their lfs sha256", func() {
+		body := []byte(`{"siblings":[
+			{"rfilename":"Model-APEX-I-Quality.gguf","size":10,"lfs":{"sha256":"aa","size":10}},
+			{"rfilename":"README.md"},
+			{"rfilename":"mmproj.gguf","size":5,"lfs":{"sha256":"bb","size":5}}
+		]}`)
+
+		files, err := ParseRepoFiles(body)
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(HaveLen(2))
+		Expect(files[0].Name).To(Equal("Model-APEX-I-Quality.gguf"))
+		Expect(files[0].SHA256).To(Equal("aa"))
+		Expect(files[1].Name).To(Equal("mmproj.gguf"))
+	})
+
+	It("reports a gguf that carries no lfs sha256", func() {
+		body := []byte(`{"siblings":[{"rfilename":"mmproj.gguf","size":5}]}`)
+
+		_, err := ParseRepoFiles(body)
+
+		Expect(err).To(MatchError(ErrNoSHA256))
+	})
+})

.github/ci/apexentries/hub.go

@@ -0,0 +1,143 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+// IndexText is the gallery index seen as text: the entries it declares plus the
+// exact lines each one occupies, which is what splicing a variants block into an
+// entry the gallery already ships requires.
+//
+// It is a second, narrower read of the same file LoadExisting parses. The two
+// answer different questions: LoadExisting answers "do these weights already
+// exist anywhere", this one answers "where in the file does this entry live".
+type IndexText struct {
+	Lines   []string
+	Entries []*indexEntry
+
+	byName map[string]*indexEntry
+}
+
+// indexEntry is one entry of the index: its name, the variants it already
+// declares, and its coordinates in the file.
+type indexEntry struct {
+	Name     string       `yaml:"name"`
+	Variants []VariantRef `yaml:"variants"`
+
+	Pos galleryedit.Entry `yaml:"-"`
+}
+
+// LoadIndexText reads the gallery index for editing.
+func LoadIndexText(path string) (*IndexText, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return nil, err
+	}
+	return ParseIndexText(string(raw))
+}
+
+// ParseIndexText pairs the decoded entries with the top level list items the
+// text actually contains.
+//
+// If the two views disagree on how many entries there are then every line number
+// a splice would compute is suspect, and the failure mode is writing a variants
+// block into the wrong model. The parse refuses instead.
+func ParseIndexText(text string) (*IndexText, error) {
+	var entries []*indexEntry
+	if err := yaml.Unmarshal([]byte(text), &entries); err != nil {
+		return nil, fmt.Errorf("decoding gallery index: %w", err)
+	}
+
+	lines, starts := galleryedit.Scan(text)
+	if len(starts) != len(entries) {
+		return nil, fmt.Errorf("gallery index has %d decoded entries but %d top level list items; refusing to edit by line number",
+			len(entries), len(starts))
+	}
+
+	ix := &IndexText{Lines: lines, Entries: entries, byName: map[string]*indexEntry{}}
+	for i, e := range entries {
+		if e == nil {
+			return nil, fmt.Errorf("gallery index list item %d is empty; refusing to edit by line number", i)
+		}
+		end := len(lines)
+		if i+1 < len(starts) {
+			end = starts[i+1]
+		}
+		e.Pos = galleryedit.Entry{Name: e.Name, StartLine: starts[i], EndLine: end}
+
+		// First occurrence wins, matching the gallery's own resolution.
+		key := strings.ToLower(e.Name)
+		if _, seen := ix.byName[key]; !seen {
+			ix.byName[key] = e
+		}
+	}
+	return ix, nil
+}
+
+// Find looks an entry up by name, case insensitively.
+func (ix *IndexText) Find(name string) *indexEntry {
+	return ix.byName[strings.ToLower(name)]
+}
+
+// ResolveHub returns the gallery name of a family's hub and whether the gallery
+// already ships an entry under it.
+//
+// The hub is the BASE model entry, never a generated *-apex parent. Somebody
+// looking for qwen3.6-35b-a3b has to find every build of those weights under
+// that one name: the APEX imatrix rungs, the unsloth quant rungs and any
+// speculative build. A separate qwen3.6-35b-a3b-apex hub competing with the base
+// entry would split the family in two and leave whichever half the user did not
+// search for invisible.
+//
+// Both candidates are tried for the same reason CounterpartCandidates tries
+// both. The repo name and the published file stem disagree for several of these
+// repos, and either one may be what the base entry was named after.
+func ResolveHub(ix *IndexText, repoBase, stem string) (name string, exists bool) {
+	candidates := CounterpartCandidates(repoBase, stem)
+	for _, c := range candidates {
+		if n := slug(c); ix.Find(n) != nil {
+			return n, true
+		}
+	}
+	// Nothing matched, so the family needs a hub of its own under the repo
+	// derived name, which is the more reliable of the two.
+	return slug(candidates[0]), false
+}
+
+// HubLabel is the human-cased base model name, for prose rather than lookup.
+func HubLabel(repoBase, stem string) string {
+	return CounterpartCandidates(repoBase, stem)[0]
+}
+
+// filterVariants drops the references a hub must not carry: itself, and anything
+// it already lists.
+//
+// The self reference is not merely redundant. A hub that names itself makes the
+// verifier resolve the reference back to the hub, see that the hub declares
+// variants, and report a variant that declares variants of its own. It arises
+// for real rather than in theory: an unsloth rung whose weights the gallery
+// already ships under the base model name resolves, through Merge, straight back
+// to the hub that is about to reference it.
+func filterVariants(hub string, already []VariantRef, want []string) []string {
+	seen := map[string]bool{strings.ToLower(hub): true}
+	for _, v := range already {
+		seen[strings.ToLower(v.Model)] = true
+	}
+
+	var out []string
+	for _, w := range want {
+		key := strings.ToLower(w)
+		if seen[key] {
+			continue
+		}
+		seen[key] = true
+		out = append(out, w)
+	}
+	return out
+}

.github/ci/apexentries/main.go

@@ -0,0 +1,795 @@
+// Command apexentries generates gallery entries for the mudler APEX GGUF
+// repositories: one entry per imatrix tier and per unsloth quant rung, all
+// gathered under the BASE model's entry. Builds off a *-APEX-MTP-GGUF repo turn
+// speculative decoding on, because those weights retain the model's MTP heads
+// and are only worth their extra size with the heads in use.
+//
+// The base model entry is the hub. Somebody looking for qwen3.6-35b-a3b must
+// find every build of those weights under that one name, so when the gallery
+// already ships the base entry this command splices a variants block into it
+// rather than emitting a competing *-apex parent beside it. Only a family whose
+// base model the gallery does not ship at all gets a new hub entry, and that one
+// is still named for the base model.
+//
+// Builds are discovered by inspecting the filenames a repo actually publishes.
+// Repo names do not reliably predict them: mudler/gemma-4-26B-A4B-it-APEX-GGUF
+// ships gemma-4-26B-A4B-APEX-*.gguf, and six of the 45 repos drop a suffix or a
+// vendor prefix in the same way.
+package main
+
+import (
+	"encoding/json"
+	"flag"
+	"fmt"
+	"io"
+	"net/http"
+	"os"
+	"path"
+	"sort"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+const (
+	// entryTemplate carries no backend and no parameters of its own, which is
+	// why RenderChild states everything inline.
+	entryTemplate = "virtual.yaml"
+	unslothOwner  = "unsloth"
+	authorListURL = "https://huggingface.co/api/models?author=mudler&limit=300"
+)
+
+// rungRank orders the quality ladder from best to smallest. The HuggingFace API
+// returns siblings alphabetically and DiscoverAPEXTiers preserves that order, so
+// an unsorted variants list reads I-Balanced, I-Compact, I-Mini, I-Nano,
+// I-Quality. Selection ignores authored order, so this is purely so the file a
+// human reviews scans in a meaningful sequence.
+var rungRank = map[string]int{
+	"I-Quality": 0, "I-Balanced": 1, "I-Compact": 2, "I-Mini": 3, "I-Nano": 4,
+	"Quality": 5, "Balanced": 6, "Compact": 7, "Mini": 8, "Nano": 9,
+}
+
+// baseTags are the tags every generated entry carries. dflash and mtp are never
+// among them: RenderChild adds those if and only if the entry configures the
+// matching spec_type.
+var baseTags = []string{"llm", "gguf", "cpu", "gpu"}
+
+// childBuild pairs a rendered entry with its position on the quality ladder, so
+// the parent's variants list can be sorted without re-parsing entry names.
+type childBuild struct {
+	entry GalleryEntry
+	rank  int
+}
+
+// family is one APEX repo's full generated output.
+type family struct {
+	repo      string
+	repoBase  string
+	stem      string
+	hasMMProj bool
+	children  []childBuild
+
+	// skippedRepos are counterpart candidates HuggingFace would not describe.
+	// Carried on the family rather than printed and forgotten so the run can
+	// summarize them next to everything else a reviewer has to eyeball.
+	skippedRepos []string
+	census       fileCensus
+	unaccounted  int
+}
+
+// fileCensus splits the files discovery emitted nothing for into the ones a
+// reviewer must chase and the ones that are deliberately out of scope.
+//
+// Full-precision sources are the second kind: they are the unquantized weights
+// the ladder is derived FROM, not a rung of it. Folding them into the
+// unclassified total would leave a permanent benign baseline, and a permanent
+// baseline is exactly what hides the one file that ever genuinely matters.
+type fileCensus struct {
+	unclassified  int
+	fullPrecision int
+}
+
+// add accumulates one repo's census into a running total.
+func (c *fileCensus) add(o fileCensus) {
+	c.unclassified += o.unclassified
+	c.fullPrecision += o.fullPrecision
+}
+
+// sortedChildren returns the family's builds in ladder order, best first.
+func (f *family) sortedChildren() []childBuild {
+	sorted := append([]childBuild{}, f.children...)
+	sort.SliceStable(sorted, func(i, j int) bool { return sorted[i].rank < sorted[j].rank })
+	return sorted
+}
+
+func main() {
+	verify := flag.String("verify", "", "verify a gallery index and exit")
+	index := flag.String("index", "gallery/index.yaml", "gallery index to dedup against")
+	only := flag.String("only", "", "comma-separated repo names to restrict generation to")
+	out := flag.String("out", "", "write the entries to add to this file")
+	apply := flag.Bool("apply", false, "append the entries to add to -index")
+	flag.Parse()
+
+	if *verify != "" {
+		problems := Verify(*verify)
+		for _, p := range problems {
+			fmt.Fprintln(os.Stderr, p)
+		}
+		if len(problems) > 0 {
+			fmt.Fprintf(os.Stderr, "%d problem(s)\n", len(problems))
+			os.Exit(1)
+		}
+		fmt.Println("index is sound")
+		return
+	}
+
+	if err := generate(*index, *only, *out, *apply); err != nil {
+		fmt.Fprintln(os.Stderr, "error:", err)
+		os.Exit(1)
+	}
+}
+
+func generate(indexPath, only, outPath string, apply bool) error {
+	if outPath == "" && !apply {
+		return fmt.Errorf("nothing to do: pass -out <file> or -apply")
+	}
+
+	client := newHTTPClient()
+
+	repos, err := listAPEXRepos(client)
+	if err != nil {
+		return err
+	}
+	if only != "" {
+		repos = restrict(repos, only)
+	}
+	if len(repos) == 0 {
+		return fmt.Errorf("no APEX repos selected")
+	}
+	fmt.Printf("repos selected: %d\n", len(repos))
+
+	var families []family
+	var failed []string
+
+	for _, repo := range repos {
+		f, err := buildFamily(client, repo)
+		if err != nil {
+			// A missing sha256 is fatal for the family rather than skippable: an
+			// entry without one ships an unverifiable download. Report which repo
+			// and keep going, so one bad repo does not hide the state of the rest.
+			fmt.Fprintf(os.Stderr, "FAILED %s: %v\n", repo, err)
+			failed = append(failed, repo)
+			continue
+		}
+		families = append(families, *f)
+	}
+
+	existing, err := LoadExisting(indexPath)
+	if err != nil {
+		return err
+	}
+	ixText, err := LoadIndexText(indexPath)
+	if err != nil {
+		return err
+	}
+	fmt.Printf("existing index: %d names, %d weight URIs, %d lines\n",
+		len(existing.ByName), len(existing.ByURI), len(ixText.Lines))
+
+	// Only the builds go through Merge. A hub is deliberately kept out of it: a
+	// new hub carries the family's top rung as its own payload, so Merge's URI
+	// dedup would fold the hub into that rung and the family would lose the very
+	// entry point this command exists to create. Hub names are checked against
+	// the index directly, by ResolveHub.
+	var generated []GalleryEntry
+	for _, f := range families {
+		for _, c := range f.children {
+			generated = append(generated, c.entry)
+		}
+	}
+
+	add, reused := Merge(existing, generated)
+	reportReuse(existing, generated, reused)
+
+	// Variant references are resolved from `reused`, never used to decide what to
+	// emit: on a within-batch name collision Merge records reused[name] = name
+	// while the first entry of that name is still in `add`, so treating presence
+	// in `reused` as "dropped" would silently emit nothing for it.
+	added := map[string]bool{}
+	for _, e := range add {
+		added[e.Name] = true
+	}
+
+	inserts, newHubs, err := planHubs(families, ixText, reused, added)
+	if err != nil {
+		return err
+	}
+	reportHubs(ixText, inserts, newHubs)
+
+	skipped, census, fullPrecisionRepos, unaccounted := reportSkipped(families)
+
+	add = append(add, newHubs...)
+
+	fmt.Printf("\nentries generated: %d\nentries to add:    %d\nentries reused:    %d\nhubs spliced:      %d\nhubs created:      %d\nrepos skipped:     %d\nexcluded (full precision): %d files across %d repos\nunclassified:      %d\nunaccounted:       %d\n",
+		len(generated), len(add), len(reused), len(inserts), len(newHubs), len(skipped),
+		census.fullPrecision, fullPrecisionRepos, census.unclassified, unaccounted)
+
+	lines, err := galleryedit.Apply(ixText.Lines, inserts)
+	if err != nil {
+		return err
+	}
+
+	if err := writeEntries(add, lines, outPath, apply, indexPath); err != nil {
+		return err
+	}
+
+	if len(failed) > 0 {
+		return fmt.Errorf("%d repo(s) failed: %s", len(failed), strings.Join(failed, ", "))
+	}
+	return nil
+}
+
+// resolveVariant maps a generated child name onto whatever entry actually stands
+// for it after the merge. `added` is consulted first because a within-batch name
+// collision puts a name in BOTH add and reused, and the entry that was emitted
+// is the one the parent must reference.
+func resolveVariant(name string, reused map[string]string, added map[string]bool) string {
+	if added[name] {
+		return name
+	}
+	if target, ok := reused[name]; ok {
+		return target
+	}
+	return name
+}
+
+// SpecTypeForRepo reports the speculative decoding mechanism a repo's builds can
+// turn on with no extra download.
+//
+// The *-APEX-MTP-GGUF repos republish the base weights with the model's own MTP
+// heads retained, so those builds are only worth their extra size if the heads
+// are actually used. Every other APEX repo drops them, and switching MTP on
+// there would name a mechanism the weights cannot serve.
+//
+// The suffix is read off the repo the FILES come from, so nothing downstream has
+// to infer a capability from an entry name.
+func SpecTypeForRepo(repo string) string {
+	if strings.HasSuffix(path.Base(repo), "-APEX-MTP-GGUF") {
+		return "draft-mtp"
+	}
+	return ""
+}
+
+// buildFamily discovers everything one APEX repo and its unsloth counterpart
+// publish, and renders it.
+func buildFamily(client *http.Client, repo string) (*family, error) {
+	files, err := FetchRepoFiles(client, repo)
+	if err != nil {
+		return nil, err
+	}
+	if len(files) == 0 {
+		return nil, fmt.Errorf("no gguf files")
+	}
+
+	imatrix, plain := DiscoverAPEXTiers(files)
+	mmproj, hasMMProj := DiscoverMMProj(files)
+
+	census := reportUnclassified(repo, files, imatrix, plain)
+
+	// The imatrix ladder is preferred, but two of the 45 repos publish no
+	// imatrix tiers at all and must still contribute their plain ladder.
+	ladder := imatrix
+	ladderKind := "imatrix"
+	if len(ladder) == 0 {
+		ladder = plain
+		ladderKind = "plain"
+	}
+	if len(ladder) == 0 {
+		return nil, fmt.Errorf("no tiers discovered")
+	}
+
+	sortTiers(ladder)
+
+	var mm *GGUFFile
+	if hasMMProj {
+		mm = &mmproj
+	}
+
+	repoBase := strings.TrimSuffix(path.Base(repo), "-GGUF")
+	f := &family{repo: repo, repoBase: repoBase, hasMMProj: hasMMProj, census: census}
+
+	// Only the APEX ladder can carry MTP heads; the unsloth counterpart quantizes
+	// the plain weights and gets nothing from this.
+	specType := SpecTypeForRepo(repo)
+
+	for _, t := range ladder {
+		f.children = append(f.children, childBuild{
+			rank: rungRank[t.Label],
+			entry: RenderChild(ChildInput{
+				Name:     slug(repoBase) + "-" + slug(t.Label),
+				Repo:     repo,
+				Template: entryTemplate,
+				SpecType: specType,
+				Weights:  []GGUFFile{t.File},
+				MMProj:   mm,
+				BaseTags: baseTags,
+			}),
+		})
+	}
+
+	stem := FileStem(ladder[0])
+	f.stem = stem
+	fmt.Printf("%s: %d %s tier(s) [%s], stem %s, mmproj %v\n",
+		repo, len(ladder), ladderKind, tierLabels(ladder), stem, hasMMProj)
+
+	counterpart, cpFiles, skipped, err := resolveCounterpart(client, repoBase, stem)
+	f.skippedRepos = skipped
+	if err != nil {
+		return nil, err
+	}
+	if counterpart != "" {
+		builds := DiscoverUnslothQuants(cpFiles)
+
+		// Called here rather than inside Verify: a quant dropped at discovery
+		// leaves no trace at all in the finished gallery file, so the only place
+		// the shortfall is still visible is the moment of discovery.
+		unaccounted := UnaccountedQuants(cpFiles, builds)
+		f.unaccounted = len(unaccounted)
+		for _, p := range unaccounted {
+			fmt.Fprintf(os.Stderr, "UNACCOUNTED QUANT %s: %s\n", counterpart, p)
+		}
+
+		cpMMProj, hasCPMMProj := DiscoverMMProj(cpFiles)
+		var cpMM *GGUFFile
+		if hasCPMMProj {
+			cpMM = &cpMMProj
+		}
+		cpBase := strings.TrimSuffix(path.Base(counterpart), "-GGUF")
+
+		for i, b := range builds {
+			f.children = append(f.children, childBuild{
+				rank: 100 + i,
+				entry: RenderChild(ChildInput{
+					Name:     slug(cpBase) + "-" + slug(b.Quant),
+					Repo:     counterpart,
+					Template: entryTemplate,
+					Weights:  b.Files,
+					MMProj:   cpMM,
+					BaseTags: baseTags,
+				}),
+			})
+		}
+		fmt.Printf("%s: counterpart %s, %d quant build(s) %s\n", repo, counterpart, len(builds), quantLabels(builds))
+	} else {
+		fmt.Printf("%s: no unsloth counterpart\n", repo)
+	}
+
+	return f, nil
+}
+
+// planHubs decides, per family, whether the family's builds are spliced into a
+// base model entry the gallery already ships or gathered under a new hub.
+//
+// Splicing is strongly preferred and is the measured majority-adjacent case. The
+// existing entry keeps its description, icon, tags, overrides and files
+// untouched; only variant lines are added to it.
+func planHubs(families []family, ix *IndexText, reused map[string]string, added map[string]bool) ([]galleryedit.Insert, []GalleryEntry, error) {
+	// Several APEX repos can resolve to one base model, so both paths accumulate
+	// by hub name rather than assuming one family per hub.
+	wantByHub := map[string][]string{}
+	var spliceOrder []string
+
+	var newHubs []GalleryEntry
+	hubAt := map[string]int{}
+
+	for i := range families {
+		f := &families[i]
+
+		hubName, exists := ResolveHub(ix, f.repoBase, f.stem)
+		want := hubVariants(f, ix, reused, added)
+
+		if exists {
+			if _, seen := wantByHub[hubName]; !seen {
+				spliceOrder = append(spliceOrder, hubName)
+			}
+			wantByHub[hubName] = append(wantByHub[hubName], want...)
+			continue
+		}
+
+		if at, dup := hubAt[hubName]; dup {
+			for _, v := range filterVariants(hubName, newHubs[at].Variants, want) {
+				newHubs[at].Variants = append(newHubs[at].Variants, VariantRef{Model: v})
+			}
+			continue
+		}
+
+		builds := f.sortedChildren()
+		if len(builds) == 0 {
+			return nil, nil, fmt.Errorf("%s: no builds to hang a hub on", f.repo)
+		}
+		hubAt[hubName] = len(newHubs)
+		newHubs = append(newHubs, renderHub(hubName, f, builds[0], filterVariants(hubName, nil, want)))
+	}
+
+	var inserts []galleryedit.Insert
+	for _, name := range spliceOrder {
+		e := ix.Find(name)
+		items := filterVariants(name, e.Variants, wantByHub[name])
+		if len(items) == 0 {
+			continue
+		}
+		inserts = append(inserts, galleryedit.Insert{Entry: e.Pos, Variants: items})
+	}
+	return inserts, newHubs, nil
+}
+
+// hubVariants is a family's full build list, in ladder order, named as the hub
+// must reference them after the merge.
+func hubVariants(f *family, ix *IndexText, reused map[string]string, added map[string]bool) []string {
+	var out []string
+
+	// A hand-written *-apex entry is an ordinary build of these weights. It is
+	// never deleted, never renamed and never treated as a hub; it is simply
+	// referenced like any other rung.
+	if apex := slug(f.repoBase); ix.Find(apex) != nil {
+		out = append(out, apex)
+	}
+	for _, c := range f.sortedChildren() {
+		out = append(out, resolveVariant(c.entry.Name, reused, added))
+	}
+	return out
+}
+
+// renderHub builds the hub for a family whose base model the gallery does not
+// ship at all. It is named for the BASE model, never for the APEX repo.
+//
+// It carries one of the discovered builds as its own payload so it is a complete
+// installable entry rather than a bare index pointing at other entries. That
+// payload is what supplies overrides.backend, which matters beyond installation:
+// the verifier can only judge the tagging rule for a backend it can read, so a
+// hub carrying feature tags and no backend would escape the check in silence.
+//
+// The payload's own tags are kept rather than rebuilt from baseTags, so a hub
+// whose payload configures a spec_type stays tagged for it and consistent with
+// the overrides copied alongside.
+func renderHub(name string, f *family, payload childBuild, variants []string) GalleryEntry {
+	e := payload.entry
+	e.Name = name
+	e.Description = fmt.Sprintf(
+		"%s. Quality ladder and quantization rungs published by %s and its unsloth counterpart; LocalAI picks the build that fits the hardware.",
+		HubLabel(f.repoBase, f.stem), f.repo)
+
+	e.Tags = append([]string{}, payload.entry.Tags...)
+	if f.hasMMProj && !hasTag(e.Tags, "vision") {
+		e.Tags = append(e.Tags, "vision")
+	}
+
+	e.Variants = nil
+	for _, v := range variants {
+		e.Variants = append(e.Variants, VariantRef{Model: v})
+	}
+	return e
+}
+
+func hasTag(tags []string, want string) bool {
+	for _, t := range tags {
+		if t == want {
+			return true
+		}
+	}
+	return false
+}
+
+// resolveCounterpart probes the unsloth candidates in order and returns the
+// first that publishes files.
+//
+// CounterpartCandidates is handed a BARE repo name: its cleaner does not strip
+// an owner prefix, so passing "mudler/Foo-APEX-GGUF" would yield "mudler/Foo"
+// and compose into the nonsense probe "unsloth/mudler/Foo".
+//
+// It also returns the candidates HuggingFace refused to describe. Those are
+// indistinguishable from absent without credentials, so they are skipped, but
+// they are named rather than dropped: one of them could be a real gated repo
+// whose quants belong in the gallery.
+func resolveCounterpart(client *http.Client, repoBase, stem string) (string, []GGUFFile, []string, error) {
+	var unavailable []string
+	for _, cand := range CounterpartCandidates(repoBase, stem) {
+		repo := unslothOwner + "/" + cand + "-GGUF"
+		files, unreadable, err := FetchOptionalRepoFiles(client, repo)
+		if err != nil {
+			return "", nil, unavailable, fmt.Errorf("probing %s: %w", repo, err)
+		}
+		if unreadable {
+			unavailable = append(unavailable, repo)
+			continue
+		}
+		if len(files) > 0 {
+			return repo, files, unavailable, nil
+		}
+	}
+	return "", nil, unavailable, nil
+}
+
+// reportUnclassified prints the files discovery turned into nothing.
+//
+// It is a set difference on COUNTS, not a re-match of filenames: re-matching
+// would duplicate the tier regex from discover.go and the two copies would
+// drift. The likeliest trigger is a typo or case change from a publishing script
+// rather than a genuine sixth tier, and because generation falls back to the
+// plain ladder when the imatrix one is empty, a repo whose imatrix files all
+// fail to match silently downgrades the whole family instead of erroring. The
+// downstream HTTP check cannot catch that: it validates URLs that were emitted,
+// and an undiscovered tier emits none.
+// It returns the census so the run can total it.
+func reportUnclassified(repo string, files []GGUFFile, imatrix, plain []Tier) fileCensus {
+	mmprojCount, fullPrecision := 0, 0
+	for _, f := range files {
+		// The mmproj test comes first because projectors are themselves often
+		// published at f16 (mmproj-F16.gguf), and counting such a file in both
+		// buckets would understate the unclassified remainder.
+		if strings.HasPrefix(f.Name, "mmproj") {
+			mmprojCount++
+			continue
+		}
+		if IsFullPrecision(f.Name) {
+			fullPrecision++
+		}
+	}
+
+	classified := len(imatrix) + len(plain) + mmprojCount + fullPrecision
+	if classified >= len(files) {
+		return fileCensus{fullPrecision: fullPrecision}
+	}
+	fmt.Fprintf(os.Stderr, "UNCLASSIFIED %s: %d of %d .gguf files classified, %d unaccounted for\n",
+		repo, classified, len(files), len(files)-classified)
+	return fileCensus{unclassified: len(files) - classified, fullPrecision: fullPrecision}
+}
+
+// reportReuse splits Merge's single reused map into the two cases it conflates.
+//
+// A URI match means the gallery already ships exactly these weights, and
+// pointing the parent at the existing entry is correct. A NAME match with a
+// different URI means an unrelated entry happens to own the name, and
+// referencing it would point the parent at different weights than were
+// generated, substituting a build without saying so. Only the first is safe to
+// wave through.
+func reportReuse(existing *ExistingIndex, generated []GalleryEntry, reused map[string]string) {
+	byName := map[string]GalleryEntry{}
+	for _, e := range generated {
+		if _, seen := byName[e.Name]; !seen {
+			byName[e.Name] = e
+		}
+	}
+
+	var nameCollisions, uriMatches []string
+	for name, target := range reused {
+		gen := byName[name]
+		uri := ""
+		if len(gen.Files) > 0 {
+			uri = gen.Files[0].URI
+		}
+
+		switch {
+		case hasName(existing, name):
+			nameCollisions = append(nameCollisions,
+				fmt.Sprintf("  %s -> gallery entry of the same name (generated uri: %s)", name, orNone(uri)))
+		case target == name:
+			nameCollisions = append(nameCollisions,
+				fmt.Sprintf("  %s -> earlier entry of the same name in this batch (generated uri: %s)", name, orNone(uri)))
+		default:
+			uriMatches = append(uriMatches, fmt.Sprintf("  %s -> %s (same weights: %s)", name, target, orNone(uri)))
+		}
+	}
+	sort.Strings(nameCollisions)
+	sort.Strings(uriMatches)
+
+	fmt.Printf("\nNAME COLLISIONS (%d) - inspect each by hand, the target may hold different weights\n", len(nameCollisions))
+	for _, l := range nameCollisions {
+		fmt.Println(l)
+	}
+	fmt.Printf("\nURI MATCHES (%d) - the gallery or this batch already ships these exact weights\n", len(uriMatches))
+	for _, l := range uriMatches {
+		fmt.Println(l)
+	}
+}
+
+// reportHubs prints exactly what will be written where. The splices are the part
+// a human has to read: they modify entries the gallery already ships, so the
+// review needs the target, the line, and every added reference spelled out.
+func reportHubs(ix *IndexText, inserts []galleryedit.Insert, newHubs []GalleryEntry) {
+	fmt.Printf("\nHUBS SPLICED (%d) - variants added to the EXISTING base model entry, nothing else touched\n", len(inserts))
+	for _, in := range inserts {
+		e := ix.Find(in.Entry.Name)
+		fmt.Printf("  %s (line %d, %d variant(s) already declared):\n", in.Entry.Name, in.Entry.StartLine+1, len(e.Variants))
+		for _, v := range in.Variants {
+			fmt.Printf("    + - model: %s\n", galleryedit.QuoteName(v))
+		}
+	}
+
+	fmt.Printf("\nHUBS CREATED (%d) - the gallery ships no base model entry, so one is emitted for it\n", len(newHubs))
+	for _, h := range newHubs {
+		fmt.Printf("  %s:\n", h.Name)
+		for _, v := range h.Variants {
+			fmt.Printf("    - model: %s\n", v.Model)
+		}
+	}
+}
+
+// reportSkipped names the counterpart repos HuggingFace would not describe, and
+// totals the other two silent-shortfall counters alongside them.
+//
+// A skipped repo is not the same as a clean 404. HuggingFace answers 401 for a
+// nonexistent repo to an unauthenticated client, so the overwhelmingly likely
+// reading is "there is no such counterpart", which is the normal case for the
+// community merges. But a private or gated repo answers 401 too, and that one
+// WOULD have quants worth shipping. Printing the list is what keeps that
+// possibility auditable instead of silently discarded.
+func reportSkipped(families []family) ([]string, fileCensus, int, int) {
+	var skipped []string
+	var census fileCensus
+	fullPrecisionRepos, unaccounted := 0, 0
+	for _, f := range families {
+		skipped = append(skipped, f.skippedRepos...)
+		census.add(f.census)
+		if f.census.fullPrecision > 0 {
+			fullPrecisionRepos++
+		}
+		unaccounted += f.unaccounted
+	}
+	sort.Strings(skipped)
+
+	fmt.Printf("\nREPOS SKIPPED AS UNAVAILABLE (%d) - HuggingFace answered 401/403, which is indistinguishable from absent without a token; check none of these is a real gated repo\n", len(skipped))
+	for _, r := range skipped {
+		fmt.Printf("  %s\n", r)
+	}
+	return skipped, census, fullPrecisionRepos, unaccounted
+}
+
+func hasName(ix *ExistingIndex, name string) bool {
+	_, ok := ix.ByName[name]
+	return ok
+}
+
+func orNone(s string) string {
+	if s == "" {
+		return "(no files)"
+	}
+	return s
+}
+
+// writeEntries emits the additions.
+//
+// -apply does two things in one pass: it writes back the spliced lines, which
+// differ from the original only by the variant lines galleryedit inserted, and
+// then appends the new entries. New entries are APPENDED rather than merged into
+// the structure, for the same reason the splice is textual: a YAML round trip
+// over 40,000 lines would reflow the whole file into an unreviewable diff.
+func writeEntries(add []GalleryEntry, lines []string, outPath string, apply bool, indexPath string) error {
+	if apply {
+		if err := os.WriteFile(indexPath, []byte(strings.Join(lines, "\n")), 0o644); err != nil {
+			return err
+		}
+		fmt.Printf("spliced %s\n", indexPath)
+	}
+
+	if len(add) == 0 {
+		fmt.Println("nothing to append")
+		return nil
+	}
+
+	blob, err := yaml.Marshal(add)
+	if err != nil {
+		return err
+	}
+
+	if outPath != "" {
+		if err := os.WriteFile(outPath, blob, 0o644); err != nil {
+			return err
+		}
+		fmt.Printf("wrote %d entries to %s\n", len(add), outPath)
+	}
+
+	if apply {
+		f, err := os.OpenFile(indexPath, os.O_APPEND|os.O_WRONLY, 0o644)
+		if err != nil {
+			return err
+		}
+		defer f.Close()
+		if _, err := f.Write(blob); err != nil {
+			return err
+		}
+		fmt.Printf("appended %d entries to %s\n", len(add), indexPath)
+	}
+	return nil
+}
+
+// listAPEXRepos returns the mudler repos whose name marks them as APEX builds.
+func listAPEXRepos(client *http.Client) ([]string, error) {
+	req, err := http.NewRequest(http.MethodGet, authorListURL, nil)
+	if err != nil {
+		return nil, err
+	}
+	req.Header.Set("User-Agent", "localai-apexentries/1.0")
+
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != http.StatusOK {
+		return nil, fmt.Errorf("listing models: unexpected status %d", resp.StatusCode)
+	}
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return nil, err
+	}
+
+	var models []struct {
+		ID string `json:"id"`
+	}
+	if err := json.Unmarshal(body, &models); err != nil {
+		return nil, fmt.Errorf("decoding model list: %w", err)
+	}
+
+	var out []string
+	for _, m := range models {
+		if strings.Contains(m.ID, "APEX") {
+			out = append(out, m.ID)
+		}
+	}
+	sort.Strings(out)
+	return out, nil
+}
+
+func restrict(repos []string, only string) []string {
+	want := map[string]bool{}
+	for _, r := range strings.Split(only, ",") {
+		if r = strings.TrimSpace(r); r != "" {
+			want[r] = true
+		}
+	}
+
+	var out []string
+	for _, r := range repos {
+		if want[r] {
+			out = append(out, r)
+			delete(want, r)
+		}
+	}
+	// A name in -only that matched nothing is a typo, not an empty result.
+	for r := range want {
+		fmt.Fprintf(os.Stderr, "WARNING: -only names %s, which is not an APEX repo of this author\n", r)
+	}
+	return out
+}
+
+func sortTiers(tiers []Tier) {
+	sort.SliceStable(tiers, func(i, j int) bool { return rungRank[tiers[i].Label] < rungRank[tiers[j].Label] })
+}
+
+func tierLabels(tiers []Tier) string {
+	var out []string
+	for _, t := range tiers {
+		out = append(out, t.Label)
+	}
+	return strings.Join(out, ",")
+}
+
+func quantLabels(builds []QuantBuild) string {
+	var out []string
+	for _, b := range builds {
+		l := b.Quant
+		if b.Sharded {
+			l += fmt.Sprintf("(%d shards)", len(b.Files))
+		}
+		out = append(out, l)
+	}
+	return strings.Join(out, ",")
+}
+
+// slug turns a repo, tier or quant label into a gallery entry name component.
+func slug(s string) string {
+	return strings.ReplaceAll(strings.ToLower(s), "_", "-")
+}

.github/ci/apexentries/main_test.go

@@ -0,0 +1,344 @@
+package main
+
+import (
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+func mustIndex(text string) *IndexText {
+	ix, err := ParseIndexText(text)
+	ExpectWithOffset(1, err).ToNot(HaveOccurred())
+	return ix
+}
+
+// buildOf renders a realistic child so the specs exercise the payload a hub
+// actually inherits rather than a bare name.
+func buildOf(name, repo, file string, rank int) childBuild {
+	return childBuild{
+		rank: rank,
+		entry: RenderChild(ChildInput{
+			Name:     name,
+			Repo:     repo,
+			Template: entryTemplate,
+			Weights:  []GGUFFile{{Name: file, SHA256: "aa"}},
+			BaseTags: baseTags,
+		}),
+	}
+}
+
+var _ = Describe("ResolveHub", func() {
+	It("picks the base model name over the APEX name, even when both are in the gallery", func() {
+		// The hub is the entry a user searches for. If the *-apex entry were
+		// chosen the family would be gathered under a name nobody looks up, and
+		// the base entry would go on advertising only its own build.
+		ix := mustIndex("- name: qwen3.6-35b-a3b\n  url: u\n- name: qwen3.6-35b-a3b-apex\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "Qwen3.6-35B-A3B-APEX", "Qwen3.6-35B-A3B-APEX")
+
+		Expect(name).To(Equal("qwen3.6-35b-a3b"))
+		Expect(exists).To(BeTrue())
+	})
+
+	It("falls back to the stem-derived candidate when the repo-derived one is absent", func() {
+		// gemma's repo says "-it" and its published files do not, so only one of
+		// the two candidates can match whatever the base entry was named after.
+		ix := mustIndex("- name: gemma-4-26b-a4b\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "gemma-4-26B-A4B-it-APEX", "gemma-4-26B-A4B-APEX")
+
+		Expect(name).To(Equal("gemma-4-26b-a4b"))
+		Expect(exists).To(BeTrue())
+	})
+
+	It("reports the base name as absent rather than settling for the APEX entry", func() {
+		ix := mustIndex("- name: qwen3.5-35b-a3b-apex\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "Qwen3.5-35B-A3B-APEX", "Qwen3.5-35B-A3B-APEX")
+
+		Expect(name).To(Equal("qwen3.5-35b-a3b"))
+		Expect(exists).To(BeFalse())
+	})
+
+	It("strips the MTP and TQ markers as well as APEX", func() {
+		ix := mustIndex("- name: qwen3.6-35b-a3b\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "Qwen3.6-35B-A3B-APEX-MTP", "Qwen3.6-35B-A3B-APEX-MTP")
+
+		Expect(name).To(Equal("qwen3.6-35b-a3b"))
+		Expect(exists).To(BeTrue())
+	})
+})
+
+var _ = Describe("planHubs", func() {
+	noReuse := map[string]string{}
+	allAdded := func(names ...string) map[string]bool {
+		out := map[string]bool{}
+		for _, n := range names {
+			out[n] = true
+		}
+		return out
+	}
+
+	It("splices into the existing base entry instead of emitting an *-apex parent", func() {
+		ix := mustIndex("- name: step-3.7-flash\n  url: u\n- name: other\n  url: u\n")
+		fams := []family{{
+			repo:     "mudler/Step-3.7-Flash-APEX-GGUF",
+			repoBase: "Step-3.7-Flash-APEX",
+			stem:     "Step-3.7-Flash-APEX",
+			children: []childBuild{buildOf("step-3.7-flash-apex-i-quality", "mudler/Step-3.7-Flash-APEX-GGUF", "a.gguf", 0)},
+		}}
+
+		inserts, newHubs, err := planHubs(fams, ix, noReuse, allAdded("step-3.7-flash-apex-i-quality"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(newHubs).To(BeEmpty())
+		Expect(inserts).To(HaveLen(1))
+		Expect(inserts[0].Entry.Name).To(Equal("step-3.7-flash"))
+		Expect(inserts[0].Variants).To(Equal([]string{"step-3.7-flash-apex-i-quality"}))
+	})
+
+	It("merges into an entry that already declares variants, without repeating one", func() {
+		// The gallery's qwen3.6-35b-a3b already lists its APEX build. Re-adding it
+		// would put a duplicate key's worth of noise in the diff and a duplicate
+		// reference in the entry.
+		ix := mustIndex("- name: qwen3.6-35b-a3b\n  variants:\n    - model: qwen3.6-35b-a3b-apex\n  url: u\n" +
+			"- name: qwen3.6-35b-a3b-apex\n  url: u\n")
+		fams := []family{{
+			repo:     "mudler/Qwen3.6-35B-A3B-APEX-GGUF",
+			repoBase: "Qwen3.6-35B-A3B-APEX",
+			stem:     "Qwen3.6-35B-A3B-APEX",
+			children: []childBuild{buildOf("qwen3.6-35b-a3b-apex-i-quality", "mudler/Qwen3.6-35B-A3B-APEX-GGUF", "a.gguf", 0)},
+		}}
+
+		inserts, newHubs, err := planHubs(fams, ix, noReuse, allAdded("qwen3.6-35b-a3b-apex-i-quality"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(newHubs).To(BeEmpty())
+		Expect(inserts[0].Variants).To(Equal([]string{"qwen3.6-35b-a3b-apex-i-quality"}))
+
+		out, err := galleryedit.Apply(ix.Lines, inserts)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(strings.Count(strings.Join(out, "\n"), "variants:")).To(Equal(1))
+		Expect(out).To(HaveLen(len(ix.Lines) + 1))
+	})
+
+	It("never lets the hub reference itself", func() {
+		// An unsloth rung whose weights the gallery already ships under the base
+		// name resolves, through Merge, straight back to the hub. The verifier
+		// reads a self reference as a variant that declares variants of its own.
+		ix := mustIndex("- name: step-3.7-flash\n  url: u\n")
+		fams := []family{{
+			repo:     "mudler/Step-3.7-Flash-APEX-GGUF",
+			repoBase: "Step-3.7-Flash-APEX",
+			stem:     "Step-3.7-Flash-APEX",
+			children: []childBuild{buildOf("step-3.7-flash-ud-q4-k-m", "unsloth/Step-3.7-Flash-GGUF", "a.gguf", 100)},
+		}}
+
+		inserts, _, err := planHubs(fams, ix, map[string]string{"step-3.7-flash-ud-q4-k-m": "step-3.7-flash"}, map[string]bool{})
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(inserts).To(BeEmpty())
+	})
+
+	It("emits a hub named for the base model when the gallery has none", func() {
+		ix := mustIndex("- name: qwen3.5-35b-a3b-apex\n  url: u\n")
+		fams := []family{{
+			repo:      "mudler/Qwen3.5-35B-A3B-APEX-GGUF",
+			repoBase:  "Qwen3.5-35B-A3B-APEX",
+			stem:      "Qwen3.5-35B-A3B-APEX",
+			hasMMProj: true,
+			children: []childBuild{
+				buildOf("qwen3.5-35b-a3b-apex-i-quality", "mudler/Qwen3.5-35B-A3B-APEX-GGUF", "a.gguf", 0),
+				buildOf("qwen3.5-35b-a3b-ud-q6-k", "unsloth/Qwen3.5-35B-A3B-GGUF", "b.gguf", 102),
+			},
+		}}
+
+		inserts, newHubs, err := planHubs(fams, ix, noReuse,
+			allAdded("qwen3.5-35b-a3b-apex-i-quality", "qwen3.5-35b-a3b-ud-q6-k"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(inserts).To(BeEmpty())
+		Expect(newHubs).To(HaveLen(1))
+
+		hub := newHubs[0]
+		Expect(hub.Name).To(Equal("qwen3.5-35b-a3b"))
+		Expect(hub.Name).ToNot(HaveSuffix("-apex"))
+
+		// A hand-written *-apex entry is an ordinary build, referenced like any
+		// other rung and never deleted or renamed.
+		Expect(hub.Variants).To(Equal([]VariantRef{
+			{Model: "qwen3.5-35b-a3b-apex"},
+			{Model: "qwen3.5-35b-a3b-apex-i-quality"},
+			{Model: "qwen3.5-35b-a3b-ud-q6-k"},
+		}))
+
+		// The verifier skips entries with no declared backend, so a hub without
+		// one would escape the tagging check in silence.
+		Expect(hub.Overrides).To(HaveKeyWithValue("backend", "llama-cpp"))
+		Expect(hub.Files).ToNot(BeEmpty())
+		Expect(hub.Tags).To(ContainElement("vision"))
+	})
+
+	It("gathers two APEX repos that share one base model under a single hub", func() {
+		ix := mustIndex("- name: unrelated\n  url: u\n")
+		fams := []family{
+			{
+				repo:     "mudler/Solo-APEX-GGUF",
+				repoBase: "Solo-APEX",
+				stem:     "Solo-APEX",
+				children: []childBuild{buildOf("solo-apex-i-quality", "mudler/Solo-APEX-GGUF", "a.gguf", 0)},
+			},
+			{
+				repo:     "mudler/Solo-APEX-MTP-GGUF",
+				repoBase: "Solo-APEX-MTP",
+				stem:     "Solo-APEX-MTP",
+				children: []childBuild{buildOf("solo-apex-mtp-i-quality", "mudler/Solo-APEX-MTP-GGUF", "b.gguf", 0)},
+			},
+		}
+
+		_, newHubs, err := planHubs(fams, ix, noReuse, allAdded("solo-apex-i-quality", "solo-apex-mtp-i-quality"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(newHubs).To(HaveLen(1))
+		Expect(newHubs[0].Name).To(Equal("solo"))
+		Expect(newHubs[0].Variants).To(Equal([]VariantRef{
+			{Model: "solo-apex-i-quality"},
+			{Model: "solo-apex-mtp-i-quality"},
+		}))
+	})
+})
+
+var _ = Describe("hubVariants", func() {
+	It("orders builds by quality rung rather than discovery order", func() {
+		// DiscoverAPEXTiers preserves input order and the HF API returns siblings
+		// alphabetically, so an unsorted list reads I-Balanced, I-Compact, I-Mini,
+		// I-Nano, I-Quality. Selection ignores authored order; this is for the
+		// human reading the file.
+		f := family{repoBase: "X-APEX", stem: "X-APEX", children: []childBuild{
+			{rank: rungRank["I-Nano"], entry: GalleryEntry{Name: "x-i-nano"}},
+			{rank: 100, entry: GalleryEntry{Name: "x-ud-q4-k-m"}},
+			{rank: rungRank["I-Quality"], entry: GalleryEntry{Name: "x-i-quality"}},
+			{rank: rungRank["I-Compact"], entry: GalleryEntry{Name: "x-i-compact"}},
+		}}
+
+		got := hubVariants(&f, mustIndex("- name: x\n  url: u\n"), map[string]string{}, map[string]bool{})
+
+		Expect(got).To(Equal([]string{"x-i-quality", "x-i-compact", "x-i-nano", "x-ud-q4-k-m"}))
+	})
+})
+
+var _ = Describe("ParseIndexText", func() {
+	It("refuses to edit by line number when the two views of the file disagree", func() {
+		_, err := ParseIndexText("- name: one\n  url: u\n-\n")
+		Expect(err).To(MatchError(ContainSubstring("empty")))
+	})
+
+	It("records the line range of each entry", func() {
+		ix := mustIndex("- name: first\n  url: u\n- name: second\n  url: u\n")
+
+		Expect(ix.Find("FIRST").Pos.StartLine).To(Equal(0))
+		Expect(ix.Find("first").Pos.EndLine).To(Equal(2))
+		Expect(ix.Find("second").Pos.StartLine).To(Equal(2))
+	})
+})
+
+var _ = Describe("resolveVariant", func() {
+	It("keeps an entry that was emitted even when it is also in reused", func() {
+		// A within-batch name collision records reused[name] = name while the
+		// FIRST entry of that name is still in add. Treating presence in reused as
+		// "dropped" would emit nothing for it.
+		added := map[string]bool{"dup": true}
+		reused := map[string]string{"dup": "dup"}
+
+		Expect(resolveVariant("dup", reused, added)).To(Equal("dup"))
+	})
+
+	It("redirects a reused name at the entry that stands in for it", func() {
+		added := map[string]bool{}
+		reused := map[string]string{"generated": "already-in-gallery"}
+
+		Expect(resolveVariant("generated", reused, added)).To(Equal("already-in-gallery"))
+	})
+})
+
+var _ = Describe("slug", func() {
+	It("lowercases and turns quant underscores into hyphens", func() {
+		Expect(slug("UD-Q4_K_M")).To(Equal("ud-q4-k-m"))
+		Expect(slug("gemma-4-26B-A4B-it-APEX")).To(Equal("gemma-4-26b-a4b-it-apex"))
+		Expect(slug("I-Nano")).To(Equal("i-nano"))
+	})
+})
+
+var _ = Describe("sortTiers", func() {
+	It("puts the imatrix ladder in descending quality order", func() {
+		tiers := []Tier{
+			{Label: "I-Balanced"}, {Label: "I-Compact"}, {Label: "I-Mini"},
+			{Label: "I-Nano"}, {Label: "I-Quality"},
+		}
+		sortTiers(tiers)
+		Expect(tierLabels(tiers)).To(Equal("I-Quality,I-Balanced,I-Compact,I-Mini,I-Nano"))
+	})
+})
+
+var _ = Describe("restrict", func() {
+	It("keeps only the named repos", func() {
+		got := restrict([]string{"mudler/A-APEX-GGUF", "mudler/B-APEX-GGUF"}, "mudler/B-APEX-GGUF")
+		Expect(got).To(Equal([]string{"mudler/B-APEX-GGUF"}))
+	})
+
+	It("returns nothing when the filter matches nothing", func() {
+		Expect(restrict([]string{"mudler/A-APEX-GGUF"}, "mudler/typo")).To(BeEmpty())
+	})
+})
+
+var _ = Describe("reportUnclassified", func() {
+	// One real imatrix rung is always present so the specs measure how the
+	// remaining files are bucketed, not an empty-repo edge case.
+	tier := Tier{Label: "I-Quality", File: GGUFFile{Name: "Model-APEX-I-Quality.gguf"}}
+
+	censusOf := func(names ...string) fileCensus {
+		files := []GGUFFile{tier.File}
+		for _, n := range names {
+			files = append(files, GGUFFile{Name: n})
+		}
+		return reportUnclassified("mudler/Model-APEX-GGUF", files, []Tier{tier}, nil)
+	}
+
+	It("counts a flat full-precision source as excluded, not unclassified", func() {
+		got := censusOf("Carnice-MoE-35B-A3B-F16.gguf")
+		Expect(got.fullPrecision).To(Equal(1))
+		Expect(got.unclassified).To(Equal(0))
+	})
+
+	It("counts every shard of a sharded full-precision source as excluded", func() {
+		got := censusOf(
+			"MiniMax-M2.7-APEX-F16-00001-of-00003.gguf",
+			"MiniMax-M2.7-APEX-F16-00002-of-00003.gguf",
+			"MiniMax-M2.7-APEX-F16-00003-of-00003.gguf",
+		)
+		Expect(got.fullPrecision).To(Equal(3))
+		Expect(got.unclassified).To(Equal(0))
+	})
+
+	It("treats bf16 the same as f16, in either case", func() {
+		got := censusOf("Model-APEX-BF16.gguf", "Model-APEX-bf16-00001-of-00002.gguf", "Model-APEX-f16.gguf")
+		Expect(got.fullPrecision).To(Equal(3))
+		Expect(got.unclassified).To(Equal(0))
+	})
+
+	It("still reports a genuinely unknown filename as unclassified", func() {
+		got := censusOf("Model-APEX-Turbo.gguf")
+		Expect(got.unclassified).To(Equal(1))
+		Expect(got.fullPrecision).To(Equal(0))
+	})
+
+	It("separates the two kinds when a repo publishes both", func() {
+		got := censusOf("Model-APEX-F16.gguf", "Model-APEX-Turbo.gguf")
+		Expect(got.fullPrecision).To(Equal(1))
+		Expect(got.unclassified).To(Equal(1))
+	})
+})

.github/ci/apexentries/merge.go

@@ -0,0 +1,143 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+const (
+	hfShorthandPrefix = "huggingface://"
+	hfResolvePrefix   = "https://huggingface.co/"
+	hfResolveInfix    = "/resolve/main/"
+)
+
+// canonicalURI reduces the two interchangeable spellings of a HuggingFace file
+// to one key, so a generated resolve/main URI dedups against the shorthand the
+// gallery uses for the majority of its entries.
+//
+// The repo is exactly the first two path segments; everything after is the file
+// path, which may itself contain slashes because sharded quants live in a
+// subdirectory. Anything that is not recognisably one of the two forms is
+// returned unchanged rather than guessed at, so mirrors and other hosts still
+// dedup on their literal string.
+func canonicalURI(uri string) string {
+	switch {
+	case strings.HasPrefix(uri, hfShorthandPrefix):
+		rest := strings.TrimPrefix(uri, hfShorthandPrefix)
+		owner, after, ok := strings.Cut(rest, "/")
+		if !ok {
+			return uri
+		}
+		name, file, ok := strings.Cut(after, "/")
+		if !ok || owner == "" || name == "" || file == "" {
+			return uri
+		}
+		return hfShorthandPrefix + owner + "/" + name + "/" + file
+
+	case strings.HasPrefix(uri, hfResolvePrefix):
+		rest := strings.TrimPrefix(uri, hfResolvePrefix)
+		repo, file, ok := strings.Cut(rest, hfResolveInfix)
+		if !ok || file == "" {
+			return uri
+		}
+		// A repo is owner/name and nothing more; a longer prefix means this is
+		// some other huggingface.co URL that must not be rewritten.
+		owner, name, ok := strings.Cut(repo, "/")
+		if !ok || owner == "" || name == "" || strings.Contains(name, "/") {
+			return uri
+		}
+		return hfShorthandPrefix + repo + "/" + file
+
+	default:
+		return uri
+	}
+}
+
+// ExistingIndex is the lookup built from the current gallery: entry names, and
+// which entry claims each weight URI.
+type ExistingIndex struct {
+	ByName map[string]int
+	ByURI  map[string]string
+}
+
+// LoadExisting reads the gallery index for dedup purposes only. It is
+// deliberately not used to rewrite the file: the index is 40,000 lines, and a
+// YAML round trip would reflow the whole thing into an unreviewable diff.
+func LoadExisting(path string) (*ExistingIndex, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return nil, err
+	}
+
+	var entries []struct {
+		Name  string `yaml:"name"`
+		Files []struct {
+			URI string `yaml:"uri"`
+		} `yaml:"files"`
+	}
+	if err := yaml.Unmarshal(raw, &entries); err != nil {
+		return nil, fmt.Errorf("parsing %s: %w", path, err)
+	}
+
+	ix := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+	for i, e := range entries {
+		ix.ByName[e.Name] = i
+		for _, f := range e.Files {
+			if f.URI != "" {
+				ix.ByURI[canonicalURI(f.URI)] = e.Name
+			}
+		}
+	}
+	return ix, nil
+}
+
+// Merge splits generated entries into those to add and those already covered.
+// reused maps a generated name to the existing entry that stands in for it, so
+// a parent can reference what is already there instead of duplicating weights.
+// Several APEX repos share one base model, so the same counterpart rungs are
+// generated more than once in a batch. The batch has to dedup against itself as
+// well as against the gallery, tracked locally because the caller may reuse the
+// ExistingIndex it passed in.
+func Merge(existing *ExistingIndex, generated []GalleryEntry) (add []GalleryEntry, reused map[string]string) {
+	reused = map[string]string{}
+	batchNames := map[string]string{}
+	batchURIs := map[string]string{}
+
+	// Canonicalized into a local copy rather than in place: an ExistingIndex may
+	// be hand-built or reused by the caller, so Merge must not rewrite it.
+	existingURIs := make(map[string]string, len(existing.ByURI))
+	for uri, owner := range existing.ByURI {
+		existingURIs[canonicalURI(uri)] = owner
+	}
+
+	for _, e := range generated {
+		// Name is checked before URI: a name collision must block the add
+		// whatever the weights say, since duplicate names corrupt the index.
+		if _, clash := existing.ByName[e.Name]; clash {
+			reused[e.Name] = e.Name
+			continue
+		}
+		if claimant, clash := batchNames[e.Name]; clash {
+			reused[e.Name] = claimant
+			continue
+		}
+		if len(e.Files) > 0 {
+			uri := canonicalURI(e.Files[0].URI)
+			if owner, ok := existingURIs[uri]; ok {
+				reused[e.Name] = owner
+				continue
+			}
+			if claimant, ok := batchURIs[uri]; ok {
+				reused[e.Name] = claimant
+				continue
+			}
+			batchURIs[uri] = e.Name
+		}
+		batchNames[e.Name] = e.Name
+		add = append(add, e)
+	}
+	return add, reused
+}

.github/ci/apexentries/merge_test.go

@@ -0,0 +1,183 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Merge", func() {
+	It("drops a generated entry whose weight URI already exists and reports the existing name", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"qwen3.6-35b-a3b-apex": 0},
+			ByURI: map[string]string{
+				"https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Quality.gguf": "qwen3.6-35b-a3b-apex",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "x-apex-i-quality",
+			Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Quality.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("x-apex-i-quality", "qwen3.6-35b-a3b-apex"))
+	})
+
+	It("keeps a generated entry whose weights are new", func() {
+		existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+		gen := []GalleryEntry{{
+			Name:  "x-apex-i-mini",
+			Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Mini.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(reused).To(BeEmpty())
+	})
+
+	It("refuses to add an entry whose name collides with an existing one", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"x-apex-i-mini": 0},
+			ByURI:  map[string]string{},
+		}
+		gen := []GalleryEntry{{
+			Name:  "x-apex-i-mini",
+			Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/other.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("x-apex-i-mini", "x-apex-i-mini"))
+	})
+
+	// The gallery records most of its URIs in huggingface:// shorthand while
+	// render.go only ever emits the resolve/main form, so without
+	// canonicalization the majority of the file is invisible to the dedup.
+	It("matches a generated https URI against the shorthand form recorded in the gallery", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"foo-gguf-q8-0": 0},
+			ByURI: map[string]string{
+				"huggingface://unsloth/Foo-GGUF/Foo-Q8_0.gguf": "foo-gguf-q8-0",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "foo-apex-q8-0",
+			Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Foo-GGUF/resolve/main/Foo-Q8_0.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("foo-apex-q8-0", "foo-gguf-q8-0"))
+	})
+
+	It("matches a generated shorthand URI against the https form recorded in the gallery", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"foo-gguf-q8-0": 0},
+			ByURI: map[string]string{
+				"https://huggingface.co/unsloth/Foo-GGUF/resolve/main/Foo-Q8_0.gguf": "foo-gguf-q8-0",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "foo-apex-q8-0",
+			Files: []EntryFile{{URI: "huggingface://unsloth/Foo-GGUF/Foo-Q8_0.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("foo-apex-q8-0", "foo-gguf-q8-0"))
+	})
+
+	// Sharded quants live under a subdirectory, so the file path carries slashes
+	// of its own and only the first two segments are the repo.
+	It("matches across both forms when the file path has a subdirectory", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"model-ud-q4-k-m": 0},
+			ByURI: map[string]string{
+				"huggingface://unsloth/Model-GGUF/UD-Q4_K_M/Model-UD-Q4_K_M-00001-of-00002.gguf": "model-ud-q4-k-m",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "model-apex-ud-q4-k-m",
+			Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Model-GGUF/resolve/main/UD-Q4_K_M/Model-UD-Q4_K_M-00001-of-00002.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("model-apex-ud-q4-k-m", "model-ud-q4-k-m"))
+	})
+
+	// Several APEX repos share one base model, so the same unsloth rungs are
+	// generated more than once in a single batch.
+	It("adds only the first of two generated entries sharing a name", func() {
+		existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+		gen := []GalleryEntry{
+			{
+				Name:  "shared-rung-q8-0",
+				Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"}},
+			},
+			{
+				Name:  "shared-rung-q8-0",
+				Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Other-GGUF/resolve/main/Other-Q8_0.gguf"}},
+			},
+		}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(add[0].Files[0].URI).To(Equal("https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"))
+		Expect(reused).To(HaveKeyWithValue("shared-rung-q8-0", "shared-rung-q8-0"))
+	})
+
+	It("adds only the first of two generated entries sharing a primary URI", func() {
+		existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+		gen := []GalleryEntry{
+			{
+				Name:  "shared-rung-from-apex",
+				Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"}},
+			},
+			{
+				Name:  "shared-rung-from-apex-mtp",
+				Files: []EntryFile{{URI: "huggingface://unsloth/Shared-GGUF/Shared-Q8_0.gguf"}},
+			},
+		}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(add[0].Name).To(Equal("shared-rung-from-apex"))
+		Expect(reused).To(HaveKeyWithValue("shared-rung-from-apex-mtp", "shared-rung-from-apex"))
+	})
+
+	// Anything that is not a HuggingFace URI must survive untouched, so an
+	// unrecognised scheme still dedups against the very same string.
+	It("leaves a URI in neither recognised form alone and still dedups it exactly", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"mirrored-model": 0},
+			ByURI: map[string]string{
+				"https://mirror.example.com/weights/Model-Q8_0.gguf": "mirrored-model",
+			},
+		}
+		gen := []GalleryEntry{
+			{
+				Name:  "mirrored-apex",
+				Files: []EntryFile{{URI: "https://mirror.example.com/weights/Model-Q8_0.gguf"}},
+			},
+			{
+				Name:  "elsewhere-apex",
+				Files: []EntryFile{{URI: "https://mirror.example.com/weights/Other-Q8_0.gguf"}},
+			},
+		}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(add[0].Name).To(Equal("elsewhere-apex"))
+		Expect(reused).To(HaveKeyWithValue("mirrored-apex", "mirrored-model"))
+	})
+})

.github/ci/apexentries/render.go

@@ -0,0 +1,175 @@
+package main
+
+import (
+	"fmt"
+	"path"
+	"strings"
+)
+
+// EntryFile is one downloadable file of a gallery entry.
+type EntryFile struct {
+	Filename string `yaml:"filename"`
+	SHA256   string `yaml:"sha256"`
+	URI      string `yaml:"uri"`
+}
+
+// GalleryEntry is the subset of a gallery entry this generator writes.
+//
+// Named GalleryEntry rather than Entry because the test files dot-import
+// Ginkgo, whose table DSL exports an Entry that a package-level Entry would
+// collide with. The yaml tags are what the gallery index sees, so the Go
+// identifier is free to differ.
+type GalleryEntry struct {
+	Name        string         `yaml:"name"`
+	URL         string         `yaml:"url"`
+	Description string         `yaml:"description,omitempty"`
+	Tags        []string       `yaml:"tags,omitempty"`
+	Overrides   map[string]any `yaml:"overrides,omitempty"`
+	Files       []EntryFile    `yaml:"files,omitempty"`
+	Variants    []VariantRef   `yaml:"variants,omitempty"`
+}
+
+// VariantRef mirrors the gallery's variant reference: a name and nothing else.
+type VariantRef struct {
+	Model string `yaml:"model"`
+}
+
+// ChildInput is everything needed to render one non-parent entry.
+type ChildInput struct {
+	Name string
+	Repo string
+	// DraftRepo is the repo publishing the drafter, when it is not the repo
+	// publishing the weights. Speculative pairings routinely cross repos, so
+	// the drafter cannot be assumed to sit next to the weights. Empty means
+	// same-repo, which is how the *-APEX-MTP-GGUF repos ship.
+	DraftRepo string
+	Template  string
+	Weights   []GGUFFile
+	MMProj    *GGUFFile
+	SpecType  string
+	DraftFile *GGUFFile
+	BaseTags  []string
+}
+
+// specTuning is the acceptance-window tuning each spec type ships with, copied
+// from the hand-written entries that already run these two mechanisms rather
+// than invented here. The two differ because the drafters differ: self-drafted
+// MTP heads produce a short, high-confidence proposal (15+ hand-written entries
+// use 6 with a 0.75 floor), while a separate DFlash drafter is cheap enough to
+// run far ahead unconditionally (the five hand-written dflash entries use 15 and
+// set no floor).
+var specTuning = map[string][]string{
+	"draft-mtp":    {"spec_n_max:6", "spec_p_min:0.75"},
+	"draft-dflash": {"spec_n_max:15"},
+}
+
+func hfURI(repo, file string) string {
+	return fmt.Sprintf("https://huggingface.co/%s/resolve/main/%s", repo, file)
+}
+
+// localPath is where a downloaded file lands.
+//
+// The hand-written entries namespace by the repo's BARE name
+// (llama-cpp/models/<repo>/<file>), which is not unique. LiquidAI/LFM2.5-8B-A1B-GGUF
+// and unsloth/LFM2.5-8B-A1B-GGUF share a basename, so both claim
+// llama-cpp/models/LFM2.5-8B-A1B-GGUF/, and installing the second after the first
+// either overwrites weights whose recorded sha256 belongs to the other file or is
+// skipped as already present. Two owners publishing the same model name is the
+// normal case for quantizers, not an edge case, so the owner has to be in the path.
+//
+// The owner becomes its own path segment rather than being folded into the
+// directory name: owner/repo is unique on HuggingFace and "/" cannot occur inside
+// either half, so this is the only form that is collision-proof by construction.
+// It still reads as the hand-written convention with the owner restored, and the
+// extra depth is already present in the index for sharded builds.
+func localPath(kind, repo, file string) string {
+	// path.Dir yields "." for a repo named without an owner, which path.Join
+	// drops, so such a caller keeps the historical two-segment layout.
+	return path.Join("llama-cpp", kind, path.Dir(repo), path.Base(repo), file)
+}
+
+// RenderChild builds one child entry.
+//
+// The dflash/mtp tag is added if and only if this entry sets a spec_type,
+// because variant ranking reads tags and nothing else, and a tag that does not
+// match what the entry configures either promotes a build that is no faster or
+// hides one that is.
+func RenderChild(in ChildInput) GalleryEntry {
+	e := GalleryEntry{
+		Name:      in.Name,
+		URL:       fmt.Sprintf("github:mudler/LocalAI/gallery/%s@master", in.Template),
+		Tags:      append([]string{}, in.BaseTags...),
+		Overrides: map[string]any{},
+	}
+
+	// gallery/virtual.yaml carries no backend, so nothing else would name an
+	// engine for these entries. Matching the hand-written entries on
+	// known_usecases too: LocalAI would fall back to the backend defaults, but
+	// generated entries should not read differently from their neighbours.
+	e.Overrides["backend"] = "llama-cpp"
+	e.Overrides["known_usecases"] = []string{"chat"}
+
+	options := []string{"use_jinja:true"}
+
+	for _, w := range in.Weights {
+		e.Files = append(e.Files, EntryFile{
+			Filename: localPath("models", in.Repo, w.Name),
+			SHA256:   w.SHA256,
+			URI:      hfURI(in.Repo, w.Name),
+		})
+	}
+	e.Overrides["parameters"] = map[string]any{
+		"model": localPath("models", in.Repo, in.Weights[0].Name),
+	}
+
+	if in.MMProj != nil {
+		// An explicit known_usecases SUPPRESSES the backend-default fallback in
+		// core/gallery/models_types.go, so a multimodal entry left at chat-only
+		// never matches FilterGalleryModelsByUsecase(FLAG_VISION) or
+		// FilterGalleryModelsByMultimodal and vanishes from the UI's vision and
+		// multimodal filters. 19 of the 45 APEX repos ship an mmproj.
+		e.Overrides["known_usecases"] = []string{"chat", "vision"}
+		e.Overrides["mmproj"] = localPath("mmproj", in.Repo, in.MMProj.Name)
+		e.Files = append(e.Files, EntryFile{
+			Filename: localPath("mmproj", in.Repo, in.MMProj.Name),
+			SHA256:   in.MMProj.SHA256,
+			URI:      hfURI(in.Repo, in.MMProj.Name),
+		})
+	}
+
+	// A spec type is configured independently of a drafter FILE. Weights that
+	// carry their own MTP heads need no second download, and requiring one left
+	// the *-APEX-MTP-GGUF builds shipping the larger heads-bearing weights with
+	// the heads switched off: a strictly bigger download at the same speed,
+	// ranked identically to the plain rung at the same tier.
+	if in.SpecType != "" {
+		options = append(options, "spec_type:"+in.SpecType)
+		options = append(options, specTuning[in.SpecType]...)
+		// The tag is derived from the spec type this entry sets and from nothing
+		// else. Variant ranking reads tags only, so a tag taken from a repo or
+		// entry NAME would promote a build that is no faster whenever the name
+		// and the configuration disagree.
+		e.Tags = append(e.Tags, strings.TrimPrefix(in.SpecType, "draft-"))
+	}
+
+	if in.SpecType != "" && in.DraftFile != nil {
+		// Fall back to the weights repo so pairings that publish the drafter
+		// alongside the weights keep working without restating the repo.
+		draftRepo := in.DraftRepo
+		if draftRepo == "" {
+			draftRepo = in.Repo
+		}
+		draftPath := localPath("models", draftRepo, in.DraftFile.Name)
+
+		e.Overrides["draft_model"] = draftPath
+		e.Overrides["flash_attention"] = "on"
+		e.Files = append(e.Files, EntryFile{
+			Filename: draftPath,
+			SHA256:   in.DraftFile.SHA256,
+			URI:      hfURI(draftRepo, in.DraftFile.Name),
+		})
+	}
+
+	e.Overrides["options"] = options
+	return e
+}

.github/ci/apexentries/render_test.go

@@ -0,0 +1,249 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("RenderChild", func() {
+	It("tags an entry that configures draft-dflash", func() {
+		e := RenderChild(ChildInput{
+			Name:      "qwen3.5-9b-dflash",
+			Repo:      "mudler/Example-APEX-GGUF",
+			Template:  "virtual.yaml",
+			Weights:   []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			SpecType:  "draft-dflash",
+			DraftFile: &GGUFFile{Name: "Example-DFlash.Q8_0.gguf", SHA256: "b"},
+			BaseTags:  []string{"llm", "gguf"},
+		})
+
+		Expect(e.Tags).To(ContainElement("dflash"))
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Overrides["options"]).To(ContainElement("spec_type:draft-dflash"))
+		Expect(e.Overrides["draft_model"]).ToNot(BeNil())
+	})
+
+	It("does not tag an MTP-named repo that configures no speculation", func() {
+		// mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF ships MTP-bearing weights. Weights
+		// that carry the heads are not an entry that enables them, and tagging it
+		// would win the feature axis without being any faster.
+		e := RenderChild(ChildInput{
+			Name:     "qwen3.6-35b-a3b-apex-mtp-i-quality",
+			Repo:     "mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Qwen3.6-35B-A3B-APEX-MTP-I-Quality.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Tags).ToNot(ContainElement("dflash"))
+		Expect(e.Overrides).ToNot(HaveKey("draft_model"))
+	})
+
+	It("lists every shard of a sharded build and points the model at the first", func() {
+		e := RenderChild(ChildInput{
+			Name:     "step-3.7-flash-ud-q4-k-m",
+			Repo:     "unsloth/Step-3.7-Flash-GGUF",
+			Template: "virtual.yaml",
+			Weights: []GGUFFile{
+				{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf", SHA256: "a"},
+				{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00002-of-00002.gguf", SHA256: "b"},
+			},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Files).To(HaveLen(2))
+		params, ok := e.Overrides["parameters"].(map[string]any)
+		Expect(ok).To(BeTrue())
+		Expect(params["model"]).To(HaveSuffix("00001-of-00002.gguf"))
+		Expect(e.Files[0].URI).To(Equal(
+			"https://huggingface.co/unsloth/Step-3.7-Flash-GGUF/resolve/main/UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf"))
+	})
+
+	It("wires mmproj when the repo publishes one", func() {
+		e := RenderChild(ChildInput{
+			Name:     "example-i-mini",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Mini.gguf", SHA256: "a"}},
+			MMProj:   &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["mmproj"]).ToNot(BeNil())
+		Expect(e.Files).To(HaveLen(2))
+	})
+
+	It("names the engine and the usecases the hand-written entries name", func() {
+		// gallery/virtual.yaml supplies no backend, so an entry that omits one
+		// names no engine at all and cannot load.
+		e := RenderChild(ChildInput{
+			Name:     "example-i-mini",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Mini.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["backend"]).To(Equal("llama-cpp"))
+		Expect(e.Overrides["known_usecases"]).To(ContainElement("chat"))
+	})
+
+	It("draws the drafter from DraftRepo when the pairing spans two repos", func() {
+		// unsloth/Qwen3-4B-GGUF pairs with a drafter published separately by
+		// AtomicChat, so a drafter URI built from the weights repo 404s.
+		e := RenderChild(ChildInput{
+			Name:      "qwen3-4b-dflash",
+			Repo:      "unsloth/Qwen3-4B-GGUF",
+			DraftRepo: "AtomicChat/Qwen3-4B-DFlash-GGUF",
+			Template:  "virtual.yaml",
+			Weights:   []GGUFFile{{Name: "Qwen3-4B-Q4_K_M.gguf", SHA256: "a"}},
+			SpecType:  "draft-dflash",
+			DraftFile: &GGUFFile{Name: "Qwen3-4B-DFlash.Q8_0.gguf", SHA256: "b"},
+			BaseTags:  []string{"llm", "gguf"},
+		})
+
+		Expect(e.Files[0].URI).To(Equal(
+			"https://huggingface.co/unsloth/Qwen3-4B-GGUF/resolve/main/Qwen3-4B-Q4_K_M.gguf"))
+		Expect(e.Files[1].URI).To(Equal(
+			"https://huggingface.co/AtomicChat/Qwen3-4B-DFlash-GGUF/resolve/main/Qwen3-4B-DFlash.Q8_0.gguf"))
+		Expect(e.Files[1].Filename).To(Equal(
+			"llama-cpp/models/AtomicChat/Qwen3-4B-DFlash-GGUF/Qwen3-4B-DFlash.Q8_0.gguf"))
+		Expect(e.Overrides["draft_model"]).To(Equal(
+			"llama-cpp/models/AtomicChat/Qwen3-4B-DFlash-GGUF/Qwen3-4B-DFlash.Q8_0.gguf"))
+	})
+
+	It("falls back to the weights repo for the drafter when DraftRepo is empty", func() {
+		// The *-APEX-MTP-GGUF repos ship the drafter alongside the weights.
+		e := RenderChild(ChildInput{
+			Name:      "example-apex-dflash",
+			Repo:      "mudler/Example-APEX-GGUF",
+			Template:  "virtual.yaml",
+			Weights:   []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			SpecType:  "draft-dflash",
+			DraftFile: &GGUFFile{Name: "Example-DFlash.Q8_0.gguf", SHA256: "b"},
+			BaseTags:  []string{"llm", "gguf"},
+		})
+
+		Expect(e.Files[1].URI).To(Equal(
+			"https://huggingface.co/mudler/Example-APEX-GGUF/resolve/main/Example-DFlash.Q8_0.gguf"))
+		Expect(e.Files[1].Filename).To(Equal(
+			"llama-cpp/models/mudler/Example-APEX-GGUF/Example-DFlash.Q8_0.gguf"))
+	})
+})
+
+var _ = Describe("RenderChild known_usecases", func() {
+	It("declares vision alongside chat when the entry carries an mmproj", func() {
+		// An explicit known_usecases suppresses the backend-default fallback, so a
+		// chat-only multimodal entry disappears from the UI's vision filter.
+		e := RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			MMProj:   &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["known_usecases"]).To(ConsistOf("chat", "vision"))
+	})
+
+	It("leaves a text-only entry at chat", func() {
+		e := RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["known_usecases"]).To(ConsistOf("chat"))
+	})
+})
+
+var _ = Describe("localPath", func() {
+	It("keeps two repos with the same basename but different owners apart", func() {
+		// LiquidAI and unsloth both publish LFM2.5-8B-A1B-GGUF. A path built from
+		// the bare repo name gives both the same local file, so installing the
+		// second overwrites or skips the first and one of them then serves bytes
+		// that do not match its recorded sha256.
+		liquid := RenderChild(ChildInput{
+			Name:     "lfm2.5-8b-a1b-i-quality",
+			Repo:     "LiquidAI/LFM2.5-8B-A1B-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "LFM2.5-8B-A1B-Q8_0.gguf", SHA256: "33ab3b8c"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+		unsloth := RenderChild(ChildInput{
+			Name:     "lfm2.5-8b-a1b-q8-0",
+			Repo:     "unsloth/LFM2.5-8B-A1B-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "LFM2.5-8B-A1B-Q8_0.gguf", SHA256: "ec11666b"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(liquid.Files[0].Filename).ToNot(Equal(unsloth.Files[0].Filename))
+		Expect(unsloth.Files[0].Filename).To(Equal(
+			"llama-cpp/models/unsloth/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf"))
+	})
+
+	It("namespaces the mmproj by owner too", func() {
+		e := RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			MMProj:   &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["mmproj"]).To(Equal(
+			"llama-cpp/mmproj/mudler/Example-APEX-GGUF/mmproj-F16.gguf"))
+	})
+})
+
+var _ = Describe("MTP builds", func() {
+	renderTier := func(repo string) GalleryEntry {
+		return RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     repo,
+			Template: "virtual.yaml",
+			SpecType: SpecTypeForRepo(repo),
+			Weights:  []GGUFFile{{Name: "Example-I-Quality.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+	}
+
+	It("turns MTP on for a build off an APEX-MTP repo", func() {
+		// These weights retain the model's own MTP heads, so shipping them with
+		// speculation off is a strictly larger download at the same speed,
+		// ranked identically to the plain rung at the same tier.
+		e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF")
+
+		Expect(e.Overrides["options"]).To(ContainElements(
+			"spec_type:draft-mtp", "spec_n_max:6", "spec_p_min:0.75"))
+		Expect(e.Tags).To(ContainElement("mtp"))
+	})
+
+	It("needs no drafter file, because the heads travel with the weights", func() {
+		e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF")
+
+		Expect(e.Overrides).ToNot(HaveKey("draft_model"))
+		Expect(e.Files).To(HaveLen(1))
+	})
+
+	It("leaves a build off a plain APEX repo alone", func() {
+		e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-GGUF")
+
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Overrides["options"]).To(ConsistOf("use_jinja:true"))
+	})
+
+	It("leaves an unsloth counterpart rung alone", func() {
+		// The counterpart quantizes the plain weights; nothing there carries heads.
+		e := renderTier("unsloth/Qwen3.6-35B-A3B-GGUF")
+
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Overrides["options"]).To(ConsistOf("use_jinja:true"))
+	})
+})

.github/ci/apexentries/unsloth.go

@@ -0,0 +1,71 @@
+package main
+
+import (
+	"regexp"
+	"sort"
+	"strings"
+)
+
+// WantedQuants is the fixed unsloth subset this generator emits. It is a
+// deliberate subset: unsloth publishes north of 20 quants per repo, and the
+// selector needs useful fitness points rather than every rung.
+var WantedQuants = []string{"UD-Q4_K_M", "UD-Q5_K_M", "UD-Q6_K", "Q8_0"}
+
+var shardRE = regexp.MustCompile(`-(\d{5})-of-(\d{5})\.gguf$`)
+
+// QuantBuild is one unsloth quantization, which may be a single file or an
+// ordered set of shards.
+type QuantBuild struct {
+	Quant   string
+	Files   []GGUFFile
+	Sharded bool
+}
+
+// CounterpartCandidates returns the unsloth repo base names worth probing, most
+// likely first. Both derivations are needed: the repo name finds
+// unsloth/gemma-4-26B-A4B-it-GGUF, while the file stem is what matches for
+// repos whose stem is the canonical model name.
+func CounterpartCandidates(repoName, fileStem string) []string {
+	clean := func(s string) string {
+		s = strings.TrimSuffix(s, "-GGUF")
+		s = regexp.MustCompile(`-(MTP|TQ)$`).ReplaceAllString(s, "")
+		s = strings.TrimSuffix(s, "-APEX")
+		return regexp.MustCompile(`-(MTP|TQ)$`).ReplaceAllString(s, "")
+	}
+
+	out := []string{clean(repoName)}
+	if stem := clean(fileStem); stem != out[0] {
+		out = append(out, stem)
+	}
+	return out
+}
+
+// DiscoverUnslothQuants returns the wanted quants a repo publishes, handling
+// both the flat single-file layout and the sharded layout where a quant lives
+// in its own subdirectory.
+func DiscoverUnslothQuants(files []GGUFFile) []QuantBuild {
+	var out []QuantBuild
+
+	for _, q := range WantedQuants {
+		var flat []GGUFFile
+		var shards []GGUFFile
+
+		for _, f := range files {
+			switch {
+			case !strings.Contains(f.Name, "/") && strings.HasSuffix(f.Name, "-"+q+".gguf"):
+				flat = append(flat, f)
+			case strings.HasPrefix(f.Name, q+"/") && shardRE.MatchString(f.Name):
+				shards = append(shards, f)
+			}
+		}
+
+		switch {
+		case len(flat) > 0:
+			out = append(out, QuantBuild{Quant: q, Files: flat})
+		case len(shards) > 0:
+			sort.Slice(shards, func(i, j int) bool { return shards[i].Name < shards[j].Name })
+			out = append(out, QuantBuild{Quant: q, Files: shards, Sharded: true})
+		}
+	}
+	return out
+}

.github/ci/apexentries/unsloth_test.go

@@ -0,0 +1,75 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("CounterpartCandidates", func() {
+	It("offers both the repo-derived and stem-derived names", func() {
+		// mudler/gemma-4-26B-A4B-it-APEX-GGUF ships gemma-4-26B-A4B-APEX-*.gguf,
+		// and only the repo-derived name finds unsloth/gemma-4-26B-A4B-it-GGUF.
+		got := CounterpartCandidates("gemma-4-26B-A4B-it-APEX-GGUF", "gemma-4-26B-A4B-APEX")
+
+		Expect(got).To(Equal([]string{"gemma-4-26B-A4B-it", "gemma-4-26B-A4B"}))
+	})
+
+	It("strips the MTP marker", func() {
+		got := CounterpartCandidates("Qwopus3.6-35B-A3B-v1-APEX-MTP-GGUF", "Qwopus3.6-35B-A3B-v1-APEX-MTP")
+
+		Expect(got[0]).To(Equal("Qwopus3.6-35B-A3B-v1"))
+	})
+
+	It("strips the TQ marker", func() {
+		// This is the branch that folds mudler/Qwen3.5-35B-A3B-APEX-TQ-GGUF into
+		// the qwen3.5-35b-a3b hub. Without it the probe is
+		// unsloth/Qwen3.5-35B-A3B-TQ-GGUF, which does not exist, so the family
+		// silently loses every unsloth rung.
+		got := CounterpartCandidates("Qwen3.5-35B-A3B-APEX-TQ-GGUF", "Qwen3.5-35B-A3B-APEX-TQ")
+
+		Expect(got).To(Equal([]string{"Qwen3.5-35B-A3B"}))
+	})
+
+	It("does not repeat a candidate when both derivations agree", func() {
+		got := CounterpartCandidates("Qwen3.6-35B-A3B-APEX-GGUF", "Qwen3.6-35B-A3B-APEX")
+
+		Expect(got).To(Equal([]string{"Qwen3.6-35B-A3B"}))
+	})
+})
+
+var _ = Describe("DiscoverUnslothQuants", func() {
+	It("finds flat single-file quants", func() {
+		files := []GGUFFile{
+			{Name: "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf", SHA256: "a"},
+			{Name: "Qwen3.6-35B-A3B-UD-IQ1_M.gguf", SHA256: "b"},
+		}
+
+		got := DiscoverUnslothQuants(files)
+
+		Expect(got).To(HaveLen(1))
+		Expect(got[0].Quant).To(Equal("UD-Q4_K_M"))
+		Expect(got[0].Sharded).To(BeFalse())
+		Expect(got[0].Files).To(HaveLen(1))
+	})
+
+	It("collects a sharded quant from its subdirectory in shard order", func() {
+		files := []GGUFFile{
+			{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00002-of-00002.gguf", SHA256: "b"},
+			{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf", SHA256: "a"},
+		}
+
+		got := DiscoverUnslothQuants(files)
+
+		Expect(got).To(HaveLen(1))
+		Expect(got[0].Quant).To(Equal("UD-Q4_K_M"))
+		Expect(got[0].Sharded).To(BeTrue())
+		Expect(got[0].Files).To(HaveLen(2))
+		Expect(got[0].Files[0].Name).To(HaveSuffix("00001-of-00002.gguf"))
+	})
+
+	It("ignores quants outside the wanted subset", func() {
+		files := []GGUFFile{{Name: "Model-UD-IQ2_XXS.gguf", SHA256: "a"}}
+
+		Expect(DiscoverUnslothQuants(files)).To(BeEmpty())
+	})
+})

.github/ci/apexentries/verify.go

@@ -0,0 +1,312 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+type verifyEntry struct {
+	Name      string       `yaml:"name"`
+	Tags      []string     `yaml:"tags"`
+	Variants  []VariantRef `yaml:"variants"`
+	Overrides struct {
+		// Backend scopes the checks that only hold for one engine. An entry that
+		// declares none takes its configuration from the referenced url: template,
+		// which this verifier never reads, so it cannot be judged either way.
+		Backend string   `yaml:"backend"`
+		Options []string `yaml:"options"`
+		// MMProj and DraftModel name the files that are not weights. They are
+		// the only signal for it: a drafter lands in the same models/ prefix as
+		// the weights, so the path alone cannot tell them apart.
+		MMProj     string `yaml:"mmproj"`
+		DraftModel string `yaml:"draft_model"`
+	} `yaml:"overrides"`
+	Files []struct {
+		Filename string `yaml:"filename"`
+		SHA256   string `yaml:"sha256"`
+		URI      string `yaml:"uri"`
+	} `yaml:"files"`
+}
+
+// Verify checks the invariants the variants schema and the tagging rule
+// require. It returns every problem rather than the first, so one run tells the
+// author everything that needs fixing.
+func Verify(path string) []string {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return []string{fmt.Sprintf("reading %s: %v", path, err)}
+	}
+
+	var entries []verifyEntry
+	if err := yaml.Unmarshal(raw, &entries); err != nil {
+		return []string{fmt.Sprintf("parsing %s: %v", path, err)}
+	}
+
+	var problems []string
+
+	byName := map[string]verifyEntry{}
+	for _, e := range entries {
+		if _, seen := byName[e.Name]; seen {
+			problems = append(problems, fmt.Sprintf("duplicate entry name: %s", e.Name))
+			continue
+		}
+		byName[e.Name] = e
+	}
+
+	for _, e := range entries {
+		for _, v := range e.Variants {
+			target, ok := byName[v.Model]
+			if !ok {
+				problems = append(problems, fmt.Sprintf("%s: variant %q does not exist", e.Name, v.Model))
+				continue
+			}
+			if len(target.Variants) > 0 {
+				problems = append(problems, fmt.Sprintf("%s: variant %q declares variants of its own", e.Name, v.Model))
+			}
+		}
+
+		for _, f := range e.Files {
+			if requiresSHA256(f.Filename) && f.SHA256 == "" {
+				problems = append(problems, fmt.Sprintf("%s: file %s has no sha256", e.Name, f.Filename))
+			}
+		}
+
+		problems = append(problems, checkWeightCount(e)...)
+		problems = append(problems, checkFeatureTag(e, "dflash")...)
+		problems = append(problems, checkFeatureTag(e, "mtp")...)
+	}
+
+	problems = append(problems, checkPathCollisions(entries)...)
+
+	return problems
+}
+
+// checkPathCollisions catches two different upstream files claiming one local
+// path. The install layer keys on the local filename, so whichever entry is
+// installed second either overwrites weights the first entry recorded a
+// different sha256 for or is skipped as already present. Either way some entry
+// afterwards serves bytes that do not match its own checksum, and nothing at
+// install time says so.
+//
+// This is an index-wide invariant rather than a per-entry one: neither entry is
+// wrong on its own and the collision exists only in their pairing. The usual
+// source is a path scheme built from the repo's BARE name, because two owners
+// publishing the same model name is routine for quantizers.
+//
+// Sharing a path is fine when the uri is the same, which is how several entries
+// legitimately reuse one projector. Files with no uri are skipped: there is
+// nothing to compare.
+func checkPathCollisions(entries []verifyEntry) []string {
+	type source struct{ uri, entry string }
+
+	first := map[string]source{}
+	reported := map[string]bool{}
+
+	var problems []string
+	for _, e := range entries {
+		for _, f := range e.Files {
+			if f.Filename == "" || f.URI == "" {
+				continue
+			}
+			prev, seen := first[f.Filename]
+			if !seen {
+				first[f.Filename] = source{uri: f.URI, entry: e.Name}
+				continue
+			}
+			if prev.uri == f.URI || reported[f.Filename] {
+				continue
+			}
+			// Reported once per path however many entries pile onto it, so one
+			// heavily reused filename cannot bury the rest of the report.
+			reported[f.Filename] = true
+			problems = append(problems, fmt.Sprintf(
+				"local path %s is claimed by two different uris: %s (%s) and %s (%s)",
+				f.Filename, prev.uri, prev.entry, f.URI, e.Name))
+		}
+	}
+	return problems
+}
+
+// auxiliaryExtensions are the metadata formats an entry ships beside its
+// weights, where an unverified download is a nuisance rather than a hole.
+//
+// The exclusion is stated as a list of metadata formats on purpose. Requiring
+// the checksum only on a blessed list of weight formats would silently exempt
+// every format nobody has shipped yet, and it already exempted safetensors
+// weights, which are downloaded and loaded exactly like GGUF ones.
+var auxiliaryExtensions = []string{".json", ".txt", ".md"}
+
+// requiresSHA256 reports whether an unverified download of this file would be
+// a supply-chain hole rather than a cosmetic gap.
+func requiresSHA256(filename string) bool {
+	for _, ext := range auxiliaryExtensions {
+		if strings.HasSuffix(filename, ext) {
+			return false
+		}
+	}
+	return true
+}
+
+// checkWeightCount catches an entry carrying two whole models. The flat-match
+// branch in DiscoverUnslothQuants appends every match, so a quant label that is
+// a suffix of another one (Q8_0 of UD-Q8_0) collects both files into one build
+// while the rendered model: points at only the first. The result downloads
+// twice the bytes and serves whichever file sorted first, silently.
+//
+// Shards are exempt because a sharded build is legitimately many files.
+//
+// The collision is a property of llama-cpp quant discovery, so the check is
+// scoped to that backend. Multi-component TTS, ASR and diffusion engines ship an
+// encoder, a decoder and a vocoder as one model, and there the second GGUF is
+// the design rather than a bug.
+func checkWeightCount(e verifyEntry) []string {
+	if e.Overrides.Backend != "llama-cpp" {
+		return nil
+	}
+
+	var weights []string
+	for _, f := range e.Files {
+		switch {
+		case !strings.HasSuffix(f.Filename, ".gguf"):
+		case shardRE.MatchString(f.Filename):
+		case f.Filename == e.Overrides.MMProj:
+		case f.Filename == e.Overrides.DraftModel:
+		default:
+			weights = append(weights, f.Filename)
+		}
+	}
+
+	if len(weights) > 1 {
+		return []string{fmt.Sprintf("%s: more than one weight file: %s", e.Name, strings.Join(weights, ", "))}
+	}
+	return nil
+}
+
+// checkFeatureTag enforces the rule in both directions. A tag without the
+// configuration promotes a build that is no faster; configuration without the
+// tag leaves a genuinely faster build ranked as plain.
+//
+// It only speaks about backends whose declaration it can actually read, because
+// a rule applied where the evidence is invisible reports noise rather than bugs.
+func checkFeatureTag(e verifyEntry, feature string) []string {
+	decl, configured, judgeable := featureDeclaration(e, feature)
+	if !judgeable {
+		return nil
+	}
+
+	tagged := false
+	for _, t := range e.Tags {
+		if t == feature {
+			tagged = true
+			break
+		}
+	}
+
+	switch {
+	case tagged && !configured:
+		return []string{fmt.Sprintf("%s: tagged %s but sets no %s", e.Name, feature, decl)}
+	case configured && !tagged:
+		return []string{fmt.Sprintf("%s: sets %s but is not tagged %s", e.Name, decl, feature)}
+	}
+	return nil
+}
+
+// featureDeclaration implements the per-backend table in
+// .agents/adding-gallery-models.md. It returns the declaration the backend uses
+// to configure the feature, whether the entry carries it, and whether this
+// verifier is in a position to answer at all.
+func featureDeclaration(e verifyEntry, feature string) (decl string, configured, judgeable bool) {
+	switch e.Overrides.Backend {
+	case "llama-cpp":
+		decl = "spec_type:draft-" + feature
+		for _, o := range e.Overrides.Options {
+			if strings.TrimSpace(o) == decl {
+				return decl, true, true
+			}
+		}
+		return decl, false, true
+
+	case "ds4":
+		// ds4 carries the MTP heads in the weights and turns them on with
+		// mtp_path / mtp_draft. It has no dflash counterpart, so dflash is not a
+		// question that can be asked of a ds4 entry.
+		if feature != "mtp" {
+			return "", false, false
+		}
+		decl = "mtp_path:"
+		for _, o := range e.Overrides.Options {
+			o = strings.TrimSpace(o)
+			if strings.HasPrefix(o, "mtp_path:") || strings.HasPrefix(o, "mtp_draft:") {
+				return decl, true, true
+			}
+		}
+		return decl, false, true
+
+	default:
+		// sglang configures the feature with speculative_algorithm: in the
+		// referenced gallery/*.yaml, and an entry that declares no backend takes
+		// its whole configuration from its url: template. Verify reads one index
+		// file and follows neither, so it must not judge these in either
+		// direction.
+		return "", false, false
+	}
+}
+
+// UnaccountedQuants reports a wanted quant the repo demonstrably publishes but
+// that discovery produced no build for. The layout that triggers it today is
+// root-level shards, which match neither branch of DiscoverUnslothQuants; no
+// counterpart ships that way yet, but a batch generator must not drop a build
+// with nothing said about it.
+func UnaccountedQuants(files []GGUFFile, builds []QuantBuild) []string {
+	built := map[string]bool{}
+	for _, b := range builds {
+		built[b.Quant] = true
+	}
+
+	var problems []string
+	for _, q := range WantedQuants {
+		if built[q] {
+			continue
+		}
+		for _, f := range files {
+			if filePublishesQuant(f.Name, q) {
+				problems = append(problems, fmt.Sprintf("quant %s is published upstream (%s) but produced no build", q, f.Name))
+				break
+			}
+		}
+	}
+	return problems
+}
+
+// filePublishesQuant reports whether an upstream file is a publication of
+// quant q. It anchors on the quant label the way DiscoverUnslothQuants does,
+// as the trailing token of the base name or as the sharding subdirectory, so
+// the diagnostic and the discovery it audits cannot disagree about what a file
+// is.
+//
+// An unanchored match would reproduce the very collision this diagnostic warns
+// about: Q8_0 is a substring of UD-Q8_0, so a repo publishing only UD-Q8_0
+// would be reported as publishing an unbuilt Q8_0, which it does not, and
+// UD-Q8_0 is not a wanted quant at all.
+func filePublishesQuant(name, q string) bool {
+	if strings.HasPrefix(name, q+"/") {
+		return true
+	}
+
+	base := name[strings.LastIndex(name, "/")+1:]
+	// Shard numbering sits between the quant label and the extension, so it has
+	// to come off before the label can be read as the trailing token. Root-level
+	// shards are the layout that matches neither branch of
+	// DiscoverUnslothQuants, and so the layout this diagnostic mainly catches.
+	base = shardRE.ReplaceAllString(base, ".gguf")
+
+	if !strings.HasSuffix(base, "-"+q+".gguf") {
+		return false
+	}
+	// UD- is unsloth's dynamic-quant modifier, and UD-<q> is a distinct quant
+	// label rather than a publication of <q>.
+	return !strings.HasSuffix(base, "-UD-"+q+".gguf")
+}

.github/ci/apexentries/verify_test.go

@@ -0,0 +1,480 @@
+package main
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Verify", func() {
+	write := func(body string) string {
+		dir := GinkgoT().TempDir()
+		p := filepath.Join(dir, "index.yaml")
+		Expect(os.WriteFile(p, []byte(body), 0o600)).To(Succeed())
+		return p
+	}
+
+	It("passes a sound index", func() {
+		Expect(Verify(write(`
+- name: parent
+  variants:
+    - model: child
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: child
+  files:
+    - filename: b.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(BeEmpty())
+	})
+
+	It("reports a variant pointing at a missing entry", func() {
+		Expect(Verify(write(`
+- name: parent
+  variants:
+    - model: ghost
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("ghost")))
+	})
+
+	It("reports a variant that itself declares variants", func() {
+		Expect(Verify(write(`
+- name: parent
+  variants:
+    - model: child
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: child
+  variants:
+    - model: grandchild
+  files:
+    - filename: b.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+- name: grandchild
+  files:
+    - filename: c.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+`))).To(ContainElement(ContainSubstring("declares variants of its own")))
+	})
+
+	It("reports duplicate entry names", func() {
+		Expect(Verify(write(`
+- name: dup
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: dup
+  files:
+    - filename: b.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(ContainElement(ContainSubstring("duplicate entry name")))
+	})
+
+	It("reports a file with no sha256", func() {
+		Expect(Verify(write(`
+- name: one
+  files:
+    - filename: a.gguf
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("no sha256")))
+	})
+
+	It("reports an entry tagged dflash without a matching spec_type", func() {
+		Expect(Verify(write(`
+- name: liar
+  tags:
+    - dflash
+  overrides:
+    backend: llama-cpp
+    options:
+      - use_jinja:true
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("tagged dflash")))
+	})
+
+	It("reports an entry configuring spec_type without the tag", func() {
+		Expect(Verify(write(`
+- name: shy
+  overrides:
+    backend: llama-cpp
+    options:
+      - spec_type:draft-mtp
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("not tagged mtp")))
+	})
+
+	// ds4 carries the MTP heads in the weights and names them with mtp_path, so
+	// the rule holds there in a different vocabulary rather than not at all.
+	It("reports a ds4 entry configuring mtp_path without the tag", func() {
+		Expect(Verify(write(`
+- name: ds4-shy
+  overrides:
+    backend: ds4
+    options:
+      - mtp_path:model-mtp.gguf
+      - mtp_draft:2
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("not tagged mtp")))
+	})
+
+	It("reports a ds4 entry tagged mtp that configures no mtp_path", func() {
+		Expect(Verify(write(`
+- name: ds4-liar
+  tags:
+    - mtp
+  overrides:
+    backend: ds4
+    options:
+      - context_size:4096
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("tagged mtp")))
+	})
+
+	It("accepts a ds4 entry that both configures mtp_path and carries the tag", func() {
+		Expect(Verify(write(`
+- name: ds4-honest
+  tags:
+    - mtp
+  overrides:
+    backend: ds4
+    options:
+      - mtp_path:model-mtp.gguf
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(BeEmpty())
+	})
+
+	// sglang declares speculative_algorithm in the referenced gallery/*.yaml,
+	// which Verify never reads, so it may not judge such an entry either way.
+	It("says nothing about an sglang entry tagged mtp", func() {
+		Expect(Verify(write(`
+- name: sglang-mtp
+  tags:
+    - mtp
+  overrides:
+    backend: sglang
+  files: []
+`))).To(BeEmpty())
+	})
+
+	It("says nothing about the tag on an entry with no declared backend", func() {
+		Expect(Verify(write(`
+- name: templated
+  tags:
+    - mtp
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(BeEmpty())
+	})
+
+	// The flat-match branch in unsloth.go appends every match, so a repo
+	// publishing both a plain and a UD Q8_0 renders one entry holding two full
+	// models while model: points at only the first.
+	It("reports an entry holding more than one non-shard weight file", func() {
+		Expect(Verify(write(`
+- name: greedy
+  overrides:
+    backend: llama-cpp
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/repo/Model-Q8_0.gguf
+  files:
+    - filename: llama-cpp/models/repo/Model-Q8_0.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: llama-cpp/models/repo/Model-UD-Q8_0.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(ContainElement(ContainSubstring("more than one weight file")))
+	})
+
+	It("accepts many shards alongside an mmproj and a drafter", func() {
+		Expect(Verify(write(`
+- name: sharded
+  tags:
+    - mtp
+  overrides:
+    backend: llama-cpp
+    options:
+      - spec_type:draft-mtp
+    mmproj: llama-cpp/mmproj/repo/mm.gguf
+    draft_model: llama-cpp/models/repo/Model-draft.gguf
+  files:
+    - filename: llama-cpp/models/repo/Model-00001-of-00002.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: llama-cpp/models/repo/Model-00002-of-00002.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+    - filename: llama-cpp/mmproj/repo/mm.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+    - filename: llama-cpp/models/repo/Model-draft.gguf
+      sha256: dd
+      uri: https://example.com/d.gguf
+`))).To(BeEmpty())
+	})
+
+	// Multi-component TTS and ASR engines legitimately ship an encoder, a
+	// tokenizer, a vocoder and so on as one model, so the collision the weight
+	// count catches does not exist for them.
+	It("accepts a multi-component non-llama-cpp entry declaring five weights", func() {
+		Expect(Verify(write(`
+- name: multi
+  overrides:
+    backend: qwen3-tts-cpp
+  files:
+    - filename: talker.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: tokenizer.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+    - filename: vocoder.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+    - filename: encoder.gguf
+      sha256: dd
+      uri: https://example.com/d.gguf
+    - filename: vae.gguf
+      sha256: ee
+      uri: https://example.com/e.gguf
+`))).To(BeEmpty())
+	})
+
+	It("says nothing about the weight count of an entry with no declared backend", func() {
+		Expect(Verify(write(`
+- name: templated-weights
+  files:
+    - filename: model-Q4_K_M.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: model-mmproj-f16.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(BeEmpty())
+	})
+
+	It("says nothing about an auxiliary metadata file carrying no sha256", func() {
+		Expect(Verify(write(`
+- name: aux
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: params.json
+      sha256: ""
+      uri: https://example.com/params.json
+`))).To(BeEmpty())
+	})
+
+	// safetensors weights are downloaded and loaded exactly like GGUF weights,
+	// so an unverified one is the same supply-chain hole.
+	It("reports a safetensors weight carrying no sha256", func() {
+		Expect(Verify(write(`
+- name: vae
+  files:
+    - filename: wan_2.1_vae.safetensors
+      sha256: ""
+      uri: https://example.com/vae.safetensors
+`))).To(ContainElement(ContainSubstring("no sha256")))
+	})
+
+	It("says nothing about a txt or md file carrying no sha256", func() {
+		Expect(Verify(write(`
+- name: docs
+  files:
+    - filename: notes.txt
+      sha256: ""
+      uri: https://example.com/notes.txt
+    - filename: README.md
+      sha256: ""
+      uri: https://example.com/README.md
+`))).To(BeEmpty())
+	})
+})
+
+var _ = Describe("UnaccountedQuants", func() {
+	// A quant published only as root-level shards matches neither branch in
+	// DiscoverUnslothQuants, so without this diagnostic the build would vanish
+	// from a batch run with nothing said about it.
+	It("reports a wanted quant upstream publishes but discovery dropped", func() {
+		files := []GGUFFile{
+			{Name: "Model-UD-Q4_K_M-00001-of-00003.gguf", SHA256: "aa"},
+			{Name: "Model-UD-Q4_K_M-00002-of-00003.gguf", SHA256: "bb"},
+			{Name: "Model-UD-Q4_K_M-00003-of-00003.gguf", SHA256: "cc"},
+		}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).
+			To(ContainElement(ContainSubstring("UD-Q4_K_M")))
+	})
+
+	It("says nothing when every published wanted quant produced a build", func() {
+		files := []GGUFFile{
+			{Name: "Model-UD-Q4_K_M.gguf", SHA256: "aa"},
+			{Name: "UD-Q6_K/Model-UD-Q6_K-00001-of-00002.gguf", SHA256: "bb"},
+			{Name: "UD-Q6_K/Model-UD-Q6_K-00002-of-00002.gguf", SHA256: "cc"},
+		}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
+	})
+
+	It("says nothing about a wanted quant the repo does not publish at all", func() {
+		files := []GGUFFile{{Name: "Model-UD-Q4_K_M.gguf", SHA256: "aa"}}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
+	})
+
+	// UD-Q8_0 is its own quant label and is not a wanted one. Reading it as a
+	// publication of Q8_0 is the substring collision this diagnostic exists to
+	// warn about, and subdirectory-sharded UD quants are the normal unsloth
+	// layout for large repos, so the false positive would fire on every batch.
+	It("does not read a subdirectory-sharded UD-Q8_0 as a published Q8_0", func() {
+		files := []GGUFFile{
+			{Name: "UD-Q8_0/Model-UD-Q8_0-00001-of-00002.gguf", SHA256: "aa"},
+			{Name: "UD-Q8_0/Model-UD-Q8_0-00002-of-00002.gguf", SHA256: "bb"},
+		}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
+	})
+
+	// A quant in its own subdirectory but not shard-numbered matches neither
+	// branch of DiscoverUnslothQuants, so it is genuinely published and
+	// genuinely undiscovered.
+	It("reports a wanted quant published in its own subdirectory without shard numbering", func() {
+		files := []GGUFFile{{Name: "Q8_0/Model-Q8_0.gguf", SHA256: "aa"}}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).
+			To(ContainElement(ContainSubstring("quant Q8_0 is published upstream")))
+	})
+
+	// builds is empty on purpose: it isolates the file-to-quant match from
+	// whatever DiscoverUnslothQuants would have made of the same file.
+	It("matches the flat single-file layout", func() {
+		files := []GGUFFile{{Name: "Model-Q8_0.gguf", SHA256: "aa"}}
+
+		Expect(UnaccountedQuants(files, nil)).
+			To(ConsistOf(ContainSubstring("quant Q8_0 is published upstream")))
+	})
+})
+
+var _ = Describe("Verify local path collisions", func() {
+	write := func(body string) string {
+		dir := GinkgoT().TempDir()
+		p := filepath.Join(dir, "index.yaml")
+		Expect(os.WriteFile(p, []byte(body), 0o600)).To(Succeed())
+		return p
+	}
+
+	It("reports one local path claimed by two different uris", func() {
+		// The shape that shipped: LiquidAI and unsloth both publish
+		// LFM2.5-8B-A1B-GGUF, so a path built from the bare repo name gives both
+		// entries the same local file under two different checksums.
+		Expect(Verify(write(`
+- name: lfm2.5-8b-a1b
+  files:
+    - filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
+      sha256: 33ab3b8c
+      uri: https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q8_0.gguf
+- name: lfm2.5-8b-a1b-q8-0
+  files:
+    - filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
+      sha256: ec11666b
+      uri: https://huggingface.co/unsloth/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q8_0.gguf
+`))).To(ContainElement(SatisfyAll(
+			ContainSubstring("claimed by two different uris"),
+			ContainSubstring("lfm2.5-8b-a1b-q8-0"),
+		)))
+	})
+
+	It("accepts two entries reusing one file from the same uri", func() {
+		// Sibling builds of one repo legitimately share a projector.
+		Expect(Verify(write(`
+- name: a
+  files:
+    - filename: llama-cpp/mmproj/mudler/Example-GGUF/mmproj-F16.gguf
+      sha256: cc
+      uri: https://huggingface.co/mudler/Example-GGUF/resolve/main/mmproj-F16.gguf
+- name: b
+  files:
+    - filename: llama-cpp/mmproj/mudler/Example-GGUF/mmproj-F16.gguf
+      sha256: cc
+      uri: https://huggingface.co/mudler/Example-GGUF/resolve/main/mmproj-F16.gguf
+`))).To(BeEmpty())
+	})
+
+	It("reports a collision once however many entries pile onto the path", func() {
+		problems := Verify(write(`
+- name: a
+  files:
+    - filename: shared.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: b
+  files:
+    - filename: shared.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+- name: c
+  files:
+    - filename: shared.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+`))
+
+		var collisions int
+		for _, p := range problems {
+			if strings.Contains(p, "claimed by two different uris") {
+				collisions++
+			}
+		}
+		Expect(collisions).To(Equal(1))
+	})
+
+	It("says nothing about files that carry no uri", func() {
+		// A hand-written entry may record only a checksum. There is no upstream
+		// to compare, so the check cannot conclude anything either way.
+		Expect(Verify(write(`
+- name: a
+  files:
+    - filename: shared.gguf
+      sha256: aa
+- name: b
+  files:
+    - filename: shared.gguf
+      sha256: bb
+`))).To(BeEmpty())
+	})
+})

.github/ci/galleryedit/edit.go

@@ -0,0 +1,152 @@
+// Package galleryedit splices variant references into the LocalAI gallery index
+// as TEXT.
+//
+// Re-serialising the index through a YAML marshaller would reflow 40,000 lines,
+// drop the anchors and merge keys the gallery relies on, and produce a diff no
+// reviewer could read, which makes a pull request worthless even when the
+// content inside it is right. Every generator that adds variants to an entry the
+// gallery already ships therefore edits lines, and they share this package so
+// that two of them cannot drift apart on where a variants block belongs.
+package galleryedit
+
+import (
+	"fmt"
+	"regexp"
+	"sort"
+	"strings"
+)
+
+var (
+	entryStart   = regexp.MustCompile(`^-(?: |$)`)
+	inlineName   = regexp.MustCompile(`^- (?:&\S+ )?name:`)
+	keyName      = regexp.MustCompile(`^  name:`)
+	keyVariants  = regexp.MustCompile(`^  variants:\s*(.*)$`)
+	variantItem  = regexp.MustCompile(`^    - `)
+	unsafeInName = regexp.MustCompile(`[:#{}\[\],&*?|>'"%@` + "`" + `]|^\s|\s$`)
+)
+
+// Entry is the positional view of one gallery entry: what it is called and
+// which lines it occupies. Nothing about what the entry MEANS belongs here, so
+// each caller keeps its own semantic decode and only hands over the coordinates.
+type Entry struct {
+	Name string
+	// StartLine and EndLine bound the entry, zero based and half open.
+	StartLine int
+	EndLine   int
+}
+
+// Insert is one entry's pending variants addition. The caller owns the contents
+// of Variants: this package neither orders nor deduplicates them, because the
+// right order and the right dedup rule differ between generators.
+type Insert struct {
+	Entry    Entry
+	Variants []string
+}
+
+// Scan splits index text into lines and reports the line each top level list
+// item begins on.
+func Scan(text string) (lines []string, starts []int) {
+	lines = strings.Split(text, "\n")
+	for i, line := range lines {
+		if entryStart.MatchString(line) {
+			starts = append(starts, i)
+		}
+	}
+	return lines, starts
+}
+
+// Apply splices every insert into the index lines and returns the new text.
+func Apply(lines []string, inserts []Insert) ([]string, error) {
+	type edit struct {
+		at     int
+		remove int
+		insert []string
+	}
+	var edits []edit
+
+	for _, in := range inserts {
+		if len(in.Variants) == 0 {
+			continue
+		}
+
+		items := make([]string, 0, len(in.Variants))
+		for _, v := range in.Variants {
+			items = append(items, "    - model: "+QuoteName(v))
+		}
+
+		at, remove, err := insertionPoint(lines, in.Entry)
+		if err != nil {
+			return nil, err
+		}
+		block := items
+		if remove > 0 || !hasVariantsKey(lines, in.Entry) {
+			block = append([]string{"  variants:"}, items...)
+		}
+		edits = append(edits, edit{at: at, remove: remove, insert: block})
+	}
+
+	// Applying from the bottom up keeps every line number computed against the
+	// original text valid while earlier edits are still pending.
+	sort.Slice(edits, func(i, j int) bool { return edits[i].at > edits[j].at })
+
+	out := append([]string(nil), lines...)
+	for _, e := range edits {
+		tail := append([]string(nil), out[e.at+e.remove:]...)
+		out = append(out[:e.at], append(append([]string(nil), e.insert...), tail...)...)
+	}
+	return out, nil
+}
+
+func hasVariantsKey(lines []string, e Entry) bool {
+	for i := e.StartLine; i < e.EndLine; i++ {
+		if keyVariants.MatchString(lines[i]) {
+			return true
+		}
+	}
+	return false
+}
+
+// insertionPoint reports where new variant items belong, and how many existing
+// lines the insertion replaces.
+//
+// An entry with no variants key gets one right after its name, which is where
+// the hand-written families put it. An entry with an empty "variants: []" has
+// that line replaced by a block. An entry with a block gets its items appended.
+func insertionPoint(lines []string, e Entry) (at int, remove int, err error) {
+	for i := e.StartLine; i < e.EndLine; i++ {
+		m := keyVariants.FindStringSubmatch(lines[i])
+		if m == nil {
+			continue
+		}
+		if strings.TrimSpace(m[1]) == "[]" {
+			return i, 1, nil
+		}
+		if strings.TrimSpace(m[1]) != "" {
+			return 0, 0, fmt.Errorf("entry %q writes its variants inline (%q); this job only edits block lists", e.Name, strings.TrimSpace(m[1]))
+		}
+		last := i
+		for j := i + 1; j < e.EndLine && variantItem.MatchString(lines[j]); j++ {
+			last = j
+		}
+		return last + 1, 0, nil
+	}
+
+	if inlineName.MatchString(lines[e.StartLine]) {
+		return e.StartLine + 1, 0, nil
+	}
+	for i := e.StartLine; i < e.EndLine; i++ {
+		if keyName.MatchString(lines[i]) {
+			return i + 1, 0, nil
+		}
+	}
+	return 0, 0, fmt.Errorf("entry %q has no name line to anchor the insertion to", e.Name)
+}
+
+// QuoteName quotes a variant reference when the name would otherwise change
+// meaning as bare YAML. Config-suffixed names carry a ":" and always need it.
+func QuoteName(name string) string {
+	if unsafeInName.MatchString(name) {
+		return `"` + strings.ReplaceAll(name, `"`, `\"`) + `"`
+	}
+	return name
+}

.github/ci/gen-redirects.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+#
+# Generate client-side redirects for the documentation URLs that used to live at
+# the site root.
+#
+# Until this site existed, the Hugo docs site WAS localai.io, so pages
+# were published at /features/..., /getting-started/..., /faq/ and so on. The
+# docs now build under /docs/, and GitHub Pages serves static files only: there
+# is no server-side rewrite, no .htaccess, no _redirects. The only way to keep
+# every published, bookmarked and search-indexed URL alive is to leave a real
+# HTML file at the old address that sends the browser to the new one.
+#
+# Anything the main site already publishes wins: it owns /, /engines/,
+# /blog/ and friends, so an existing file is never replaced.
+#
+# Usage: gen-redirects.sh <public-dir> [base-url]
+#   public-dir  merged output directory (main site with docs/ inside it)
+#   base-url    absolute or root-relative prefix the deployment is served from,
+#               trailing slash optional (default "/")
+
+set -euo pipefail
+
+PUBLIC_DIR=${1:?usage: gen-redirects.sh <public-dir> [base-url]}
+BASE_URL=${2:-/}
+
+# Normalise to exactly one trailing slash so concatenation below is predictable.
+BASE_URL="${BASE_URL%/}/"
+
+DOCS_DIR="${PUBLIC_DIR}/docs"
+
+if [ ! -d "$DOCS_DIR" ]; then
+  echo "gen-redirects: no docs output at ${DOCS_DIR}" >&2
+  exit 1
+fi
+
+created=0
+skipped=0
+
+# Every .html file is a reachable old URL, not just directory indexes: the
+# generated model gallery ships as a bare gallery.html and used to sit at the
+# root too.
+while IFS= read -r src; do
+  rel=${src#"$DOCS_DIR"/}
+  dst="${PUBLIC_DIR}/${rel}"
+
+  if [ -e "$dst" ]; then
+    skipped=$((skipped + 1))
+    continue
+  fi
+
+  # Link to the directory, not to its index.html, so the redirect target is the
+  # canonical URL the docs site itself advertises.
+  target="${BASE_URL}docs/${rel%index.html}"
+
+  mkdir -p "$(dirname "$dst")"
+  printf '%s' '<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>Moved</title>
+<link rel="canonical" href="'"$target"'">
+<meta name="robots" content="noindex">
+<meta http-equiv="refresh" content="0; url='"$target"'">
+</head>
+<body>
+<p>This page moved to <a href="'"$target"'">'"$target"'</a>.</p>
+</body>
+</html>
+' > "$dst"
+
+  created=$((created + 1))
+done <<EOF
+$(find "$DOCS_DIR" -type f -name '*.html' | sort)
+EOF
+
+echo "gen-redirects: ${created} redirect(s) written, ${skipped} path(s) left to the main site"

.github/ci/refresh-site-counters.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# Refreshes the counters shown on the landing page from the GitHub API.
+#
+# The numbers used to be typed into the templates by hand, which meant they
+# only moved when somebody remembered, and a stale star count on the front
+# page is worse than no star count. Everything the API can answer for lives
+# in website/data/stats.yaml and is rewritten wholesale by this script.
+#
+# Anything the API cannot answer for (the Discord member count) is read back
+# out of the existing file and carried through untouched.
+set -euo pipefail
+
+REPO="${REPO:-mudler/LocalAI}"
+OUT="${OUT:-website/data/stats.yaml}"
+
+# The contributors and releases endpoints are paginated and never report a
+# total. Asking for one item per page makes the last page number equal to the
+# item count, which the Link header hands over.
+count_via_link_header() {
+  local path="$1" link last
+  link=$(gh api -i "${path}?per_page=1" 2>/dev/null | tr -d '\r' | grep -i '^link:' || true)
+  if [ -z "$link" ]; then
+    # No Link header means a single page, so count that page directly.
+    gh api "${path}?per_page=100" --jq 'length'
+    return
+  fi
+  last=$(sed -n 's/.*[?&]page=\([0-9]*\)>; rel="last".*/\1/p' <<<"$link")
+  [ -n "$last" ] || { gh api "${path}?per_page=100" --jq 'length'; return; }
+  printf '%s\n' "$last"
+}
+
+read -r stars forks < <(gh api "repos/${REPO}" --jq '"\(.stargazers_count) \(.forks_count)"')
+contributors=$(count_via_link_header "repos/${REPO}/contributors")
+releases=$(count_via_link_header "repos/${REPO}/releases")
+
+# Not derivable from the GitHub API, so keep whatever is already on disk.
+discord=$(sed -n 's/^discord: *\([0-9]*\).*/\1/p' "$OUT" 2>/dev/null | head -1)
+discord="${discord:-0}"
+
+for n in stars forks contributors releases; do
+  v="${!n}"
+  [[ "$v" =~ ^[0-9]+$ ]] && [ "$v" -gt 0 ] || {
+    echo "refusing to write: ${n} came back as '${v}'" >&2
+    exit 1
+  }
+done
+
+cat > "$OUT" <<YAML
+# Counters shown on the landing page.
+#
+# The four GitHub fields are rewritten by .github/ci/refresh-site-counters.sh,
+# which runs weekly from .github/workflows/refresh-site-counters.yml. Editing
+# them by hand works but will be overwritten on the next run.
+stars: ${stars}
+forks: ${forks}
+contributors: ${contributors}
+releases: ${releases}
+
+# The GitHub API cannot answer for this one, so it is maintained by hand and
+# the refresh script carries it through untouched.
+discord: ${discord}
+YAML
+
+echo "stars=${stars} forks=${forks} contributors=${contributors} releases=${releases} discord=${discord}"

.github/ci/variantproposals/body.go

@@ -0,0 +1,133 @@
+package main
+
+import (
+	"fmt"
+	"strings"
+)
+
+// RenderBody writes the pull request body.
+//
+// The body is the product of this job, not the diff. Grouping is a judgement
+// call that has gone wrong in both directions before, so a reviewer has to be
+// able to accept or reject each family from the body alone, without opening
+// HuggingFace to work out whether two entries hold the same weights.
+func RenderBody(r *Result, ledgerPath string) string {
+	var b strings.Builder
+
+	b.WriteString("## Proposed gallery variant groupings\n\n")
+	b.WriteString("This is a proposal, not a decision. The gallery agent adds one build per model and never joins an existing family, so entries that are alternative builds of the same weights drift apart as the gallery grows. This job re-applies the grouping heuristics from the manual sweeps and asks a human to confirm.\n\n")
+	b.WriteString("Each family below lists the parent, the variants, and the evidence that they are the same weights. **Reject anything whose evidence you do not believe.**\n\n")
+	b.WriteString(fmt.Sprintf("To decline a family permanently, add one line to `%s` in this pull request and close it:\n\n", ledgerPath))
+	b.WriteString("```yaml\npairs:\n  - {parent: some-model, variant: some-model-thing, reason: \"different finetune\"}\n```\n\n")
+
+	b.WriteString(fmt.Sprintf("### Proposed families (%d)\n\n", len(r.Families)))
+	if len(r.Families) == 0 {
+		b.WriteString("None.\n\n")
+	}
+	for _, f := range r.Families {
+		b.WriteString(fmt.Sprintf("#### `%s`\n\n", f.Parent))
+		b.WriteString("| variant | signals | evidence |\n|---|---|---|\n")
+		for _, p := range f.Proposals {
+			b.WriteString(fmt.Sprintf("| `%s` | %s | %s |\n", p.Variant, joinSignals(p.Evidence.Signals), describeEvidence(p.Evidence)))
+		}
+		b.WriteString("\n")
+	}
+
+	b.WriteString(fmt.Sprintf("### Declined by the ledger (%d)\n\n", len(r.Suppressed)))
+	if len(r.Suppressed) == 0 {
+		b.WriteString("Nothing the heuristics found was already on the ledger.\n\n")
+	} else {
+		b.WriteString("Candidates the heuristics found and the ledger has already settled. They are listed so the ledger's effect stays visible rather than silently shrinking the job's output.\n\n")
+		for _, s := range r.Suppressed {
+			b.WriteString(fmt.Sprintf("- `%s` + `%s`: %s\n", s.A, s.B, s.Reason))
+		}
+		b.WriteString("\n")
+	}
+
+	if len(r.AliasSkipped) > 0 {
+		b.WriteString(fmt.Sprintf("### Aliases, not variants (%d)\n\n", len(r.AliasSkipped)))
+		b.WriteString("These entries install byte for byte the same payload. An alias exists so clients can send a particular name; folding it under another entry would hide that name.\n\n")
+		for _, s := range r.AliasSkipped {
+			b.WriteString(fmt.Sprintf("- `%s` + `%s`: %s\n", s.A, s.B, s.Reason))
+		}
+		b.WriteString("\n")
+	}
+
+	if len(r.Refusals) > 0 {
+		b.WriteString(fmt.Sprintf("### Found but refused (%d)\n\n", len(r.Refusals)))
+		b.WriteString("Candidates the heuristics found but the authoring rules would not let this job write. They need a human edit or a rule change.\n\n")
+		for _, ref := range r.Refusals {
+			b.WriteString(fmt.Sprintf("- %s: %s\n", codeList(ref.Members), ref.Reason))
+		}
+		b.WriteString("\n")
+	}
+
+	b.WriteString("---\n\nOpened by `.github/ci/variantproposals`. Heuristics and the rejection ledger live there and in the ledger file; a wrong proposal is a bug in one of the two.\n")
+	return b.String()
+}
+
+func joinSignals(signals []Signal) string {
+	if len(signals) == 0 {
+		return "inferred through another member of the family"
+	}
+	out := make([]string, 0, len(signals))
+	for _, s := range signals {
+		out = append(out, "`"+string(s)+"`")
+	}
+	return strings.Join(out, ", ")
+}
+
+func describeEvidence(e Evidence) string {
+	var parts []string
+	if e.SharedStem != "" {
+		parts = append(parts, fmt.Sprintf("same name once quantization markers are stripped: `%s`", e.SharedStem))
+	}
+	if e.SharedFile != "" {
+		parts = append(parts, fmt.Sprintf("same primary weight filename once quantization markers are stripped: `%s`", e.SharedFile))
+	}
+	if e.SharedRepo != "" {
+		parts = append(parts, fmt.Sprintf("same upstream repo `%s`", e.SharedRepo))
+	}
+	if len(e.QuantTokens) > 0 {
+		parts = append(parts, "differing quantization tokens: `"+strings.Join(e.QuantTokens, "`, `")+"`")
+	}
+	if len(parts) == 0 {
+		return "reached this family through another member"
+	}
+	return strings.Join(parts, "; ")
+}
+
+func codeList(names []string) string {
+	out := make([]string, 0, len(names))
+	for _, n := range names {
+		out = append(out, "`"+n+"`")
+	}
+	return strings.Join(out, " + ")
+}
+
+// RenderSummary is the terminal-facing digest of a run, so the workflow log
+// says what happened without anyone opening the pull request.
+func RenderSummary(r *Result) string {
+	var b strings.Builder
+	fmt.Fprintf(&b, "families proposed: %d\n", len(r.Families))
+	for _, f := range r.Families {
+		names := make([]string, 0, len(f.Proposals))
+		for _, p := range f.Proposals {
+			names = append(names, p.Variant)
+		}
+		fmt.Fprintf(&b, "  %s <- %s\n", f.Parent, strings.Join(names, ", "))
+	}
+	fmt.Fprintf(&b, "declined by ledger: %d\n", len(r.Suppressed))
+	for _, s := range r.Suppressed {
+		fmt.Fprintf(&b, "  %s\n", s)
+	}
+	fmt.Fprintf(&b, "aliases skipped: %d\n", len(r.AliasSkipped))
+	for _, s := range r.AliasSkipped {
+		fmt.Fprintf(&b, "  %s\n", s)
+	}
+	fmt.Fprintf(&b, "refused: %d\n", len(r.Refusals))
+	for _, ref := range r.Refusals {
+		fmt.Fprintf(&b, "  %s: %s\n", strings.Join(ref.Members, " + "), ref.Reason)
+	}
+	return b.String()
+}

.github/ci/variantproposals/edit.go

@@ -0,0 +1,42 @@
+package main
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+// ApplyFamilies writes the proposed variant lists into the index text.
+//
+// The line editing itself lives in galleryedit, shared with the apexentries
+// generator. Both jobs add variants to entries the gallery already ships, and a
+// second answer to "where does a variants block go" would drift from this one;
+// see that package for why the edit is textual rather than a YAML round trip.
+func ApplyFamilies(ix *Index, families []Family) ([]string, error) {
+	byName, _ := ix.ByName()
+
+	var inserts []galleryedit.Insert
+	for _, f := range families {
+		entry, ok := byName[strings.ToLower(f.Parent)]
+		if !ok {
+			return nil, fmt.Errorf("parent %q is not in the index", f.Parent)
+		}
+
+		variants := make([]string, 0, len(f.Proposals))
+		for _, p := range f.Proposals {
+			variants = append(variants, p.Variant)
+		}
+
+		inserts = append(inserts, galleryedit.Insert{
+			Entry: galleryedit.Entry{
+				Name:      entry.Name,
+				StartLine: entry.StartLine,
+				EndLine:   entry.EndLine,
+			},
+			Variants: variants,
+		})
+	}
+
+	return galleryedit.Apply(ix.Lines, inserts)
+}

.github/ci/variantproposals/edit_test.go

@@ -0,0 +1,153 @@
+package main
+
+import (
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("ApplyFamilies", func() {
+	apply := func(ix *Index, families []Family) []string {
+		lines, err := ApplyFamilies(ix, families)
+		ExpectWithOffset(1, err).ToNot(HaveOccurred())
+		return lines
+	}
+
+	// insertedLines is what a reviewer would see in the diff. A textual editor
+	// that reflowed the file would show thousands here, which is the failure
+	// this whole approach exists to avoid.
+	insertedLines := func(before, after []string) int {
+		remaining := map[string]int{}
+		for _, l := range before {
+			remaining[l]++
+		}
+		n := 0
+		for _, l := range after {
+			if remaining[l] > 0 {
+				remaining[l]--
+				continue
+			}
+			n++
+		}
+		return n
+	}
+
+	It("adds a variants block right after the entry's name and touches nothing else", func() {
+		ix := indexOf(
+			entryYAML("foo-model", "acme/repo", "foo-model-Q4_K_M.gguf", "aa"),
+			entryYAML("foo-model-q8_0", "acme/repo", "foo-model-Q8_0.gguf", "bb"),
+		)
+		out := apply(ix, []Family{{Parent: "foo-model", Proposals: []Proposal{{Variant: "foo-model-q8_0"}}}})
+
+		Expect(out[0]).To(Equal("- name: foo-model"))
+		Expect(out[1]).To(Equal("  variants:"))
+		Expect(out[2]).To(Equal("    - model: foo-model-q8_0"))
+		Expect(len(out)).To(Equal(len(ix.Lines) + 2))
+		Expect(insertedLines(ix.Lines, out)).To(Equal(2))
+	})
+
+	It("appends to a variants block that already exists", func() {
+		ix := indexOf(`- name: partial
+  variants:
+    - model: partial-q8_0
+  url: u
+  overrides:
+    parameters:
+      model: partial-Q4_K_M.gguf
+`, entryYAML("partial-f16", "acme/repo", "partial-f16.gguf", "cc"))
+		out := apply(ix, []Family{{Parent: "partial", Proposals: []Proposal{{Variant: "partial-f16"}}}})
+
+		Expect(out[1]).To(Equal("  variants:"))
+		Expect(out[2]).To(Equal("    - model: partial-q8_0"))
+		Expect(out[3]).To(Equal("    - model: partial-f16"))
+		Expect(out[4]).To(Equal("  url: u"))
+	})
+
+	It("replaces an explicit empty list rather than leaving two variants keys", func() {
+		ix := indexOf(`- name: emptied
+  variants: []
+  url: u
+`, entryYAML("emptied-q8_0", "acme/repo", "emptied-Q8_0.gguf", "cc"))
+		out := apply(ix, []Family{{Parent: "emptied", Proposals: []Proposal{{Variant: "emptied-q8_0"}}}})
+
+		Expect(strings.Join(out[:4], "\n")).To(Equal("- name: emptied\n  variants:\n    - model: emptied-q8_0\n  url: u"))
+		Expect(strings.Count(strings.Join(out, "\n"), "variants:")).To(Equal(1))
+	})
+
+	It("quotes a config-suffixed name so the reference stays a string", func() {
+		ix := indexOf(
+			entryYAML("phi-2-chat", "acme/repo", "phi-2-chat-Q4_K_M.gguf", "aa"),
+			entryYAML("phi-2-chat:Q8_0", "acme/repo", "phi-2-chat-Q8_0.gguf", "bb"),
+		)
+		out := apply(ix, []Family{{Parent: "phi-2-chat", Proposals: []Proposal{{Variant: "phi-2-chat:Q8_0"}}}})
+		Expect(out[2]).To(Equal(`    - model: "phi-2-chat:Q8_0"`))
+
+		// The result has to still be a gallery, and the reference has to
+		// resolve to the entry it names.
+		reparsed, err := ParseIndex(strings.Join(out, "\n"))
+		Expect(err).ToNot(HaveOccurred())
+		Expect(reparsed.Entries[0].Variants).To(ConsistOf(VariantRef{Model: "phi-2-chat:Q8_0"}))
+	})
+
+	It("keeps line numbers correct when several entries are edited at once", func() {
+		ix := indexOf(
+			entryYAML("alpha", "acme/repo", "alpha-Q4_K_M.gguf", "aa"),
+			entryYAML("alpha-q8_0", "acme/repo", "alpha-Q8_0.gguf", "bb"),
+			entryYAML("beta", "acme/repo", "beta-Q4_K_M.gguf", "cc"),
+			entryYAML("beta-q8_0", "acme/repo", "beta-Q8_0.gguf", "dd"),
+		)
+		out := apply(ix, []Family{
+			{Parent: "alpha", Proposals: []Proposal{{Variant: "alpha-q8_0"}}},
+			{Parent: "beta", Proposals: []Proposal{{Variant: "beta-q8_0"}}},
+		})
+
+		reparsed, err := ParseIndex(strings.Join(out, "\n"))
+		Expect(err).ToNot(HaveOccurred())
+		Expect(reparsed.Entries).To(HaveLen(4))
+		Expect(reparsed.Entries[0].Variants).To(ConsistOf(VariantRef{Model: "alpha-q8_0"}))
+		Expect(reparsed.Entries[2].Variants).To(ConsistOf(VariantRef{Model: "beta-q8_0"}))
+		Expect(reparsed.Entries[1].Variants).To(BeEmpty())
+		Expect(reparsed.Entries[3].Variants).To(BeEmpty())
+	})
+
+	It("fails loudly rather than editing an entry it cannot find", func() {
+		ix := indexOf(entryYAML("only", "acme/repo", "only-Q4_K_M.gguf", "aa"))
+		_, err := ApplyFamilies(ix, []Family{{Parent: "missing", Proposals: []Proposal{{Variant: "x"}}}})
+		Expect(err).To(MatchError(ContainSubstring("not in the index")))
+	})
+})
+
+var _ = Describe("ParseIndex", func() {
+	It("records the anchor an entry defines and the anchor an entry merges", func() {
+		ix := indexOf(`- &anc
+  name: anchored
+  url: u
+`, `- !!merge <<: *anc
+  name: child
+`)
+		Expect(ix.Entries[0].AnchorName).To(Equal("anc"))
+		Expect(ix.Entries[1].MergesFrom).To(Equal("anc"))
+		Expect(ix.MergeChildren("anc")).To(HaveLen(1))
+	})
+
+	It("carries merged values into the child, so an inherited variants key is visible", func() {
+		ix := indexOf(`- &anc
+  name: anchored
+  url: u
+  variants:
+    - model: something
+`, `- !!merge <<: *anc
+  name: child
+`)
+		Expect(ix.Entries[1].HasVariants()).To(BeTrue())
+	})
+
+	It("refuses a list item that decodes to nothing", func() {
+		// Every line number the editor works from comes from pairing decoded
+		// entries with top level list items. If those two views can disagree,
+		// the editor writes into the wrong entry, so the parse refuses instead.
+		_, err := ParseIndex("- name: one\n  url: u\n-\n")
+		Expect(err).To(MatchError(ContainSubstring("empty")))
+	})
+})

.github/ci/variantproposals/index.go

@@ -0,0 +1,281 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"regexp"
+	"sort"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+// File is the subset of a gallery file entry the proposer reads.
+type File struct {
+	Filename string `yaml:"filename"`
+	URI      string `yaml:"uri"`
+	SHA256   string `yaml:"sha256"`
+}
+
+// VariantRef mirrors the gallery's variant reference.
+type VariantRef struct {
+	Model string `yaml:"model"`
+}
+
+// GalleryEntry is one gallery entry, carrying both the semantics the heuristics need
+// and the text range the editor needs.
+//
+// The two views are kept together deliberately. The editor must not round-trip
+// the index through a YAML marshaller: the gallery is 40,000 lines and a
+// reflowed diff cannot be reviewed, which defeats the entire point of a job
+// whose output is a human decision.
+type GalleryEntry struct {
+	Name       string         `yaml:"name"`
+	URL        string         `yaml:"url"`
+	ConfigFile map[string]any `yaml:"config_file"`
+	Overrides  map[string]any `yaml:"overrides"`
+	Files      []File         `yaml:"files"`
+	Variants   []VariantRef   `yaml:"variants"`
+
+	// Index is the entry's position in gallery order.
+	Index int `yaml:"-"`
+	// StartLine and EndLine bound the entry's lines, zero based and half open.
+	StartLine int `yaml:"-"`
+	EndLine   int `yaml:"-"`
+	// AnchorName is set when the entry defines a YAML anchor. Adding a variants
+	// key to such an entry is inherited by everything that merges it, which is
+	// why proposals involving anchors get special treatment.
+	AnchorName string `yaml:"-"`
+	// MergesFrom is the anchor this entry pulls in with "!!merge <<:".
+	MergesFrom string `yaml:"-"`
+}
+
+// Index is a parsed gallery index: entries plus the exact lines they came from.
+type Index struct {
+	Lines   []string
+	Entries []*GalleryEntry
+}
+
+var (
+	anchorStart = regexp.MustCompile(`^- &(\S+)`)
+	mergeStart  = regexp.MustCompile(`^- !!merge <<: \*(\S+)`)
+)
+
+// LoadIndex reads and parses a gallery index file.
+func LoadIndex(path string) (*Index, error) {
+	data, err := os.ReadFile(path)
+	if err != nil {
+		return nil, err
+	}
+	return ParseIndex(string(data))
+}
+
+// ParseIndex builds an Index from the raw text of a gallery index.
+//
+// The YAML decode and the textual scan are cross checked against each other: if
+// they disagree on how many entries there are, every line number the editor
+// would use is suspect, so the run fails rather than editing the wrong entry.
+func ParseIndex(text string) (*Index, error) {
+	var entries []*GalleryEntry
+	if err := yaml.Unmarshal([]byte(text), &entries); err != nil {
+		return nil, fmt.Errorf("decoding gallery index: %w", err)
+	}
+
+	lines, starts := galleryedit.Scan(text)
+	if len(starts) != len(entries) {
+		return nil, fmt.Errorf("gallery index has %d decoded entries but %d top level list items; refusing to edit by line number", len(entries), len(starts))
+	}
+
+	for i, e := range entries {
+		if e == nil {
+			return nil, fmt.Errorf("gallery index list item %d is empty; refusing to edit by line number", i)
+		}
+		e.Index = i
+		e.StartLine = starts[i]
+		if i+1 < len(starts) {
+			e.EndLine = starts[i+1]
+		} else {
+			e.EndLine = len(lines)
+		}
+		if m := anchorStart.FindStringSubmatch(lines[e.StartLine]); m != nil {
+			e.AnchorName = m[1]
+		}
+		if m := mergeStart.FindStringSubmatch(lines[e.StartLine]); m != nil {
+			e.MergesFrom = m[1]
+		}
+	}
+
+	return &Index{Lines: lines, Entries: entries}, nil
+}
+
+// MergeChildren lists the entries that pull in the given anchor.
+func (ix *Index) MergeChildren(anchor string) []*GalleryEntry {
+	var out []*GalleryEntry
+	for _, e := range ix.Entries {
+		if e.MergesFrom == anchor {
+			out = append(out, e)
+		}
+	}
+	return out
+}
+
+// ByName indexes entries by lowercased name. A name appearing twice keeps the
+// first occurrence, matching the gallery's own first-match-wins resolution, and
+// the duplicates are returned so the caller can refuse to touch them: a
+// proposal naming an ambiguous entry cannot be reviewed.
+func (ix *Index) ByName() (map[string]*GalleryEntry, map[string]int) {
+	byName := make(map[string]*GalleryEntry, len(ix.Entries))
+	counts := make(map[string]int, len(ix.Entries))
+	for _, e := range ix.Entries {
+		key := strings.ToLower(e.Name)
+		counts[key]++
+		if _, seen := byName[key]; !seen {
+			byName[key] = e
+		}
+	}
+	dupes := map[string]int{}
+	for name, n := range counts {
+		if n > 1 {
+			dupes[name] = n
+		}
+	}
+	return byName, dupes
+}
+
+// Installable reports whether installing this entry would put anything on disk.
+// A variant target that installs nothing is a dead end for the selector, so it
+// is never proposed as one.
+func (e *GalleryEntry) Installable() bool {
+	return e.URL != "" || len(e.ConfigFile) > 0 || len(e.Overrides) > 0 || len(e.Files) > 0
+}
+
+// HasVariants reports whether the entry already offers builds of its own. Such
+// an entry cannot be a variant target: nesting is what the gallery's own
+// resolution refuses.
+func (e *GalleryEntry) HasVariants() bool {
+	return len(e.Variants) > 0
+}
+
+// auxiliaryFile matches the shared side files that several unrelated models
+// legitimately hand out the same copy of. Grouping on one of these is how an
+// earlier sweep linked four wan-2.1 entries to each other and Z-Image-Turbo to
+// qwen3-4b: they shared a text encoder, not weights.
+var auxiliaryFile = regexp.MustCompile(`(?i)(mmproj|vae|clip|t5|umt5|text_?encoder|tokenizer|\bae\b|^ae\.|scheduler|config)`)
+
+// IsAuxiliaryFile reports whether a filename is a side file rather than the
+// model's own weights.
+func IsAuxiliaryFile(filename string) bool {
+	base := filename
+	if i := strings.LastIndex(base, "/"); i >= 0 {
+		base = base[i+1:]
+	}
+	return auxiliaryFile.MatchString(base)
+}
+
+// PrimaryWeightFile returns the filename of the entry's own weights, and
+// whether one could be identified unambiguously.
+//
+// The declared overrides.parameters.model wins because that is the file the
+// backend is actually pointed at. Falling back to the file list only works when
+// exactly one non-auxiliary file is present; anything else is ambiguous, and
+// guessing is precisely the failure mode this heuristic has already had.
+func (e *GalleryEntry) PrimaryWeightFile() (string, bool) {
+	if params, ok := e.Overrides["parameters"].(map[string]any); ok {
+		if model, ok := params["model"].(string); ok && model != "" && !IsAuxiliaryFile(model) {
+			return model, true
+		}
+	}
+	var candidates []string
+	for _, f := range e.Files {
+		if f.Filename == "" || IsAuxiliaryFile(f.Filename) {
+			continue
+		}
+		candidates = append(candidates, f.Filename)
+	}
+	if len(candidates) == 1 {
+		return candidates[0], true
+	}
+	return "", false
+}
+
+// SourceRepo returns the upstream repository the entry's files come from, as a
+// coarse "host + owner + repo" key.
+func (e *GalleryEntry) SourceRepo() string {
+	for _, f := range e.Files {
+		if f.URI == "" {
+			continue
+		}
+		return repoKey(f.URI)
+	}
+	return ""
+}
+
+func repoKey(uri string) string {
+	u := strings.ToLower(uri)
+	u = strings.TrimPrefix(u, "huggingface://")
+	u = strings.TrimPrefix(u, "https://huggingface.co/")
+	u = strings.TrimPrefix(u, "http://huggingface.co/")
+	parts := strings.Split(u, "/")
+	if len(parts) >= 2 {
+		return parts[0] + "/" + parts[1]
+	}
+	return u
+}
+
+// SameInstallPayload reports whether two entries install byte for byte the same
+// thing.
+//
+// Entries like this are aliases, not variants. whisper-1 exists so a client
+// speaking the OpenAI API can send that name and get whisper-base; folding it
+// under whisper-base as a variant would hide the very name clients send.
+func SameInstallPayload(a, b *GalleryEntry) bool {
+	if a.URL != b.URL {
+		return false
+	}
+	if !sameYAML(a.Overrides, b.Overrides) || !sameYAML(a.ConfigFile, b.ConfigFile) {
+		return false
+	}
+	return sameChecksums(a.Files, b.Files)
+}
+
+func sameChecksums(a, b []File) bool {
+	if len(a) != len(b) || len(a) == 0 {
+		return false
+	}
+	ha := make([]string, 0, len(a))
+	hb := make([]string, 0, len(b))
+	for _, f := range a {
+		if f.SHA256 == "" {
+			return false
+		}
+		ha = append(ha, f.SHA256)
+	}
+	for _, f := range b {
+		if f.SHA256 == "" {
+			return false
+		}
+		hb = append(hb, f.SHA256)
+	}
+	sort.Strings(ha)
+	sort.Strings(hb)
+	for i := range ha {
+		if ha[i] != hb[i] {
+			return false
+		}
+	}
+	return true
+}
+
+func sameYAML(a, b any) bool {
+	ba, err := yaml.Marshal(a)
+	if err != nil {
+		return false
+	}
+	bb, err := yaml.Marshal(b)
+	if err != nil {
+		return false
+	}
+	return string(ba) == string(bb)
+}

.github/ci/variantproposals/ledger.go

@@ -0,0 +1,180 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"sort"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+// Ledger records the grouping decisions a human has already made against the
+// proposer, so a declined candidate stays declined instead of coming back every
+// night until reviewers stop reading the job's pull requests.
+//
+// It is checked in next to the gallery and is meant to be edited inside the
+// proposal pull request itself: declining a family is adding one flow-mapping
+// line under pairs or groups and closing the PR.
+type Ledger struct {
+	// Tokens are name segments that mark a distinct model rather than another
+	// build of the same one: finetune names, language codes, product suffixes.
+	// A candidate whose two names differ by any of these is never proposed.
+	Tokens []LedgerToken `yaml:"tokens"`
+	// Pairs are individual candidates a human considered and declined. Order
+	// does not matter: the pair is matched both ways round.
+	Pairs []LedgerPair `yaml:"pairs"`
+	// Groups decline every pair drawn from a set at once, for families like a
+	// per-language release where listing each pair would be unreadable.
+	Groups []LedgerGroup `yaml:"groups"`
+}
+
+type LedgerToken struct {
+	Token  string `yaml:"token"`
+	Reason string `yaml:"reason"`
+}
+
+type LedgerPair struct {
+	Parent  string `yaml:"parent"`
+	Variant string `yaml:"variant"`
+	Reason  string `yaml:"reason"`
+}
+
+type LedgerGroup struct {
+	Members []string `yaml:"members"`
+	Reason  string   `yaml:"reason"`
+}
+
+// LoadLedger reads a ledger file. A missing file is not an error: a gallery
+// that has declined nothing yet is a legitimate state, and failing the job over
+// it would only teach people to keep an empty file around.
+func LoadLedger(path string) (*Ledger, error) {
+	data, err := os.ReadFile(path)
+	if os.IsNotExist(err) {
+		return &Ledger{}, nil
+	}
+	if err != nil {
+		return nil, err
+	}
+	return ParseLedger(data)
+}
+
+func ParseLedger(data []byte) (*Ledger, error) {
+	l := &Ledger{}
+	if err := yaml.Unmarshal(data, l); err != nil {
+		return nil, fmt.Errorf("parsing ledger: %w", err)
+	}
+	for i, t := range l.Tokens {
+		if strings.TrimSpace(t.Token) == "" {
+			return nil, fmt.Errorf("ledger tokens[%d] has an empty token", i)
+		}
+	}
+	for i, p := range l.Pairs {
+		if strings.TrimSpace(p.Parent) == "" || strings.TrimSpace(p.Variant) == "" {
+			return nil, fmt.Errorf("ledger pairs[%d] needs both parent and variant", i)
+		}
+	}
+	return l, nil
+}
+
+// Suppression is a ledger hit: why a candidate was not proposed, in words a
+// reviewer can check against the ledger file.
+type Suppression struct {
+	A      string
+	B      string
+	Reason string
+}
+
+func (s Suppression) String() string {
+	return fmt.Sprintf("%s + %s: %s", s.A, s.B, s.Reason)
+}
+
+// Suppresses reports whether the ledger has already declined pairing these two
+// entries, and why.
+//
+// The token rule is applied to the segments the two names do not share. Two
+// builds of the same weights differ only in quantization markers, so any
+// ledgered token showing up in that difference is by construction a claim that
+// the entries are different models.
+func (l *Ledger) Suppresses(a, b string) (Suppression, bool) {
+	la, lb := strings.ToLower(a), strings.ToLower(b)
+	for _, p := range l.Pairs {
+		lp, lv := strings.ToLower(p.Parent), strings.ToLower(p.Variant)
+		if (lp == la && lv == lb) || (lp == lb && lv == la) {
+			return Suppression{A: a, B: b, Reason: p.Reason}, true
+		}
+	}
+	for _, g := range l.Groups {
+		var seenA, seenB bool
+		for _, m := range g.Members {
+			lm := strings.ToLower(m)
+			if lm == la {
+				seenA = true
+			}
+			if lm == lb {
+				seenB = true
+			}
+		}
+		if seenA && seenB {
+			return Suppression{A: a, B: b, Reason: g.Reason}, true
+		}
+	}
+	diff := differingSegments(la, lb)
+	for _, t := range l.Tokens {
+		token := strings.ToLower(strings.TrimSpace(t.Token))
+		if _, ok := diff[token]; ok {
+			reason := t.Reason
+			if reason == "" {
+				reason = fmt.Sprintf("names differ by %q", token)
+			}
+			return Suppression{A: a, B: b, Reason: fmt.Sprintf("%s (token %q)", reason, token)}, true
+		}
+	}
+	return Suppression{}, false
+}
+
+// segments splits a name into the atoms the token rules are written against.
+func segments(name string) []string {
+	fields := strings.FieldsFunc(strings.ToLower(name), func(r rune) bool {
+		return r == '-' || r == '_' || r == '.' || r == ':' || r == '/'
+	})
+	return fields
+}
+
+// differingSegments returns the set of segments present in exactly one of the
+// two names.
+func differingSegments(a, b string) map[string]struct{} {
+	setA := map[string]int{}
+	for _, s := range segments(a) {
+		setA[s]++
+	}
+	setB := map[string]int{}
+	for _, s := range segments(b) {
+		setB[s]++
+	}
+	diff := map[string]struct{}{}
+	for s := range setA {
+		if setB[s] == 0 {
+			diff[s] = struct{}{}
+		}
+	}
+	for s := range setB {
+		if setA[s] == 0 {
+			diff[s] = struct{}{}
+		}
+	}
+	return diff
+}
+
+// SortedSuppressions gives the ledger's effect on one run in a stable order, so
+// the pull request body reads the same way for the same gallery.
+func SortedSuppressions(in []Suppression) []Suppression {
+	out := append([]Suppression(nil), in...)
+	sort.Slice(out, func(i, j int) bool {
+		if out[i].A != out[j].A {
+			return out[i].A < out[j].A
+		}
+		return out[i].B < out[j].B
+	})
+	return out
+}

.github/ci/variantproposals/main.go

@@ -0,0 +1,65 @@
+// Command variant-proposals looks for gallery entries that are alternative
+// builds of the same weights but are not grouped under one another, and writes
+// a proposal for a human to accept or reject.
+//
+// It never decides. Grouping has gone wrong repeatedly in both directions, so
+// the job's value is catching drift and surfacing candidates with their
+// evidence, not automating the call. The scheduled workflow feeds its output to
+// a pull request in the same shape as .github/checksum_checker.sh.
+package main
+
+import (
+	"flag"
+	"fmt"
+	"os"
+	"strings"
+)
+
+func main() {
+	index := flag.String("index", "gallery/index.yaml", "path to the gallery index")
+	ledger := flag.String("ledger", "gallery/variant-exclusions.yaml", "path to the rejection ledger")
+	bodyOut := flag.String("body-out", "", "write the pull request body here")
+	apply := flag.Bool("apply", false, "write the proposed groupings back into the index")
+	flag.Parse()
+
+	if err := run(*index, *ledger, *bodyOut, *apply); err != nil {
+		fmt.Fprintln(os.Stderr, "variant-proposals:", err)
+		os.Exit(1)
+	}
+}
+
+func run(indexPath, ledgerPath, bodyOut string, apply bool) error {
+	ix, err := LoadIndex(indexPath)
+	if err != nil {
+		return err
+	}
+	ledger, err := LoadLedger(ledgerPath)
+	if err != nil {
+		return err
+	}
+
+	result := Propose(ix, ledger)
+	fmt.Print(RenderSummary(result))
+
+	if !result.HasProposals() {
+		// An empty pull request every night is how a proposal job gets muted.
+		fmt.Println("nothing to propose")
+		return nil
+	}
+
+	if bodyOut != "" {
+		if err := os.WriteFile(bodyOut, []byte(RenderBody(result, ledgerPath)), 0o644); err != nil {
+			return err
+		}
+	}
+
+	if !apply {
+		return nil
+	}
+
+	lines, err := ApplyFamilies(ix, result.Families)
+	if err != nil {
+		return err
+	}
+	return os.WriteFile(indexPath, []byte(strings.Join(lines, "\n")), 0o644)
+}

.github/ci/variantproposals/propose.go

@@ -0,0 +1,615 @@
+package main
+
+import (
+	"fmt"
+	"regexp"
+	"sort"
+	"strings"
+)
+
+// Signal names the grouping heuristic that linked two entries.
+type Signal string
+
+const (
+	// SignalName is "same name once quantization markers are stripped".
+	SignalName Signal = "name-modulo-quant"
+	// SignalConfigSuffix is the ":" convention, foo:q8_0 as a build of foo.
+	SignalConfigSuffix Signal = "config-suffix"
+	// SignalWeightFile is "same primary weight filename once quantization
+	// markers are stripped", auxiliary files excluded.
+	SignalWeightFile Signal = "weight-filename"
+)
+
+// Evidence is what a reviewer needs in order to agree or disagree without
+// opening HuggingFace: what the two entries share, and what differs.
+type Evidence struct {
+	Signals     []Signal
+	SharedStem  string
+	SharedFile  string
+	SharedRepo  string
+	QuantTokens []string
+}
+
+// Proposal is one variant target offered to one parent.
+type Proposal struct {
+	Variant  string
+	Evidence Evidence
+}
+
+// Family is a complete proposal: one parent gaining one or more variants.
+type Family struct {
+	Parent    string
+	Proposals []Proposal
+}
+
+// Refusal is a family the heuristics found but the rules would not let through.
+// Refusals are reported rather than dropped: a candidate the job keeps refusing
+// is either a rule worth revisiting or a gallery bug worth fixing.
+type Refusal struct {
+	Members []string
+	Reason  string
+}
+
+// Result is one run of the proposer.
+type Result struct {
+	Families     []Family
+	Refusals     []Refusal
+	Suppressed   []Suppression
+	AliasSkipped []Suppression
+}
+
+// HasProposals reports whether the run found anything to open a pull request
+// about. A job that opens an empty pull request every night is a job people
+// filter out of their inbox.
+func (r *Result) HasProposals() bool {
+	return len(r.Families) > 0
+}
+
+// sizeToken matches a parameter-count marker: 8b, 1.7b, a3b for an active
+// expert count, e2b for the Gemma effective sizes, 8x7b for a mixture.
+//
+// This is a structural rule rather than a ledger entry because it is about the
+// shape of the token, not about any one model. Different parameter sizes were
+// mis-grouped by an earlier sweep and the failure is systematic.
+var sizeToken = regexp.MustCompile(`^(?:[0-9]+(?:\.[0-9]+)?[bm]|[ae][0-9]+(?:\.[0-9]+)?b|[0-9]+x[0-9]+(?:\.[0-9]+)?b)$`)
+
+func differsByParameterSize(a, b string) (string, bool) {
+	for seg := range differingSegments(a, b) {
+		if sizeToken.MatchString(seg) {
+			return seg, true
+		}
+	}
+	return "", false
+}
+
+// genericFileStem lists weight filenames too generic to be evidence of
+// anything. Two entries both shipping "model.safetensors" share a convention,
+// not a set of weights.
+var genericFileStem = map[string]struct{}{
+	"model": {}, "weights": {}, "pytorch_model": {}, "diffusion_pytorch_model": {},
+	"consolidated": {}, "ggml-model": {}, "model-00001-of-00002": {},
+}
+
+// minFileStemLength keeps short, collision-prone filename stems from linking
+// unrelated entries.
+const minFileStemLength = 6
+
+type pair struct {
+	a, b     int
+	evidence Evidence
+}
+
+// Propose runs the grouping heuristics over a gallery index and returns what it
+// would offer a human, what it refused, and what the ledger silenced.
+//
+// Nothing here touches the network or git, and the index is not modified.
+func Propose(ix *Index, ledger *Ledger) *Result {
+	if ledger == nil {
+		ledger = &Ledger{}
+	}
+	result := &Result{}
+
+	byName, dupes := ix.ByName()
+
+	// Existing relationships. A target already claimed must not be claimed
+	// again, and two entries already in one family need no proposal.
+	claimedBy := map[string]string{}
+	familyOf := map[string]string{}
+	for _, e := range ix.Entries {
+		if !e.HasVariants() {
+			continue
+		}
+		familyOf[strings.ToLower(e.Name)] = strings.ToLower(e.Name)
+		for _, v := range e.Variants {
+			target := strings.ToLower(v.Model)
+			if _, taken := claimedBy[target]; !taken {
+				claimedBy[target] = strings.ToLower(e.Name)
+			}
+			familyOf[target] = strings.ToLower(e.Name)
+		}
+	}
+
+	candidates := map[[2]int]*Evidence{}
+
+	addPair := func(i, j int, sig Signal, apply func(*Evidence)) {
+		if i == j {
+			return
+		}
+		if i > j {
+			i, j = j, i
+		}
+		key := [2]int{i, j}
+		ev, ok := candidates[key]
+		if !ok {
+			ev = &Evidence{}
+			candidates[key] = ev
+		}
+		for _, s := range ev.Signals {
+			if s == sig {
+				apply(ev)
+				return
+			}
+		}
+		ev.Signals = append(ev.Signals, sig)
+		apply(ev)
+	}
+
+	// Signal 1 and 2: entries sharing a name stem.
+	byStem := map[string][]int{}
+	for _, e := range ix.Entries {
+		if e.Name == "" {
+			continue
+		}
+		byStem[NameStem(e.Name)] = append(byStem[NameStem(e.Name)], e.Index)
+	}
+	for stem, members := range byStem {
+		if len(members) < 2 {
+			continue
+		}
+		for i := 0; i < len(members); i++ {
+			for j := i + 1; j < len(members); j++ {
+				a, b := ix.Entries[members[i]], ix.Entries[members[j]]
+				sig := SignalName
+				if HasConfigSuffix(a.Name) || HasConfigSuffix(b.Name) {
+					sig = SignalConfigSuffix
+				}
+				// The bare parent carries no marker in its name, so the
+				// evidence would read "differs by q8_0" and say nothing about
+				// what the parent is. The weight filenames fill that in.
+				fa, _ := a.PrimaryWeightFile()
+				fb, _ := b.PrimaryWeightFile()
+				addPair(members[i], members[j], sig, func(ev *Evidence) {
+					ev.SharedStem = stem
+					ev.QuantTokens = quantDifference(a.Name, b.Name, fa, fb)
+				})
+			}
+		}
+	}
+
+	// Signal 3: entries whose own weight file is the same file at a different
+	// quantization. Auxiliary files never take part.
+	byFile := map[string][]int{}
+	for _, e := range ix.Entries {
+		primary, ok := e.PrimaryWeightFile()
+		if !ok {
+			continue
+		}
+		stem := FileStem(primary)
+		if len(stem) < minFileStemLength {
+			continue
+		}
+		if _, generic := genericFileStem[stem]; generic {
+			continue
+		}
+		byFile[stem] = append(byFile[stem], e.Index)
+	}
+	for stem, members := range byFile {
+		if len(members) < 2 {
+			continue
+		}
+		for i := 0; i < len(members); i++ {
+			for j := i + 1; j < len(members); j++ {
+				a, b := ix.Entries[members[i]], ix.Entries[members[j]]
+				// The filename alone is not evidence. Publishers reuse the
+				// upstream filename for finetunes and for models that merely
+				// embed the base weights: bert-embeddings, an ultravox audio
+				// model and a roleplay finetune all ship a file called
+				// llama-3.2-1b-instruct-q4_k_m.gguf. Requiring the same
+				// upstream repository turns the signal back into what it
+				// claims to be, one repo publishing one file at two
+				// quantizations. Two repos holding the same weights is a fact
+				// no filename proves, so it stays a human call.
+				repo := a.SourceRepo()
+				if repo == "" || repo != b.SourceRepo() {
+					continue
+				}
+				fa, _ := a.PrimaryWeightFile()
+				fb, _ := b.PrimaryWeightFile()
+				addPair(members[i], members[j], SignalWeightFile, func(ev *Evidence) {
+					ev.SharedFile = stem
+					ev.SharedRepo = repo
+					if len(ev.QuantTokens) == 0 {
+						ev.QuantTokens = quantDifference(fa, fb)
+					}
+				})
+			}
+		}
+	}
+
+	// Filter candidates. Everything dropped here is dropped for a reason a
+	// reviewer can read back off the ledger or the rules.
+	var kept []pair
+	for key, ev := range candidates {
+		a, b := ix.Entries[key[0]], ix.Entries[key[1]]
+		la, lb := strings.ToLower(a.Name), strings.ToLower(b.Name)
+		if la == lb {
+			continue
+		}
+		if dupes[la] > 0 || dupes[lb] > 0 {
+			result.Refusals = append(result.Refusals, Refusal{
+				Members: []string{a.Name, b.Name},
+				Reason:  "one of these names appears more than once in the gallery, so a variant reference to it is ambiguous",
+			})
+			continue
+		}
+		if fa, fb := familyOf[la], familyOf[lb]; fa != "" && fa == fb {
+			continue
+		}
+		if seg, differs := differsByParameterSize(la, lb); differs {
+			result.Suppressed = append(result.Suppressed, Suppression{
+				A: a.Name, B: b.Name, Reason: fmt.Sprintf("different parameter sizes (segment %q)", seg),
+			})
+			continue
+		}
+		if s, ok := ledger.Suppresses(a.Name, b.Name); ok {
+			result.Suppressed = append(result.Suppressed, s)
+			continue
+		}
+		if SameInstallPayload(a, b) {
+			result.AliasSkipped = append(result.AliasSkipped, Suppression{
+				A: a.Name, B: b.Name,
+				Reason: "identical install payload; these are aliases of one build, not alternative builds",
+			})
+			continue
+		}
+		kept = append(kept, pair{a: key[0], b: key[1], evidence: *ev})
+	}
+
+	sort.Slice(kept, func(i, j int) bool {
+		if kept[i].a != kept[j].a {
+			return kept[i].a < kept[j].a
+		}
+		return kept[i].b < kept[j].b
+	})
+
+	// Components. A pair from either signal joins the same family, so a chain
+	// of alternative builds discovered by different signals stays one family
+	// rather than two overlapping ones that would double claim a target.
+	parent := map[int]int{}
+	var find func(int) int
+	find = func(x int) int {
+		if p, ok := parent[x]; ok && p != x {
+			parent[x] = find(p)
+			return parent[x]
+		}
+		if _, ok := parent[x]; !ok {
+			parent[x] = x
+		}
+		return parent[x]
+	}
+	union := func(x, y int) {
+		rx, ry := find(x), find(y)
+		if rx != ry {
+			parent[ry] = rx
+		}
+	}
+	evidenceFor := map[[2]int]Evidence{}
+	for _, p := range kept {
+		union(p.a, p.b)
+		evidenceFor[[2]int{p.a, p.b}] = p.evidence
+	}
+
+	components := map[int][]int{}
+	for _, p := range kept {
+		for _, m := range []int{p.a, p.b} {
+			root := find(m)
+			if !contains(components[root], m) {
+				components[root] = append(components[root], m)
+			}
+		}
+	}
+
+	roots := make([]int, 0, len(components))
+	for r := range components {
+		roots = append(roots, r)
+	}
+	sort.Ints(roots)
+
+	proposedTargets := map[string]string{}
+	for _, root := range roots {
+		members := components[root]
+		sort.Ints(members)
+		family, refusal := buildFamily(ix, members, evidenceFor, claimedBy, proposedTargets, byName)
+		if refusal != nil {
+			result.Refusals = append(result.Refusals, *refusal)
+			continue
+		}
+		if family == nil {
+			continue
+		}
+		for _, p := range family.Proposals {
+			proposedTargets[strings.ToLower(p.Variant)] = family.Parent
+		}
+		result.Families = append(result.Families, *family)
+	}
+
+	sort.Slice(result.Families, func(i, j int) bool { return result.Families[i].Parent < result.Families[j].Parent })
+	result.Suppressed = SortedSuppressions(result.Suppressed)
+	result.AliasSkipped = SortedSuppressions(result.AliasSkipped)
+	result.Refusals = dedupeRefusals(result.Refusals)
+	return result
+}
+
+// dedupeRefusals collapses the same refusal reached from both orderings of a
+// pair, and sorts what is left. A reviewer reading the same complaint twice
+// learns to skim the section.
+func dedupeRefusals(in []Refusal) []Refusal {
+	seen := map[string]struct{}{}
+	var out []Refusal
+	for _, r := range in {
+		members := append([]string(nil), r.Members...)
+		sort.Strings(members)
+		key := strings.Join(members, "\x00") + "\x00" + r.Reason
+		if _, dup := seen[key]; dup {
+			continue
+		}
+		seen[key] = struct{}{}
+		out = append(out, r)
+	}
+	sort.Slice(out, func(i, j int) bool {
+		if a, b := strings.Join(out[i].Members, ","), strings.Join(out[j].Members, ","); a != b {
+			return a < b
+		}
+		return out[i].Reason < out[j].Reason
+	})
+	return out
+}
+
+func contains(xs []int, x int) bool {
+	for _, v := range xs {
+		if v == x {
+			return true
+		}
+	}
+	return false
+}
+
+// buildFamily turns a connected component into a proposal, or refuses it.
+func buildFamily(ix *Index, members []int, evidenceFor map[[2]int]Evidence, claimedBy map[string]string, proposedTargets map[string]string, byName map[string]*GalleryEntry) (*Family, *Refusal) {
+	names := make([]string, 0, len(members))
+	for _, m := range members {
+		names = append(names, ix.Entries[m].Name)
+	}
+
+	parentIdx, err := selectParent(ix, members)
+	if err != nil {
+		return nil, &Refusal{Members: names, Reason: err.Error()}
+	}
+	parentEntry := ix.Entries[parentIdx]
+	parentName := strings.ToLower(parentEntry.Name)
+
+	// A parent that is itself somebody's variant would create a chain, which
+	// the gallery's own resolution refuses to install.
+	if owner, claimed := claimedBy[parentName]; claimed {
+		return nil, &Refusal{Members: names, Reason: fmt.Sprintf("the natural parent %q is already a variant of %q; proposing it as a parent would nest variants", parentEntry.Name, owner)}
+	}
+	if owner, claimed := proposedTargets[parentName]; claimed {
+		return nil, &Refusal{Members: names, Reason: fmt.Sprintf("the natural parent %q is already proposed as a variant of %q; proposing it as a parent would nest variants", parentEntry.Name, owner)}
+	}
+
+	// Adding a variants key to an anchor is inherited by every entry that
+	// merges it, silently grouping models nobody proposed. Handling that means
+	// editing each merging child too, which is a larger change than this job
+	// should make unsupervised, so it refuses and hands the reviewer the list.
+	if parentEntry.AnchorName != "" {
+		children := ix.MergeChildren(parentEntry.AnchorName)
+		if len(children) > 0 {
+			childNames := make([]string, 0, len(children))
+			for _, c := range children {
+				childNames = append(childNames, c.Name)
+			}
+			return nil, &Refusal{
+				Members: names,
+				Reason: fmt.Sprintf("the parent %q defines YAML anchor &%s, and a variants key added there is inherited by the %d entries that merge it (%s). Grouping this family by hand also means adding an explicit `variants: []` to each of those entries",
+					parentEntry.Name, parentEntry.AnchorName, len(children), strings.Join(childNames, ", ")),
+			}
+		}
+	}
+
+	existing := map[string]struct{}{}
+	for _, v := range parentEntry.Variants {
+		existing[strings.ToLower(v.Model)] = struct{}{}
+	}
+
+	family := &Family{Parent: parentEntry.Name}
+	for _, m := range members {
+		if m == parentIdx {
+			continue
+		}
+		target := ix.Entries[m]
+		lower := strings.ToLower(target.Name)
+		if _, already := existing[lower]; already {
+			continue
+		}
+		if target.HasVariants() {
+			return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q already offers variants of its own, so it cannot itself be a variant target", target.Name)}
+		}
+		if !target.Installable() {
+			return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q has no url, config_file, overrides or files, so it is not independently installable", target.Name)}
+		}
+		if owner, claimed := claimedBy[lower]; claimed && owner != parentName {
+			return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q is already a variant of %q; a target claimed by two parents is not something the gallery resolves predictably", target.Name, owner)}
+		}
+		if owner, claimed := proposedTargets[lower]; claimed && owner != parentEntry.Name {
+			return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q is already proposed as a variant of %q in this same run", target.Name, owner)}
+		}
+		family.Proposals = append(family.Proposals, Proposal{
+			Variant:  target.Name,
+			Evidence: lookupEvidence(evidenceFor, parentIdx, m),
+		})
+	}
+
+	if len(family.Proposals) == 0 {
+		return nil, nil
+	}
+	sort.Slice(family.Proposals, func(i, j int) bool { return family.Proposals[i].Variant < family.Proposals[j].Variant })
+	return family, nil
+}
+
+func lookupEvidence(evidenceFor map[[2]int]Evidence, a, b int) Evidence {
+	if a > b {
+		a, b = b, a
+	}
+	if ev, ok := evidenceFor[[2]int{a, b}]; ok {
+		return ev
+	}
+	// The two entries reached the same family through a third one. Say so
+	// rather than inventing evidence that was never observed for this pair.
+	return Evidence{Signals: []Signal{SignalName}}
+}
+
+// selectParent picks the entry the others should hang off.
+//
+// The bare name wins when there is one: it is the name a user types and the one
+// documentation links to. Otherwise the smallest build wins, judged by the
+// quantization token in the entry's own weight filename, so the default install
+// is the one most hosts can actually run.
+func selectParent(ix *Index, members []int) (int, error) {
+	// The family's own stem: the one the most members reduce to, shortest name
+	// breaking a tie. An entry named exactly that is the bare entry.
+	stemCount := map[string]int{}
+	for _, m := range members {
+		stemCount[NameStem(ix.Entries[m].Name)]++
+	}
+	// Only a stem two or more members reduce to is the family's own stem. A
+	// stem reached by exactly one member is just that member's name, and
+	// treating it as the family stem would crown whichever name happens to be
+	// shortest rather than whichever build is the base one.
+	familyStem := ""
+	for stem, n := range stemCount {
+		if n < 2 {
+			continue
+		}
+		if familyStem == "" || n > stemCount[familyStem] ||
+			(n == stemCount[familyStem] && len(stem) < len(familyStem)) ||
+			(n == stemCount[familyStem] && len(stem) == len(familyStem) && stem < familyStem) {
+			familyStem = stem
+		}
+	}
+
+	var bare []int
+	for _, m := range members {
+		e := ix.Entries[m]
+		if HasConfigSuffix(e.Name) {
+			continue
+		}
+		if strings.ToLower(e.Name) == familyStem {
+			bare = append(bare, m)
+		}
+	}
+	if len(bare) == 1 {
+		return bare[0], nil
+	}
+	if len(bare) > 1 {
+		names := make([]string, 0, len(bare))
+		for _, m := range bare {
+			names = append(names, ix.Entries[m].Name)
+		}
+		return 0, fmt.Errorf("more than one entry is named exactly %q (%s), so which one is the base build is a judgement this job will not make", familyStem, strings.Join(names, ", "))
+	}
+
+	// No shared stem to be named after. An entry whose name every other member
+	// extends is still recognisably the base one, and this is the only handle
+	// left for families whose weights carry no readable quantization token at
+	// all, such as the ONNX builds.
+	if prefix, ok := uniquePrefixMember(ix, members); ok {
+		return prefix, nil
+	}
+
+	best := -1
+	bestWidth := 1 << 20
+	for _, m := range members {
+		e := ix.Entries[m]
+		width := unknownWidth
+		if primary, ok := e.PrimaryWeightFile(); ok {
+			width = BuildWidth(primary)
+		}
+		// Members are visited in gallery order, so a strict comparison leaves
+		// the earliest entry holding a tie and the choice is deterministic.
+		if width < bestWidth {
+			best, bestWidth = m, width
+		}
+	}
+	if best < 0 {
+		return 0, fmt.Errorf("no member could be identified as the smallest build")
+	}
+	if bestWidth == unknownWidth {
+		names := make([]string, 0, len(members))
+		for _, m := range members {
+			names = append(names, ix.Entries[m].Name)
+		}
+		return 0, fmt.Errorf("no member declares a weight file whose quantization can be read (%s), so the smallest build cannot be identified", strings.Join(names, ", "))
+	}
+	return best, nil
+}
+
+// uniquePrefixMember reports the single member whose name every other member's
+// name starts with, if there is exactly one.
+func uniquePrefixMember(ix *Index, members []int) (int, bool) {
+	found := -1
+	for _, m := range members {
+		name := strings.ToLower(ix.Entries[m].Name)
+		isPrefix := true
+		for _, other := range members {
+			if other == m {
+				continue
+			}
+			if !strings.HasPrefix(strings.ToLower(ix.Entries[other].Name), name) {
+				isPrefix = false
+				break
+			}
+		}
+		if !isPrefix {
+			continue
+		}
+		if found >= 0 {
+			return 0, false
+		}
+		found = m
+	}
+	return found, found >= 0
+}
+
+// quantDifference lists the quantization tokens that tell two names apart. It
+// is the compact form of the evidence: "these differ only by q4_k_m vs q8_0".
+func quantDifference(names ...string) []string {
+	var out []string
+	seen := map[string]struct{}{}
+	for _, name := range names {
+		// Filenames arrive here too, so the extension goes first and "/" counts
+		// as a separator. "_" deliberately does not: it holds "q4_k_m" together.
+		trimmed := weightExtension.ReplaceAllString(name, "")
+		for _, seg := range strings.FieldsFunc(strings.ToLower(trimmed), func(r rune) bool { return r == '-' || r == '/' }) {
+			if !IsQuantToken(seg) {
+				continue
+			}
+			if _, ok := seen[seg]; ok {
+				continue
+			}
+			seen[seg] = struct{}{}
+			out = append(out, seg)
+		}
+	}
+	sort.Strings(out)
+	return out
+}

.github/ci/variantproposals/propose_test.go

@@ -0,0 +1,429 @@
+package main
+
+import (
+	"fmt"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// entryYAML writes one gallery entry with a single weight file, which is the
+// shape almost every real entry has. Specs that need something else write the
+// YAML out by hand.
+func entryYAML(name, repo, filename, sha string) string {
+	return fmt.Sprintf(`- name: %s
+  url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  overrides:
+    parameters:
+      model: %s
+  files:
+    - filename: %s
+      uri: huggingface://%s/%s
+      sha256: %s
+`, name, filename, filename, repo, filename, sha)
+}
+
+func indexOf(entries ...string) *Index {
+	ix, err := ParseIndex(strings.Join(entries, ""))
+	ExpectWithOffset(1, err).ToNot(HaveOccurred())
+	return ix
+}
+
+// familyNames flattens a result into "parent <- variant, variant" strings, the
+// form the specs assert against.
+func familyNames(r *Result) []string {
+	out := make([]string, 0, len(r.Families))
+	for _, f := range r.Families {
+		names := make([]string, 0, len(f.Proposals))
+		for _, p := range f.Proposals {
+			names = append(names, p.Variant)
+		}
+		out = append(out, f.Parent+" <- "+strings.Join(names, ", "))
+	}
+	return out
+}
+
+func refusalReasons(r *Result) string {
+	var b strings.Builder
+	for _, ref := range r.Refusals {
+		b.WriteString(strings.Join(ref.Members, " + ") + ": " + ref.Reason + "\n")
+	}
+	return b.String()
+}
+
+func suppressionReasons(r *Result) string {
+	var b strings.Builder
+	for _, s := range r.Suppressed {
+		b.WriteString(s.String() + "\n")
+	}
+	return b.String()
+}
+
+var _ = Describe("Propose", func() {
+	Describe("the grouping signals", func() {
+		It("groups entries whose names differ only by a quantization marker", func() {
+			ix := indexOf(
+				entryYAML("foo-model", "acme/foo-GGUF", "foo-model-Q4_K_M.gguf", "aa"),
+				entryYAML("foo-model-q8_0", "acme/foo-GGUF", "foo-model-Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(ConsistOf("foo-model <- foo-model-q8_0"))
+			Expect(r.Families[0].Proposals[0].Evidence.Signals).To(ContainElement(SignalName))
+			Expect(r.Families[0].Proposals[0].Evidence.SharedStem).To(Equal("foo-model"))
+			Expect(r.Families[0].Proposals[0].Evidence.QuantTokens).To(ContainElements("q4_k_m", "q8_0"))
+		})
+
+		It("groups entries that use the colon config-suffix convention", func() {
+			ix := indexOf(
+				entryYAML("bar-model", "acme/bar-GGUF", "bar-model-Q4_K_M.gguf", "aa"),
+				entryYAML("bar-model:grammar-functioncall", "acme/bar-GGUF", "bar-model-Q4_K_M-grammar.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(ConsistOf("bar-model <- bar-model:grammar-functioncall"))
+			Expect(r.Families[0].Proposals[0].Evidence.Signals).To(ContainElement(SignalConfigSuffix))
+		})
+
+		It("groups entries whose own weight file is the same file at another quantization", func() {
+			// The names share no stem, so only the filename signal can link
+			// these two.
+			ix := indexOf(
+				entryYAML("omni-cpp", "Serveurperso/Omni-GGUF", "omnivoice-base-Q8_0.gguf", "aa"),
+				entryYAML("omni-cpp-hq", "Serveurperso/Omni-GGUF", "omnivoice-base-BF16.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(ConsistOf("omni-cpp <- omni-cpp-hq"))
+			ev := r.Families[0].Proposals[0].Evidence
+			Expect(ev.Signals).To(ConsistOf(SignalWeightFile))
+			Expect(ev.SharedFile).To(Equal("omnivoice-base"))
+			Expect(ev.SharedRepo).To(Equal("serveurperso/omni-gguf"))
+		})
+
+		It("does not let a shared auxiliary file link unrelated models", func() {
+			// Both entries ship the same text encoder. That is a packaging
+			// convention, not evidence of shared weights: this is how an
+			// earlier sweep linked four wan-2.1 entries to each other.
+			ix := indexOf(`- name: wan-2.1-t2v
+  url: u
+  files:
+    - filename: wan-2.1-t2v-Q4_K_M.gguf
+      uri: huggingface://acme/wan/wan-2.1-t2v-Q4_K_M.gguf
+      sha256: aa
+    - filename: umt5-xxl-encoder-Q8_0.gguf
+      uri: huggingface://acme/wan/umt5-xxl-encoder-Q8_0.gguf
+      sha256: cc
+`, `- name: z-image-turbo
+  url: u
+  files:
+    - filename: z-image-turbo-Q4_K_M.gguf
+      uri: huggingface://acme/wan/z-image-turbo-Q4_K_M.gguf
+      sha256: bb
+    - filename: umt5-xxl-encoder-Q8_0.gguf
+      uri: huggingface://acme/wan/umt5-xxl-encoder-Q8_0.gguf
+      sha256: cc
+`)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+		})
+
+		It("does not treat a shared filename in two different repos as evidence", func() {
+			// A finetune republished under the base model's filename is the
+			// most common way this signal misfires.
+			ix := indexOf(
+				entryYAML("llama-3.2-3b-instruct", "hugging-quants/Llama-3.2-3B-Instruct-GGUF", "llama-3.2-3b-instruct-q4_k_m.gguf", "aa"),
+				entryYAML("llama-3.2-3b-shiro-roleplay", "someone/Shiro-GGUF", "Llama-3.2-3B-Instruct.Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+		})
+	})
+
+	Describe("what must never be proposed", func() {
+		It("does not group different parameter sizes that share a prefix", func() {
+			ix := indexOf(
+				entryYAML("qwen3-tts-cpp-0.6b-base", "Serveurperso/Qwen3-TTS-GGUF", "qwen3-tts-talker-Q4_K_M.gguf", "aa"),
+				entryYAML("qwen3-tts-cpp-1.7b-base", "Serveurperso/Qwen3-TTS-GGUF", "qwen3-tts-talker-Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(suppressionReasons(r)).To(ContainSubstring("different parameter sizes"))
+		})
+
+		It("does not group the Gemma effective sizes", func() {
+			ix := indexOf(
+				entryYAML("gemma-4-e2b-it", "google/gemma-GGUF", "gemma-4-it-Q4_K_M.gguf", "aa"),
+				entryYAML("gemma-4-e4b-it", "google/gemma-GGUF", "gemma-4-it-Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(suppressionReasons(r)).To(ContainSubstring("different parameter sizes"))
+		})
+
+		It("does not group entries with a byte-identical install payload", func() {
+			// whisper-1 exists so OpenAI-compatible clients can send that name.
+			// Folding it under whisper-base would hide the name they send.
+			payload := `  url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
+  overrides:
+    parameters:
+      model: ggml-whisper-base.bin
+  files:
+    - filename: ggml-whisper-base.bin
+      uri: huggingface://ggerganov/whisper.cpp/ggml-base.bin
+      sha256: aa
+`
+			ix := indexOf("- name: whisper-base\n"+payload, "- name: whisper-1\n"+payload)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(r.AliasSkipped).To(HaveLen(1))
+			Expect(r.AliasSkipped[0].Reason).To(ContainSubstring("aliases"))
+		})
+
+		DescribeTable("declines the categories the ledger records",
+			func(nameA, nameB string, ledgerYAML string) {
+				ix := indexOf(
+					entryYAML(nameA, "acme/repo", "shared-weights-Q4_K_M.gguf", "aa"),
+					entryYAML(nameB, "acme/repo", "shared-weights-Q8_0.gguf", "bb"),
+				)
+				ledger, err := ParseLedger([]byte(ledgerYAML))
+				Expect(err).ToNot(HaveOccurred())
+
+				// Without the ledger these would be proposed, which is what
+				// makes the ledger load bearing rather than decorative.
+				Expect(familyNames(Propose(ix, nil))).ToNot(BeEmpty())
+
+				r := Propose(ix, ledger)
+				Expect(familyNames(r)).To(BeEmpty())
+				Expect(r.Suppressed).To(HaveLen(1))
+			},
+			Entry("a finetune", "base-model", "base-model-abliterated",
+				"tokens:\n  - {token: abliterated, reason: finetune}\n"),
+			Entry("a distill", "base-model", "base-model-distilled",
+				"tokens:\n  - {token: distilled, reason: distilled}\n"),
+			Entry("English-only versus multilingual ASR", "whisper-small", "whisper-small-en",
+				"pairs:\n  - {parent: whisper-small, variant: whisper-small-en, reason: English-only versus multilingual}\n"),
+			Entry("two products sharing a prefix", "vibevoice-cpp", "vibevoice-cpp-asr",
+				"pairs:\n  - {parent: vibevoice-cpp, variant: vibevoice-cpp-asr, reason: different products}\n"),
+			Entry("a per-language release", "kokoros-de", "kokoros-ja",
+				"groups:\n  - {members: [kokoros, kokoros-de, kokoros-ja], reason: different languages}\n"),
+		)
+
+		It("reports the ledger's reason so its effect stays visible", func() {
+			ix := indexOf(
+				entryYAML("base-model", "acme/repo", "shared-weights-Q4_K_M.gguf", "aa"),
+				entryYAML("base-model-heretic", "acme/repo", "shared-weights-Q8_0.gguf", "bb"),
+			)
+			ledger, err := ParseLedger([]byte("tokens:\n  - {token: heretic, reason: \"finetune, not a re-quantization\"}\n"))
+			Expect(err).ToNot(HaveOccurred())
+			r := Propose(ix, ledger)
+			Expect(suppressionReasons(r)).To(ContainSubstring("finetune, not a re-quantization"))
+			Expect(suppressionReasons(r)).To(ContainSubstring(`token "heretic"`))
+		})
+	})
+
+	Describe("parent selection", func() {
+		It("picks the bare-named entry when one exists", func() {
+			ix := indexOf(
+				entryYAML("base-model-q8_0", "acme/repo", "base-model-Q8_0.gguf", "aa"),
+				entryYAML("base-model", "acme/repo", "base-model-Q4_K_M.gguf", "bb"),
+				entryYAML("base-model-f16", "acme/repo", "base-model-f16.gguf", "cc"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(ConsistOf("base-model <- base-model-f16, base-model-q8_0"))
+		})
+
+		It("picks the smallest build when no entry is bare-named", func() {
+			ix := indexOf(
+				entryYAML("ced-base-f16", "acme/repo", "ced-base-f16.gguf", "aa"),
+				entryYAML("ced-base-q8", "acme/repo", "ced-base-Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(ConsistOf("ced-base-q8 <- ced-base-f16"))
+		})
+
+		It("judges the smallest build by the quantization in the model filename, not the name", func() {
+			// The names carry no marker at all; only the filenames say which
+			// build is which.
+			ix := indexOf(
+				entryYAML("thing-hq", "acme/repo", "thing-weights-BF16.gguf", "aa"),
+				entryYAML("thing-lite", "acme/repo", "thing-weights-Q4_K_M.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(ConsistOf("thing-lite <- thing-hq"))
+		})
+	})
+
+	Describe("the rules a proposal has to respect", func() {
+		It("refuses to nest: a target that already offers variants of its own", func() {
+			ix := indexOf(
+				entryYAML("nest-model", "acme/repo", "nest-model-Q4_K_M.gguf", "aa"),
+				`- name: nest-model-q8_0
+  url: u
+  variants:
+    - model: nest-model-q8_0-mtp
+  overrides:
+    parameters:
+      model: nest-model-Q8_0.gguf
+  files:
+    - filename: nest-model-Q8_0.gguf
+      uri: huggingface://acme/repo/nest-model-Q8_0.gguf
+      sha256: bb
+`,
+				entryYAML("nest-model-q8_0-mtp", "other/repo", "nest-model-mtp.gguf", "cc"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(refusalReasons(r)).To(ContainSubstring("already offers variants of its own"))
+		})
+
+		It("refuses to nest: a parent that is already somebody else's variant", func() {
+			ix := indexOf(
+				`- name: outer
+  url: u
+  variants:
+    - model: middle
+  overrides:
+    parameters:
+      model: outer-Q4_K_M.gguf
+  files:
+    - filename: outer-Q4_K_M.gguf
+      uri: huggingface://acme/repo/outer-Q4_K_M.gguf
+      sha256: aa
+`,
+				entryYAML("middle", "acme/other", "middle-Q4_K_M.gguf", "bb"),
+				entryYAML("middle-q8_0", "acme/other", "middle-Q8_0.gguf", "cc"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(refusalReasons(r)).To(ContainSubstring("would nest variants"))
+		})
+
+		It("refuses to let two parents claim one target", func() {
+			ix := indexOf(
+				`- name: claimant
+  url: u
+  variants:
+    - model: contested-q8_0
+  overrides:
+    parameters:
+      model: claimant-Q4_K_M.gguf
+  files:
+    - filename: claimant-Q4_K_M.gguf
+      uri: huggingface://acme/repo/claimant-Q4_K_M.gguf
+      sha256: aa
+`,
+				entryYAML("contested", "acme/other", "contested-Q4_K_M.gguf", "bb"),
+				entryYAML("contested-q8_0", "acme/other", "contested-Q8_0.gguf", "cc"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(refusalReasons(r)).To(ContainSubstring("already a variant of"))
+		})
+
+		It("refuses a target that is not independently installable", func() {
+			ix := indexOf(
+				entryYAML("stub-model", "acme/repo", "stub-model-Q4_K_M.gguf", "aa"),
+				"- name: stub-model-q8_0\n  description: a stanza nobody finished\n",
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(refusalReasons(r)).To(ContainSubstring("not independently installable"))
+		})
+
+		It("refuses a family whose parent defines a merge anchor, naming the entries that would inherit", func() {
+			ix := indexOf(
+				`- &anchored
+  name: anchored-model
+  url: u
+  overrides:
+    parameters:
+      model: anchored-Q4_K_M.gguf
+  files:
+    - filename: anchored-Q4_K_M.gguf
+      uri: huggingface://acme/repo/anchored-Q4_K_M.gguf
+      sha256: aa
+`,
+				`- !!merge <<: *anchored
+  name: anchored-child
+  variants: []
+  overrides:
+    parameters:
+      model: unrelated-child-Q4_K_M.gguf
+  files:
+    - filename: unrelated-child-Q4_K_M.gguf
+      uri: huggingface://other/repo/unrelated-child-Q4_K_M.gguf
+      sha256: cc
+`,
+				entryYAML("anchored-model-q8_0", "acme/repo", "anchored-Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(refusalReasons(r)).To(ContainSubstring("defines YAML anchor &anchored"))
+			Expect(refusalReasons(r)).To(ContainSubstring("anchored-child"))
+			Expect(refusalReasons(r)).To(ContainSubstring("variants: []"))
+		})
+
+		It("refuses an entry whose name is not unique in the gallery", func() {
+			ix := indexOf(
+				entryYAML("twin", "acme/repo", "twin-Q4_K_M.gguf", "aa"),
+				entryYAML("twin", "acme/repo", "twin-Q4_K_M.gguf", "aa"),
+				entryYAML("twin-q8_0", "acme/repo", "twin-Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(BeEmpty())
+			Expect(refusalReasons(r)).To(ContainSubstring("appears more than once"))
+		})
+
+		It("says nothing about a pair that is already grouped", func() {
+			ix := indexOf(
+				`- name: settled
+  url: u
+  variants:
+    - model: settled-q8_0
+  overrides:
+    parameters:
+      model: settled-Q4_K_M.gguf
+  files:
+    - filename: settled-Q4_K_M.gguf
+      uri: huggingface://acme/repo/settled-Q4_K_M.gguf
+      sha256: aa
+`,
+				entryYAML("settled-q8_0", "acme/repo", "settled-Q8_0.gguf", "bb"),
+			)
+			r := Propose(ix, nil)
+			Expect(r.HasProposals()).To(BeFalse())
+			Expect(r.Refusals).To(BeEmpty())
+			Expect(r.Suppressed).To(BeEmpty())
+		})
+
+		It("adds only the missing members to a family that already exists", func() {
+			ix := indexOf(
+				`- name: partial
+  url: u
+  variants:
+    - model: partial-q8_0
+  overrides:
+    parameters:
+      model: partial-Q4_K_M.gguf
+  files:
+    - filename: partial-Q4_K_M.gguf
+      uri: huggingface://acme/repo/partial-Q4_K_M.gguf
+      sha256: aa
+`,
+				entryYAML("partial-q8_0", "acme/repo", "partial-Q8_0.gguf", "bb"),
+				entryYAML("partial-f16", "acme/repo", "partial-f16.gguf", "cc"),
+			)
+			r := Propose(ix, nil)
+			Expect(familyNames(r)).To(ConsistOf("partial <- partial-f16"))
+		})
+	})
+
+	It("does not modify the index it was given", func() {
+		text := entryYAML("foo-model", "acme/foo-GGUF", "foo-model-Q4_K_M.gguf", "aa") +
+			entryYAML("foo-model-q8_0", "acme/foo-GGUF", "foo-model-Q8_0.gguf", "bb")
+		ix, err := ParseIndex(text)
+		Expect(err).ToNot(HaveOccurred())
+		before := strings.Join(ix.Lines, "\n")
+		Propose(ix, nil)
+		Expect(strings.Join(ix.Lines, "\n")).To(Equal(before))
+	})
+})

.github/ci/variantproposals/quant.go

@@ -0,0 +1,151 @@
+package main
+
+import (
+	"regexp"
+	"strconv"
+	"strings"
+)
+
+// Quantization and precision markers that distinguish one build of a set of
+// weights from another build of the same weights. Stripping them from a name
+// is what lets the proposer notice that two entries are the same model.
+//
+// qat and apex are in this list on a maintainer ruling: they are quantization
+// techniques applied to published weights, not separate weights. Names that use
+// "apex" to mean a finetune are handled by the rejection ledger instead, because
+// no amount of pattern matching can tell the two uses apart.
+const quantAlternation = `q[2-8](?:_[0-9a-z]+)*|pq[2-8](?:_[0-9a-z]+)*|iq[1-9][0-9a-z]*(?:_[0-9a-z]+)*|i1|` +
+	`f16|f32|bf16|fp16|fp32|fp8|fp4|nvfp4|mxfp4(?:_moe)*|awq|gptq|qat|apex|gguf|ggml|[0-9]+bit|g[0-9]+`
+
+// quantSegment matches a whole hyphen-delimited segment of an entry name.
+// Names separate their parts with "-" and keep quantization tokens internally
+// joined with "_", so a segment is the right unit here: "q4_k_m" arrives whole.
+var quantSegment = regexp.MustCompile(`^(?:` + quantAlternation + `)$`)
+
+// quantFileSuffix matches a trailing quantization token in a weight filename.
+// Filenames mix "-", "_" and "." as separators, so unlike entry names they
+// cannot be split into segments up front without tearing "Q4_K_M" apart.
+var quantFileSuffix = regexp.MustCompile(`(?i)[-_.](?:` + quantAlternation + `)$`)
+
+var weightExtension = regexp.MustCompile(`(?i)\.(gguf|ggml|safetensors|bin|pt|pth|onnx)$`)
+
+// IsQuantToken reports whether a single name segment is a quantization or
+// precision marker rather than part of the model's identity.
+func IsQuantToken(segment string) bool {
+	return quantSegment.MatchString(strings.ToLower(segment))
+}
+
+// NameStem reduces an entry name to the identity it shares with its alternative
+// builds: the config suffix after ":" is dropped, then trailing quantization
+// segments are stripped.
+//
+// It implements the first two grouping signals together because they answer the
+// same question. "foo:q8_0" and "foo-q8_0" are both alternative builds of "foo",
+// and the caller that needs to report which convention was used can compare the
+// name against the stem itself.
+//
+// At least one segment always survives, so a name made entirely of quantization
+// tokens does not collapse to the empty stem and swallow every other such name.
+func NameStem(name string) string {
+	base := strings.ToLower(strings.TrimSpace(name))
+	if i := strings.Index(base, ":"); i >= 0 {
+		base = base[:i]
+	}
+	segments := strings.Split(base, "-")
+	for len(segments) > 1 && quantSegment.MatchString(segments[len(segments)-1]) {
+		segments = segments[:len(segments)-1]
+	}
+	return strings.Join(segments, "-")
+}
+
+// HasConfigSuffix reports whether a name uses the ":" convention for naming a
+// config variant of another entry.
+func HasConfigSuffix(name string) bool {
+	return strings.Contains(name, ":")
+}
+
+// FileStem reduces a weight filename to the identity shared by its other
+// quantizations: directories, extension and trailing quantization tokens go.
+//
+// This is the third grouping signal. It is the one that has misfired before, so
+// callers must filter auxiliary files out before handing a filename here: a
+// shared text encoder is not evidence of shared weights.
+func FileStem(filename string) string {
+	base := filename
+	if i := strings.LastIndex(base, "/"); i >= 0 {
+		base = base[i+1:]
+	}
+	base = weightExtension.ReplaceAllString(base, "")
+	for {
+		stripped := quantFileSuffix.ReplaceAllString(base, "")
+		if stripped == base {
+			break
+		}
+		base = stripped
+	}
+	return strings.ToLower(base)
+}
+
+// bitsPerWeight ranks quantization tokens so the smallest build of a family can
+// be identified when no bare-named entry exists to be the parent.
+//
+// The figures are nominal bits per weight, not measured file sizes. Ranking is
+// all that is asked of them, and a nominal figure is available from the name
+// alone without downloading anything.
+func bitsPerWeight(token string) (int, bool) {
+	t := strings.ToLower(token)
+	switch {
+	case t == "i1":
+		return 1, true
+	case strings.HasPrefix(t, "nvfp4"), strings.HasPrefix(t, "mxfp4"), t == "fp4":
+		return 4, true
+	case t == "fp8":
+		return 8, true
+	case t == "f16", t == "bf16", t == "fp16":
+		return 16, true
+	case t == "f32", t == "fp32":
+		return 32, true
+	case t == "awq", t == "gptq":
+		return 4, true
+	}
+	if m := regexp.MustCompile(`^p?q([1-9])`).FindStringSubmatch(t); m != nil {
+		n, _ := strconv.Atoi(m[1])
+		return n, true
+	}
+	if m := regexp.MustCompile(`^iq([1-9])`).FindStringSubmatch(t); m != nil {
+		n, _ := strconv.Atoi(m[1])
+		return n, true
+	}
+	if m := regexp.MustCompile(`^([0-9]+)bit$`).FindStringSubmatch(t); m != nil {
+		n, _ := strconv.Atoi(m[1])
+		return n, true
+	}
+	return 0, false
+}
+
+// unknownWidth sorts after every recognised quantization so an entry whose
+// build cannot be read from its filename never wins the "smallest build" tie
+// break by accident.
+const unknownWidth = 1 << 10
+
+// BuildWidth reports the nominal bits per weight of the build a filename holds.
+// An unreadable filename gets unknownWidth.
+func BuildWidth(filename string) int {
+	base := filename
+	if i := strings.LastIndex(base, "/"); i >= 0 {
+		base = base[i+1:]
+	}
+	base = weightExtension.ReplaceAllString(base, "")
+	best := unknownWidth
+	for {
+		m := quantFileSuffix.FindString(base)
+		if m == "" {
+			break
+		}
+		if bits, ok := bitsPerWeight(m[1:]); ok && bits < best {
+			best = bits
+		}
+		base = base[:len(base)-len(m)]
+	}
+	return best
+}

.github/ci/variantproposals/quant_test.go

@@ -0,0 +1,92 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("quantization markers", func() {
+	DescribeTable("NameStem strips the markers that distinguish builds, not models",
+		func(name, expected string) {
+			Expect(NameStem(name)).To(Equal(expected))
+		},
+		Entry("plain q4", "foo-model-q4_k_m", "foo-model"),
+		Entry("q8_0", "foo-model-q8_0", "foo-model"),
+		Entry("q5_1", "foo-model-q5_1", "foo-model"),
+		Entry("q2 with group size", "ternary-bonsai-8b-q2-g64", "ternary-bonsai-8b"),
+		Entry("iq variant", "ideogram-4-iq4nl-ggml", "ideogram-4"),
+		Entry("i1 imatrix", "orca-agent-v0.1-i1", "orca-agent-v0.1"),
+		Entry("f16", "ced-base-f16", "ced-base"),
+		Entry("bf16", "some-model-bf16", "some-model"),
+		Entry("fp8", "some-model-fp8", "some-model"),
+		Entry("nvfp4", "qwen3.6-27b-nvfp4", "qwen3.6-27b"),
+		Entry("mxfp4_moe", "huihui-qwen3-vl-30b-a3b-instruct-abliterated-mxfp4_moe", "huihui-qwen3-vl-30b-a3b-instruct-abliterated"),
+		Entry("pq2", "ternary-bonsai-8b-pq2", "ternary-bonsai-8b"),
+		Entry("awq", "some-model-awq", "some-model"),
+		Entry("gptq", "some-model-gptq", "some-model"),
+		Entry("Nbit", "qwen3-8b-mlx-4bit", "qwen3-8b-mlx"),
+		Entry("gguf", "some-model-gguf", "some-model"),
+		Entry("ggml", "flux.1-dev-ggml", "flux.1-dev"),
+		Entry("qat is a quantization technique", "gemma-3-27b-it-qat", "gemma-3-27b-it"),
+		Entry("apex is a quantization technique", "qwen3.6-35b-a3b-apex", "qwen3.6-35b-a3b"),
+		Entry("stacked markers", "gemma-4-e2b-it-qat-q4_0", "gemma-4-e2b-it"),
+		Entry("the config suffix is dropped", "phi-2-chat:Q8_0", "phi-2-chat"),
+		Entry("a non-quant config suffix is dropped too", "meta-llama-3.1-8b-instruct:grammar-functioncall", "meta-llama-3.1-8b-instruct"),
+	)
+
+	DescribeTable("NameStem leaves alone what identifies a different model",
+		func(name, expected string) {
+			Expect(NameStem(name)).To(Equal(expected))
+		},
+		Entry("parameter size", "qwen3-tts-cpp-0.6b-base", "qwen3-tts-cpp-0.6b-base"),
+		Entry("language suffix", "kokoros-de", "kokoros-de"),
+		Entry("English-only ASR", "whisper-small-en", "whisper-small-en"),
+		Entry("finetune", "qwen3-30b-a3b-abliterated", "qwen3-30b-a3b-abliterated"),
+		Entry("product suffix", "vibevoice-cpp-asr", "vibevoice-cpp-asr"),
+	)
+
+	It("never strips a name down to nothing", func() {
+		Expect(NameStem("q4_k_m")).To(Equal("q4_k_m"))
+		Expect(NameStem("f16-q8_0")).To(Equal("f16"))
+	})
+
+	DescribeTable("FileStem reduces a weight filename to the weights it holds",
+		func(filename, expected string) {
+			Expect(FileStem(filename)).To(Equal(expected))
+		},
+		Entry("directory and extension go", "bonsai/models/Ternary-Bonsai-8B-gguf/Ternary-Bonsai-8B-Q2_0.gguf", "ternary-bonsai-8b"),
+		Entry("underscored quant token stays whole", "Llama-3.2-1B-Instruct-Q4_K_M.gguf", "llama-3.2-1b-instruct"),
+		Entry("dot separated quant token", "Llama-3.2-3B-Instruct.Q4_K_M.gguf", "llama-3.2-3b-instruct"),
+		Entry("group size suffix", "Ternary-Bonsai-8B-Q2_0_g64.gguf", "ternary-bonsai-8b"),
+		Entry("bf16", "omnivoice-cpp-hq/omnivoice-base-BF16.gguf", "omnivoice-base"),
+		Entry("safetensors", "some/dir/Model-Name-fp8.safetensors", "model-name"),
+	)
+
+	DescribeTable("BuildWidth reads the nominal width out of a filename",
+		func(filename string, expected int) {
+			Expect(BuildWidth(filename)).To(Equal(expected))
+		},
+		Entry("q4", "foo-Q4_K_M.gguf", 4),
+		Entry("q8", "foo-Q8_0.gguf", 8),
+		Entry("q2", "foo-Q2_0.gguf", 2),
+		Entry("f16", "foo-f16.gguf", 16),
+		Entry("bf16", "foo-BF16.gguf", 16),
+		Entry("iq3", "foo-iq3_xxs.gguf", 3),
+		Entry("nothing readable sorts last", "foo.gguf", unknownWidth),
+	)
+
+	It("treats an auxiliary file as never being the model's own weights", func() {
+		for _, f := range []string{
+			"mmproj-model-f16.gguf",
+			"dir/vae-BF16.gguf",
+			"clip_l.safetensors",
+			"umt5-xxl-encoder-Q8_0.gguf",
+			"t5xxl_fp16.safetensors",
+			"ae.safetensors",
+			"omnivoice-tokenizer-Q8_0.gguf",
+		} {
+			Expect(IsAuxiliaryFile(f)).To(BeTrue(), "expected %q to be auxiliary", f)
+		}
+		Expect(IsAuxiliaryFile("gemma-3-27b-it-Q4_K_M.gguf")).To(BeFalse())
+	})
+})

.github/ci/variantproposals/variantproposals_suite_test.go

@@ -0,0 +1,13 @@
+package main
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestVariantProposals(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "gallery variant proposals")
+}

.github/dependabot.yml

@@ -45,6 +45,16 @@ updates:
     directory: "/backend/python/diffusers"
     schedule:
       interval: "weekly"
+    # torch and transformers are deliberately pinned in this backend (see
+    # backend/python/diffusers/requirements-*.txt and issue #9979), and the
+    # l4t12 variant resolves them from the Jetson pip index
+    # (https://pypi.jetson-ai-lab.io/jp6/cu129/). dependabot cannot authenticate
+    # against that index and fails the whole weekly update with a
+    # private_source_authentication_failure. Ignore the two pinned deps we don't
+    # want bumped anyway so the job stays green.
+    ignore:
+    - dependency-name: "torch"
+    - dependency-name: "transformers"
   - package-ecosystem: "pip"
     directory: "/backend/python/exllama"
     schedule:

.github/gh_curl.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+# Shared curl wrapper for the nightly dependency-bump scripts.
+#
+# The bump workflow fans out to ~25 parallel matrix jobs, each querying
+# api.github.com. Anonymous API calls are capped at 60/hour per source IP and
+# GitHub-hosted runners egress through shared NAT addresses, so a random handful
+# of jobs were getting rate-limited (HTTP 403 -> curl exit 22, empty response)
+# every single night. Authenticating with GITHUB_TOKEN lifts the ceiling to
+# 1000/hour; the retries absorb whatever transient blips remain.
+
+# Wraps curl with GitHub auth (when a token is present) plus retry/timeout
+# hardening. Callers pass their own headers and the URL.
+gh_curl() {
+    # The bump scripts run under `set -x`; without this the Authorization header
+    # would be echoed into the job log on every call.
+    local had_xtrace=0
+    case "$-" in
+        *x*) had_xtrace=1; set +x ;;
+    esac
+
+    local args=(
+        --silent --show-error --location --fail
+        # --retry-all-errors so 403 rate-limit responses are retried too; plain
+        # --retry only covers 408/429/5xx. curl honours Retry-After when sent.
+        --retry 5 --retry-delay 3 --retry-all-errors
+        --connect-timeout 15 --max-time 60
+    )
+    if [ -n "${GITHUB_TOKEN:-}" ]; then
+        args+=(--header "Authorization: Bearer ${GITHUB_TOKEN}")
+    fi
+
+    curl "${args[@]}" "$@"
+    local rc=$?
+
+    if [ "$had_xtrace" -eq 1 ]; then
+        set -x
+    fi
+    return $rc
+}

.github/workflows/backend.yml

@@ -32,16 +32,30 @@ jobs:
     if: github.repository == 'mudler/LocalAI'
     runs-on: ubuntu-latest
     outputs:
-      matrix-singlearch: ${{ steps.set-matrix.outputs['matrix-singlearch'] }}
       matrix-multiarch: ${{ steps.set-matrix.outputs['matrix-multiarch'] }}
       matrix-darwin: ${{ steps.set-matrix.outputs['matrix-darwin'] }}
       merge-matrix-multiarch: ${{ steps.set-matrix.outputs['merge-matrix-multiarch'] }}
-      merge-matrix-singlearch: ${{ steps.set-matrix.outputs['merge-matrix-singlearch'] }}
-      has-backends-singlearch: ${{ steps.set-matrix.outputs['has-backends-singlearch'] }}
       has-backends-multiarch: ${{ steps.set-matrix.outputs['has-backends-multiarch'] }}
       has-backends-darwin: ${{ steps.set-matrix.outputs['has-backends-darwin'] }}
       has-merges-multiarch: ${{ steps.set-matrix.outputs['has-merges-multiarch'] }}
-      has-merges-singlearch: ${{ steps.set-matrix.outputs['has-merges-singlearch'] }}
+      # Single-arch backends are sharded across SINGLEARCH_SHARDS matrix jobs to
+      # stay under GitHub's 256-jobs-per-matrix limit (see changed-backends.js).
+      matrix-singlearch-1: ${{ steps.set-matrix.outputs['matrix-singlearch-1'] }}
+      merge-matrix-singlearch-1: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-1'] }}
+      has-backends-singlearch-1: ${{ steps.set-matrix.outputs['has-backends-singlearch-1'] }}
+      has-merges-singlearch-1: ${{ steps.set-matrix.outputs['has-merges-singlearch-1'] }}
+      matrix-singlearch-2: ${{ steps.set-matrix.outputs['matrix-singlearch-2'] }}
+      merge-matrix-singlearch-2: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-2'] }}
+      has-backends-singlearch-2: ${{ steps.set-matrix.outputs['has-backends-singlearch-2'] }}
+      has-merges-singlearch-2: ${{ steps.set-matrix.outputs['has-merges-singlearch-2'] }}
+      matrix-singlearch-3: ${{ steps.set-matrix.outputs['matrix-singlearch-3'] }}
+      merge-matrix-singlearch-3: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-3'] }}
+      has-backends-singlearch-3: ${{ steps.set-matrix.outputs['has-backends-singlearch-3'] }}
+      has-merges-singlearch-3: ${{ steps.set-matrix.outputs['has-merges-singlearch-3'] }}
+      matrix-singlearch-4: ${{ steps.set-matrix.outputs['matrix-singlearch-4'] }}
+      merge-matrix-singlearch-4: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-4'] }}
+      has-backends-singlearch-4: ${{ steps.set-matrix.outputs['has-backends-singlearch-4'] }}
+      has-merges-singlearch-4: ${{ steps.set-matrix.outputs['has-merges-singlearch-4'] }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@v7
@@ -109,9 +123,9 @@ jobs:
   # take their full ~6h cold without blocking manifest assembly for the
   # multi-arch backends whose per-arch digests would otherwise sit untagged
   # on quay long enough to be GC'd.
-  backend-jobs-singlearch:
+  backend-jobs-singlearch-1:
     needs: generate-matrix
-    if: needs.generate-matrix.outputs['has-backends-singlearch'] == 'true'
+    if: needs.generate-matrix.outputs['has-backends-singlearch-1'] == 'true'
     uses: ./.github/workflows/backend_build.yml
     with:
       tag-latest: ${{ matrix.tag-latest }}
@@ -138,7 +152,100 @@ jobs:
     strategy:
       fail-fast: false
       max-parallel: 8
-      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch']) }}
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-1']) }}
+
+  backend-jobs-singlearch-2:
+    needs: generate-matrix
+    if: needs.generate-matrix.outputs['has-backends-singlearch-2'] == 'true'
+    uses: ./.github/workflows/backend_build.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+      build-type: ${{ matrix.build-type }}
+      cuda-major-version: ${{ matrix.cuda-major-version }}
+      cuda-minor-version: ${{ matrix.cuda-minor-version }}
+      platforms: ${{ matrix.platforms }}
+      platform-tag: ${{ matrix.platform-tag || '' }}
+      runs-on: ${{ matrix.runs-on }}
+      builder-base-image: ${{ matrix.builder-base-image || '' }}
+      base-image: ${{ matrix.base-image }}
+      backend: ${{ matrix.backend }}
+      dockerfile: ${{ matrix.dockerfile }}
+      skip-drivers: ${{ matrix.skip-drivers }}
+      context: ${{ matrix.context }}
+      ubuntu-version: ${{ matrix.ubuntu-version }}
+      amdgpu-targets: ${{ matrix.amdgpu-targets || 'gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201' }}
+    secrets:
+      dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
+      dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      max-parallel: 8
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-2']) }}
+
+  backend-jobs-singlearch-3:
+    needs: generate-matrix
+    if: needs.generate-matrix.outputs['has-backends-singlearch-3'] == 'true'
+    uses: ./.github/workflows/backend_build.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+      build-type: ${{ matrix.build-type }}
+      cuda-major-version: ${{ matrix.cuda-major-version }}
+      cuda-minor-version: ${{ matrix.cuda-minor-version }}
+      platforms: ${{ matrix.platforms }}
+      platform-tag: ${{ matrix.platform-tag || '' }}
+      runs-on: ${{ matrix.runs-on }}
+      builder-base-image: ${{ matrix.builder-base-image || '' }}
+      base-image: ${{ matrix.base-image }}
+      backend: ${{ matrix.backend }}
+      dockerfile: ${{ matrix.dockerfile }}
+      skip-drivers: ${{ matrix.skip-drivers }}
+      context: ${{ matrix.context }}
+      ubuntu-version: ${{ matrix.ubuntu-version }}
+      amdgpu-targets: ${{ matrix.amdgpu-targets || 'gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201' }}
+    secrets:
+      dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
+      dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      max-parallel: 8
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-3']) }}
+
+  backend-jobs-singlearch-4:
+    needs: generate-matrix
+    if: needs.generate-matrix.outputs['has-backends-singlearch-4'] == 'true'
+    uses: ./.github/workflows/backend_build.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+      build-type: ${{ matrix.build-type }}
+      cuda-major-version: ${{ matrix.cuda-major-version }}
+      cuda-minor-version: ${{ matrix.cuda-minor-version }}
+      platforms: ${{ matrix.platforms }}
+      platform-tag: ${{ matrix.platform-tag || '' }}
+      runs-on: ${{ matrix.runs-on }}
+      builder-base-image: ${{ matrix.builder-base-image || '' }}
+      base-image: ${{ matrix.base-image }}
+      backend: ${{ matrix.backend }}
+      dockerfile: ${{ matrix.dockerfile }}
+      skip-drivers: ${{ matrix.skip-drivers }}
+      context: ${{ matrix.context }}
+      ubuntu-version: ${{ matrix.ubuntu-version }}
+      amdgpu-targets: ${{ matrix.amdgpu-targets || 'gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201' }}
+    secrets:
+      dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
+      dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      max-parallel: 8
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-4']) }}
 
   # Apply tags to per-arch digests via `imagetools create`. Split into two
   # jobs that mirror the build split so each merge waits ONLY on its
@@ -174,10 +281,12 @@ jobs:
       fail-fast: false
       matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-multiarch']) }}
 
-  backend-merge-jobs-singlearch:
-    needs: [generate-matrix, backend-jobs-singlearch]
-    # See note on backend-merge-jobs-multiarch above for !cancelled().
-    if: ${{ !cancelled() && needs.generate-matrix.outputs['has-merges-singlearch'] == 'true' }}
+  # One merge shard per build shard: backend-merge-jobs-singlearch-<n> needs only
+  # backend-jobs-singlearch-<n>, preserving the "merge waits only on its own
+  # build" property while staying under the 256-jobs-per-matrix limit.
+  backend-merge-jobs-singlearch-1:
+    needs: [generate-matrix, backend-jobs-singlearch-1]
+    if: ${{ !cancelled() && needs.generate-matrix.outputs['has-merges-singlearch-1'] == 'true' }}
     uses: ./.github/workflows/backend_merge.yml
     with:
       tag-latest: ${{ matrix.tag-latest }}
@@ -189,7 +298,55 @@ jobs:
       quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
     strategy:
       fail-fast: false
-      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch']) }}
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-1']) }}
+
+  backend-merge-jobs-singlearch-2:
+    needs: [generate-matrix, backend-jobs-singlearch-2]
+    if: ${{ !cancelled() && needs.generate-matrix.outputs['has-merges-singlearch-2'] == 'true' }}
+    uses: ./.github/workflows/backend_merge.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+    secrets:
+      dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
+      dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-2']) }}
+
+  backend-merge-jobs-singlearch-3:
+    needs: [generate-matrix, backend-jobs-singlearch-3]
+    if: ${{ !cancelled() && needs.generate-matrix.outputs['has-merges-singlearch-3'] == 'true' }}
+    uses: ./.github/workflows/backend_merge.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+    secrets:
+      dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
+      dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-3']) }}
+
+  backend-merge-jobs-singlearch-4:
+    needs: [generate-matrix, backend-jobs-singlearch-4]
+    if: ${{ !cancelled() && needs.generate-matrix.outputs['has-merges-singlearch-4'] == 'true' }}
+    uses: ./.github/workflows/backend_merge.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+    secrets:
+      dockerUsername: ${{ secrets.DOCKERHUB_USERNAME }}
+      dockerPassword: ${{ secrets.DOCKERHUB_PASSWORD }}
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-4']) }}
 
   backend-jobs-darwin:
     needs: generate-matrix

.github/workflows/backend_build_darwin.yml

@@ -82,7 +82,7 @@ jobs:
       # as the Linux registry cache.
       - name: Restore Homebrew cache
         id: brew-cache
-        uses: actions/cache/restore@v4
+        uses: actions/cache/restore@v6
         with:
           path: |
             ~/Library/Caches/Homebrew/downloads
@@ -142,7 +142,7 @@ jobs:
 
       - name: Save Homebrew cache
         if: github.event_name != 'pull_request' && steps.brew-cache.outputs.cache-hit != 'true'
-        uses: actions/cache/save@v4
+        uses: actions/cache/save@v6
         with:
           path: |
             ~/Library/Caches/Homebrew/downloads
@@ -178,7 +178,7 @@ jobs:
       - name: Restore ccache
         if: inputs.backend == 'llama-cpp'
         id: ccache-cache
-        uses: actions/cache/restore@v4
+        uses: actions/cache/restore@v6
         with:
           path: ~/Library/Caches/ccache
           key: ccache-llama-${{ runner.arch }}-${{ steps.llama-version.outputs.version }}-${{ github.run_id }}
@@ -211,7 +211,7 @@ jobs:
       - name: Restore Python wheel cache
         if: inputs.lang == 'python'
         id: pyenv-cache
-        uses: actions/cache/restore@v4
+        uses: actions/cache/restore@v6
         with:
           path: |
             ~/Library/Caches/pip
@@ -244,8 +244,19 @@ jobs:
           make protogen-go
           make backends/privacy-filter-darwin
 
+      # audio-cpp is a C++/ggml backend like ds4 and privacy-filter - a single
+      # grpc-server with otool dylib bundling, plus bundled VAD assets - so it
+      # gets its own bespoke darwin script rather than the generic
+      # build-darwin-go-backend path, which would look for a backend/go/audio-cpp
+      # that does not exist. Keep this set in sync with DARWIN_BESPOKE_BUILDERS
+      # in scripts/lib/backend-filter.mjs.
+      - name: Build audio-cpp backend (Darwin Metal)
+        if: inputs.backend == 'audio-cpp'
+        run: |
+          make backends/audio-cpp-darwin
+
       - name: Build ${{ inputs.backend }}-darwin
-        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4' && inputs.backend != 'privacy-filter'
+        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4' && inputs.backend != 'privacy-filter' && inputs.backend != 'audio-cpp'
         run: |
           make protogen-go
           BACKEND=${{ inputs.backend }} BUILD_TYPE=${{ inputs.build-type }} USE_PIP=${{ inputs.use-pip }} make build-darwin-${{ inputs.lang }}-backend
@@ -256,14 +267,14 @@ jobs:
 
       - name: Save ccache
         if: inputs.backend == 'llama-cpp' && github.event_name != 'pull_request'
-        uses: actions/cache/save@v4
+        uses: actions/cache/save@v6
         with:
           path: ~/Library/Caches/ccache
           key: ccache-llama-${{ runner.arch }}-${{ steps.llama-version.outputs.version }}-${{ github.run_id }}
 
       - name: Save Python wheel cache
         if: inputs.lang == 'python' && github.event_name != 'pull_request' && steps.pyenv-cache.outputs.cache-hit != 'true'
-        uses: actions/cache/save@v4
+        uses: actions/cache/save@v6
         with:
           path: |
             ~/Library/Caches/pip

.github/workflows/backend_pr.yml

@@ -11,16 +11,30 @@ jobs:
   generate-matrix:
     runs-on: ubuntu-latest
     outputs:
-      matrix-singlearch: ${{ steps.set-matrix.outputs['matrix-singlearch'] }}
       matrix-multiarch: ${{ steps.set-matrix.outputs['matrix-multiarch'] }}
       matrix-darwin: ${{ steps.set-matrix.outputs['matrix-darwin'] }}
       merge-matrix-multiarch: ${{ steps.set-matrix.outputs['merge-matrix-multiarch'] }}
-      merge-matrix-singlearch: ${{ steps.set-matrix.outputs['merge-matrix-singlearch'] }}
-      has-backends-singlearch: ${{ steps.set-matrix.outputs['has-backends-singlearch'] }}
       has-backends-multiarch: ${{ steps.set-matrix.outputs['has-backends-multiarch'] }}
       has-backends-darwin: ${{ steps.set-matrix.outputs['has-backends-darwin'] }}
       has-merges-multiarch: ${{ steps.set-matrix.outputs['has-merges-multiarch'] }}
-      has-merges-singlearch: ${{ steps.set-matrix.outputs['has-merges-singlearch'] }}
+      # Single-arch backends are sharded across SINGLEARCH_SHARDS matrix jobs to
+      # stay under GitHub's 256-jobs-per-matrix limit (see changed-backends.js).
+      matrix-singlearch-1: ${{ steps.set-matrix.outputs['matrix-singlearch-1'] }}
+      merge-matrix-singlearch-1: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-1'] }}
+      has-backends-singlearch-1: ${{ steps.set-matrix.outputs['has-backends-singlearch-1'] }}
+      has-merges-singlearch-1: ${{ steps.set-matrix.outputs['has-merges-singlearch-1'] }}
+      matrix-singlearch-2: ${{ steps.set-matrix.outputs['matrix-singlearch-2'] }}
+      merge-matrix-singlearch-2: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-2'] }}
+      has-backends-singlearch-2: ${{ steps.set-matrix.outputs['has-backends-singlearch-2'] }}
+      has-merges-singlearch-2: ${{ steps.set-matrix.outputs['has-merges-singlearch-2'] }}
+      matrix-singlearch-3: ${{ steps.set-matrix.outputs['matrix-singlearch-3'] }}
+      merge-matrix-singlearch-3: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-3'] }}
+      has-backends-singlearch-3: ${{ steps.set-matrix.outputs['has-backends-singlearch-3'] }}
+      has-merges-singlearch-3: ${{ steps.set-matrix.outputs['has-merges-singlearch-3'] }}
+      matrix-singlearch-4: ${{ steps.set-matrix.outputs['matrix-singlearch-4'] }}
+      merge-matrix-singlearch-4: ${{ steps.set-matrix.outputs['merge-matrix-singlearch-4'] }}
+      has-backends-singlearch-4: ${{ steps.set-matrix.outputs['has-backends-singlearch-4'] }}
+      has-merges-singlearch-4: ${{ steps.set-matrix.outputs['has-merges-singlearch-4'] }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@v7
@@ -71,10 +85,10 @@ jobs:
       fail-fast: true
       max-parallel: 8
       matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-multiarch']) }}
-  backend-jobs-singlearch:
+  backend-jobs-singlearch-1:
     needs: generate-matrix
+    if: needs.generate-matrix.outputs['has-backends-singlearch-1'] == 'true'
     uses: ./.github/workflows/backend_build.yml
-    if: needs.generate-matrix.outputs['has-backends-singlearch'] == 'true'
     with:
       tag-latest: ${{ matrix.tag-latest }}
       tag-suffix: ${{ matrix.tag-suffix }}
@@ -98,7 +112,94 @@ jobs:
     strategy:
       fail-fast: true
       max-parallel: 8
-      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch']) }}
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-1']) }}
+
+  backend-jobs-singlearch-2:
+    needs: generate-matrix
+    if: needs.generate-matrix.outputs['has-backends-singlearch-2'] == 'true'
+    uses: ./.github/workflows/backend_build.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+      build-type: ${{ matrix.build-type }}
+      cuda-major-version: ${{ matrix.cuda-major-version }}
+      cuda-minor-version: ${{ matrix.cuda-minor-version }}
+      platforms: ${{ matrix.platforms }}
+      platform-tag: ${{ matrix.platform-tag || '' }}
+      runs-on: ${{ matrix.runs-on }}
+      builder-base-image: ${{ matrix.builder-base-image || '' }}
+      base-image: ${{ matrix.base-image }}
+      backend: ${{ matrix.backend }}
+      dockerfile: ${{ matrix.dockerfile }}
+      skip-drivers: ${{ matrix.skip-drivers }}
+      context: ${{ matrix.context }}
+      ubuntu-version: ${{ matrix.ubuntu-version }}
+      amdgpu-targets: ${{ matrix.amdgpu-targets || 'gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201' }}
+    secrets:
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: true
+      max-parallel: 8
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-2']) }}
+
+  backend-jobs-singlearch-3:
+    needs: generate-matrix
+    if: needs.generate-matrix.outputs['has-backends-singlearch-3'] == 'true'
+    uses: ./.github/workflows/backend_build.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+      build-type: ${{ matrix.build-type }}
+      cuda-major-version: ${{ matrix.cuda-major-version }}
+      cuda-minor-version: ${{ matrix.cuda-minor-version }}
+      platforms: ${{ matrix.platforms }}
+      platform-tag: ${{ matrix.platform-tag || '' }}
+      runs-on: ${{ matrix.runs-on }}
+      builder-base-image: ${{ matrix.builder-base-image || '' }}
+      base-image: ${{ matrix.base-image }}
+      backend: ${{ matrix.backend }}
+      dockerfile: ${{ matrix.dockerfile }}
+      skip-drivers: ${{ matrix.skip-drivers }}
+      context: ${{ matrix.context }}
+      ubuntu-version: ${{ matrix.ubuntu-version }}
+      amdgpu-targets: ${{ matrix.amdgpu-targets || 'gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201' }}
+    secrets:
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: true
+      max-parallel: 8
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-3']) }}
+
+  backend-jobs-singlearch-4:
+    needs: generate-matrix
+    if: needs.generate-matrix.outputs['has-backends-singlearch-4'] == 'true'
+    uses: ./.github/workflows/backend_build.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+      build-type: ${{ matrix.build-type }}
+      cuda-major-version: ${{ matrix.cuda-major-version }}
+      cuda-minor-version: ${{ matrix.cuda-minor-version }}
+      platforms: ${{ matrix.platforms }}
+      platform-tag: ${{ matrix.platform-tag || '' }}
+      runs-on: ${{ matrix.runs-on }}
+      builder-base-image: ${{ matrix.builder-base-image || '' }}
+      base-image: ${{ matrix.base-image }}
+      backend: ${{ matrix.backend }}
+      dockerfile: ${{ matrix.dockerfile }}
+      skip-drivers: ${{ matrix.skip-drivers }}
+      context: ${{ matrix.context }}
+      ubuntu-version: ${{ matrix.ubuntu-version }}
+      amdgpu-targets: ${{ matrix.amdgpu-targets || 'gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201' }}
+    secrets:
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: true
+      max-parallel: 8
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['matrix-singlearch-4']) }}
   backend-merge-jobs-multiarch:
     needs: [generate-matrix, backend-jobs-multiarch]
     # backend_merge.yml's push-side steps are all gated on
@@ -118,9 +219,9 @@ jobs:
       fail-fast: false
       matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-multiarch']) }}
 
-  backend-merge-jobs-singlearch:
-    needs: [generate-matrix, backend-jobs-singlearch]
-    if: ${{ !cancelled() && github.event_name != 'pull_request' && needs.generate-matrix.outputs['has-merges-singlearch'] == 'true' }}
+  backend-merge-jobs-singlearch-1:
+    needs: [generate-matrix, backend-jobs-singlearch-1]
+    if: ${{ !cancelled() && github.event_name != 'pull_request' && needs.generate-matrix.outputs['has-merges-singlearch-1'] == 'true' }}
     uses: ./.github/workflows/backend_merge.yml
     with:
       tag-latest: ${{ matrix.tag-latest }}
@@ -130,7 +231,49 @@ jobs:
       quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
     strategy:
       fail-fast: false
-      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch']) }}
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-1']) }}
+
+  backend-merge-jobs-singlearch-2:
+    needs: [generate-matrix, backend-jobs-singlearch-2]
+    if: ${{ !cancelled() && github.event_name != 'pull_request' && needs.generate-matrix.outputs['has-merges-singlearch-2'] == 'true' }}
+    uses: ./.github/workflows/backend_merge.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+    secrets:
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-2']) }}
+
+  backend-merge-jobs-singlearch-3:
+    needs: [generate-matrix, backend-jobs-singlearch-3]
+    if: ${{ !cancelled() && github.event_name != 'pull_request' && needs.generate-matrix.outputs['has-merges-singlearch-3'] == 'true' }}
+    uses: ./.github/workflows/backend_merge.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+    secrets:
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-3']) }}
+
+  backend-merge-jobs-singlearch-4:
+    needs: [generate-matrix, backend-jobs-singlearch-4]
+    if: ${{ !cancelled() && github.event_name != 'pull_request' && needs.generate-matrix.outputs['has-merges-singlearch-4'] == 'true' }}
+    uses: ./.github/workflows/backend_merge.yml
+    with:
+      tag-latest: ${{ matrix.tag-latest }}
+      tag-suffix: ${{ matrix.tag-suffix }}
+    secrets:
+      quayUsername: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+      quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.generate-matrix.outputs['merge-matrix-singlearch-4']) }}
   backend-jobs-darwin:
     needs: generate-matrix
     uses: ./.github/workflows/backend_build_darwin.yml

.github/workflows/build-test.yaml

@@ -5,6 +5,32 @@ on:
     branches:
       - master
   pull_request:
+    # GoReleaser and the darwin launcher take no gallery, docs or markdown
+    # input, so a diff confined to these paths cannot change either binary.
+    # The darwin job matters here: macOS is the scarcest runner class.
+    #
+    # backend/{cpp,go,python}/**: GoReleaser builds ./cmd/local-ai and the
+    # launcher builds ./cmd/launcher; neither compiles a backend. The
+    # before-hooks still matter, but their input is backend/backend.proto
+    # (protogen-go) and go.mod/go.sum (go mod tidy), none of which live under
+    # these prefixes, so a change to any of those still triggers a full run.
+    # See .agents/ci-caching.md.
+    paths-ignore:
+      - 'gallery/**'
+      - 'docs/**'
+      - 'examples/**'
+      - '**/*.md'
+      - 'backend/cpp/**'
+      - 'backend/go/**'
+      - 'backend/python/**'
+
+# Supersede an in-flight run when a PR gets a new push. Keyed on the PR number
+# so every push to the same PR shares a group; on a master push the key falls
+# back to github.sha (unique per commit) and cancel-in-progress is false, so
+# master runs never cancel each other -- each commit is built on its own.
+concurrency:
+  group: ci-build-test-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
   build-test:
@@ -18,9 +44,15 @@ jobs:
         uses: actions/setup-go@v5
         with:
           go-version: 1.25
+      # A PR builds only the host target. The three-platform cross-compile
+      # (linux/amd64, linux/arm64, darwin/arm64) is the bulk of this job's
+      # ~6.6min median and no PR consumes the resulting binaries. The
+      # before-hooks (protogen-go, react-ui, go mod tidy) run either way, so the
+      # "is the release build broken" signal is unchanged. master and tags still
+      # build everything.
       - name: Run GoReleaser
         run: |
-          make dev-dist
+          make ${{ github.event_name == 'pull_request' && 'dev-dist-single' || 'dev-dist' }}
   launcher-build-darwin:
     runs-on: macos-latest
     steps:

.github/workflows/bump_deps.yaml

@@ -22,10 +22,22 @@ jobs:
             variable: "TURBOQUANT_VERSION"
             branch: "feature/turboquant-kv-cache"
             file: "backend/cpp/turboquant/Makefile"
+          - repository: "PrismML-Eng/llama.cpp"
+            variable: "BONSAI_VERSION"
+            branch: "prism"
+            file: "backend/cpp/bonsai/Makefile"
           - repository: "antirez/ds4"
             variable: "DS4_VERSION"
             branch: "main"
             file: "backend/cpp/ds4/Makefile"
+          - repository: "0xShug0/audio.cpp"
+            variable: "AUDIO_CPP_VERSION"
+            branch: "main"
+            file: "backend/cpp/audio-cpp/Makefile"
+          - repository: "meituan-longcat/LongCat-Video"
+            variable: "LONGCAT_VIDEO_VERSION"
+            branch: "main"
+            file: "backend/python/longcat-video/Makefile"
           - repository: "localai-org/privacy-filter.cpp"
             variable: "PRIVACY_FILTER_VERSION"
             branch: "master"
@@ -42,10 +54,26 @@ jobs:
             variable: "PARAKEET_VERSION"
             branch: "master"
             file: "backend/go/parakeet-cpp/Makefile"
-          - repository: "mudler/ced.cpp"
-            variable: "CED_VERSION"
+          - repository: "mudler/vllm.cpp"
+            variable: "VLLM_CPP_VERSION"
+            branch: "main"
+            file: "backend/go/vllm-cpp/Makefile"
+          - repository: "localai-org/moss-transcribe.cpp"
+            variable: "MOSS_VERSION"
             branch: "master"
+            file: "backend/go/moss-transcribe-cpp/Makefile"
+          - repository: "localai-org/ced.cpp"
+            variable: "CED_VERSION"
+            branch: "main"
             file: "backend/go/ced/Makefile"
+          - repository: "localai-org/voice-detect.cpp"
+            variable: "VOICEDETECT_VERSION"
+            branch: "master"
+            file: "backend/go/voice-detect/Makefile"
+          - repository: "mudler/face-detect.cpp"
+            variable: "FACEDETECT_VERSION"
+            branch: "master"
+            file: "backend/go/face-detect/Makefile"
           - repository: "mudler/depth-anything.cpp"
             variable: "DEPTHANYTHING_VERSION"
             branch: "master"
@@ -54,6 +82,10 @@ jobs:
             variable: "STABLEDIFFUSION_GGML_VERSION"
             branch: "master"
             file: "backend/go/stablediffusion-ggml/Makefile"
+          - repository: "localai-org/trellis2cpp"
+            variable: "TRELLIS2CPP_VERSION"
+            branch: "pbr-textures"
+            file: "backend/go/trellis2cpp/Makefile"
           - repository: "mudler/go-piper"
             variable: "PIPER_VERSION"
             branch: "master"
@@ -70,7 +102,7 @@ jobs:
             variable: "SAM3_VERSION"
             branch: "main"
             file: "backend/go/sam3-cpp/Makefile"
-          - repository: "mudler/rf-detr.cpp"
+          - repository: "localai-org/rf-detr.cpp"
             variable: "RFDETR_VERSION"
             branch: "main"
             file: "backend/go/rfdetr-cpp/Makefile"
@@ -78,10 +110,14 @@ jobs:
             variable: "LOCATEANYTHING_VERSION"
             branch: "master"
             file: "backend/go/locate-anything-cpp/Makefile"
-          - repository: "ServeurpersoCom/qwentts.cpp"
-            variable: "QWEN3TTS_CPP_VERSION"
-            branch: "master"
-            file: "backend/go/qwen3-tts-cpp/Makefile"
+          # qwentts.cpp is held, not tracked: upstream master hangs in synthesis
+          # (see the comment on QWEN3TTS_CPP_VERSION in the backend Makefile).
+          # Leaving it here would re-bump the pin back onto the hang every night.
+          # Restore this entry once the upstream fix lands.
+          # - repository: "ServeurpersoCom/qwentts.cpp"
+          #   variable: "QWEN3TTS_CPP_VERSION"
+          #   branch: "master"
+          #   file: "backend/go/qwen3-tts-cpp/Makefile"
           - repository: "ServeurpersoCom/omnivoice.cpp"
             variable: "OMNIVOICE_VERSION"
             branch: "master"
@@ -90,11 +126,21 @@ jobs:
             variable: "VIBEVOICE_CPP_VERSION"
             branch: "master"
             file: "backend/go/vibevoice-cpp/Makefile"
+          - repository: "mudler/magpie-tts.cpp"
+            variable: "MAGPIETTS_CPP_VERSION"
+            branch: "main"
+            file: "backend/go/magpie-tts-cpp/Makefile"
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v7
       - name: Bump dependencies 🔧
         id: bump
+        env:
+          # This job fans out to ~25 parallel matrix entries, all querying
+          # api.github.com from runner IPs that share the 60/hour anonymous
+          # rate limit. Authenticating raises it to 1000/hour, which is what
+          # kept a random handful of these red every night.
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_deps.sh ${{ matrix.repository }} ${{ matrix.branch }} ${{ matrix.variable }} ${{ matrix.file }}
           {
@@ -131,6 +177,8 @@ jobs:
       - uses: actions/checkout@v7
       - name: Bump vLLM cu130 wheel pin 🔧
         id: bump
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_vllm_wheel.sh vllm-project/vllm backend/python/vllm/requirements-cublas13-after.txt VLLM_VERSION
           {
@@ -167,6 +215,8 @@ jobs:
       - uses: actions/checkout@v7
       - name: Bump vllm-metal pin 🔧
         id: bump
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_vllm_metal.sh vllm-project/vllm-metal backend/python/vllm/install.sh VLLM_METAL_VERSION
           {

.github/workflows/bump_docs.yaml

@@ -15,6 +15,10 @@ jobs:
     steps:
       - uses: actions/checkout@v7
       - name: Bump dependencies 🔧
+        env:
+          # Authenticated API calls get 1000 req/hour instead of the 60/hour
+          # anonymous cap that is shared across every job on the runner's IP.
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_docs.sh ${{ matrix.repository }}
       - name: Create Pull Request

.github/workflows/ci-tools-tests.yaml

@@ -0,0 +1,39 @@
+---
+# The packages under .github/ci/ are invisible to `go list ./...`, so neither
+# `make lint` nor the repository test run ever touches them. Their specs are
+# dead weight until a workflow names each package explicitly.
+name: 'CI tool tests'
+on:
+  pull_request:
+    paths:
+      - '.github/ci/**'
+      - '.github/workflows/ci-tools-tests.yaml'
+  push:
+    branches:
+      - master
+    paths:
+      - '.github/ci/**'
+jobs:
+  ci-tools:
+    name: 'Test the .github/ci generators'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-go@v5
+        with:
+          go-version-file: go.mod
+          cache: false
+
+      # The discovery heuristics are the risky part of these tools. A regression
+      # produces confident, wrong gallery entries, which is worse than no tool.
+      - name: 'Test the APEX entry generator'
+        run: go test ./.github/ci/apexentries/
+
+      - name: 'Test the variant proposer'
+        run: go test ./.github/ci/variantproposals/
+
+      # Shared by both generators above. Its behaviour is exercised through their
+      # specs; this step exists so a break in the shared package fails under its
+      # own name rather than as a puzzling failure in whichever caller ran first.
+      - name: 'Test the shared gallery editor'
+        run: go test ./.github/ci/galleryedit/

.github/workflows/gallery_variant_proposals.yaml

@@ -0,0 +1,54 @@
+name: Propose gallery variant groupings
+on:
+  schedule:
+    - cron: 0 4 * * 1
+  workflow_dispatch:
+jobs:
+  variant_proposals:
+    if: github.repository == 'mudler/LocalAI'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-go@v5
+        with:
+          go-version-file: go.mod
+          cache: false
+
+      # The heuristics are the risky part of this job. A regression in them
+      # produces confident, wrong proposals, which is worse than no job at all.
+      - name: Test the proposer
+        run: go test ./.github/ci/variantproposals/
+
+      - name: Propose groupings 🔧
+        id: propose
+        run: |
+          rm -f /tmp/variant-proposals-body.md
+          go run ./.github/ci/variantproposals \
+            -index gallery/index.yaml \
+            -ledger gallery/variant-exclusions.yaml \
+            -body-out /tmp/variant-proposals-body.md \
+            -apply
+          if [ -s /tmp/variant-proposals-body.md ]; then
+            echo "have_proposals=true" >> "$GITHUB_OUTPUT"
+            {
+              echo 'body<<VARIANT_PROPOSAL_BODY_EOF'
+              cat /tmp/variant-proposals-body.md
+              echo VARIANT_PROPOSAL_BODY_EOF
+            } >> "$GITHUB_OUTPUT"
+          else
+            echo "have_proposals=false" >> "$GITHUB_OUTPUT"
+          fi
+
+      # No body file means the proposer found nothing. Opening an empty pull
+      # request every run is how a proposal job gets muted by its reviewers.
+      - name: Create Pull Request
+        if: steps.propose.outputs.have_proposals == 'true'
+        uses: peter-evans/create-pull-request@v8
+        with:
+          token: ${{ secrets.UPDATE_BOT_TOKEN }}
+          push-to-fork: ci-forks/LocalAI
+          commit-message: 'chore(model-gallery): propose variant groupings'
+          title: 'chore(model-gallery): propose variant groupings for review'
+          branch: "propose/variant-groupings"
+          body: ${{ steps.propose.outputs.body }}
+          signoff: true

.github/workflows/gh-pages.yml

@@ -1,4 +1,4 @@
-name: Deploy docs to GitHub Pages
+name: Deploy site to GitHub Pages
 
 on:
   push:
@@ -6,9 +6,11 @@ on:
       - master
     paths:
       - 'docs/**'
+      - 'website/**'
       - 'gallery/**'
       - 'images/**'
       - '.github/ci/modelslist.go'
+      - '.github/ci/gen-redirects.sh'
       - '.github/workflows/gh-pages.yml'
   workflow_dispatch:
 
@@ -23,7 +25,20 @@ concurrency:
 
 jobs:
   build:
-    runs-on: ubuntu-latest
+    # Self-hosted. This workflow is push-to-master + workflow_dispatch only, so
+    # it never executes pull-request code and a fork cannot reach the runner
+    # with untrusted changes. The repository guard keeps forks (whose own master
+    # pushes would otherwise queue forever against a label they do not have) on
+    # the hosted pool.
+    #
+    # Why: the GitHub-hosted pool is shared account-wide and has repeatedly
+    # starved (2026-07-31: 35 consecutive minutes at zero scheduled jobs, while
+    # arc-runner-set kept completing work throughout). Publishing the site is
+    # small, frequent, and must not sit behind a saturated hosted queue.
+    #
+    # Needs only git, tar and curl on the runner: setup-go and actions-hugo
+    # fetch their own toolchains, and no step uses sudo, apt, make or unzip.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     env:
       HUGO_VERSION: "0.146.3"
     steps:
@@ -49,25 +64,46 @@ jobs:
         id: pages
         uses: actions/configure-pages@v6
 
+      # The gallery page is generated from the model index and shipped as a
+      # static asset of the docs site, so it has to exist before Hugo runs.
       - name: Generate gallery
         run: go run ./.github/ci/modelslist.go ./gallery/index.yaml > docs/static/gallery.html
 
-      - name: Build site
+      # Two Hugo sites, one Pages artifact: the main site owns the root,
+      # the docs site is nested under /docs/.
+      - name: Build the main site
+        working-directory: website
+        run: hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+
+      - name: Build documentation site
         working-directory: docs
         run: |
           mkdir -p layouts/_default
-          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/docs/"
+
+      - name: Merge documentation into the main site
+        run: |
+          mkdir -p website/public/docs
+          cp -R docs/public/. website/public/docs/
+
+      # Keeps the pre-split URLs alive; see the script header.
+      - name: Generate legacy URL redirects
+        run: .github/ci/gen-redirects.sh website/public "${{ steps.pages.outputs.base_url }}/"
 
       - name: Upload artifact
         uses: actions/upload-pages-artifact@v5
         with:
-          path: docs/public
+          path: website/public
 
   deploy:
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}
-    runs-on: ubuntu-latest
+    # Same routing as build: a hosted slot for a ~10s deploy is exactly the kind
+    # of job that should not block on a starved pool. deploy-pages authenticates
+    # with the job's OIDC token (id-token: write above), which self-hosted
+    # runners issue the same way hosted ones do.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     needs: build
     steps:
       - name: Deploy to GitHub Pages

.github/workflows/image-pr.yml

@@ -3,7 +3,28 @@
   
   on:
     pull_request:
-  
+      # None of these seven image builds can observe a diff confined to these
+      # paths. Gallery metadata is parsed at runtime and never copied into an
+      # image; docs and markdown never enter one at all. Gallery content is
+      # still checked by yaml-check.yml and by
+      # core/gallery/variants_lint_test.go under 'tests'.
+      #
+      # backend/{cpp,go,python}/**: this workflow builds the core image, whose
+      # only compiled output is `make build` -> `go build ./cmd/local-ai`. The
+      # per-backend trees are copied into the builder but nothing in them reaches
+      # the binary or the final stage. backend/backend.proto is deliberately not
+      # listed: it feeds protogen-go and so does change the binary, and it does
+      # not live under any of these prefixes, so it still triggers a full run.
+      # See .agents/ci-caching.md.
+      paths-ignore:
+        - 'gallery/**'
+        - 'docs/**'
+        - 'examples/**'
+        - '**/*.md'
+        - 'backend/cpp/**'
+        - 'backend/go/**'
+        - 'backend/python/**'
+
   concurrency:
     group: ci-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
     cancel-in-progress: ${{ github.event_name == 'pull_request' }}
@@ -52,7 +73,7 @@
               tag-latest: 'false'
               tag-suffix: '-gpu-nvidia-cuda-13'
               runs-on: 'ubuntu-latest'
-              base-image: "ubuntu:22.04"
+              base-image: "ubuntu:24.04"
               makeflags: "--jobs=3 --output-sync=target"
               ubuntu-version: '2404'
             - build-type: 'hipblas'

.github/workflows/image.yml

@@ -13,8 +13,53 @@
     cancel-in-progress: ${{ github.event_name == 'pull_request' }}
   
   jobs:
-    hipblas-jobs:
+    # Decide once whether this push can change any image. Gallery metadata is
+    # fetched at runtime and never baked into an image, and docs/markdown never
+    # enter one, so a push confined to those paths produces byte-identical
+    # images. On 2026-07-30, 12 of the 23 queued runs of this workflow were
+    # commits like "add 1 new model to gallery" or a docs fix, each rebuilding
+    # all 18 images.
+    #
+    # A job-level gate rather than `paths-ignore` on the trigger: paths-ignore
+    # would also apply to tag pushes, and a tag created on an existing commit
+    # carries an empty commits list, which would silently skip the release image
+    # build. Tags short-circuit to "build" below, as does a push whose base
+    # commit cannot be resolved -- the same run-everything posture the backend
+    # matrix filter takes for a truncated diff.
+    changes:
       if: github.repository == 'mudler/LocalAI'
+      runs-on: ubuntu-latest
+      outputs:
+        build: ${{ steps.decide.outputs.build }}
+      steps:
+        - uses: actions/checkout@v7
+          with:
+            fetch-depth: 0
+        - id: decide
+          env:
+            BEFORE: ${{ github.event.before }}
+            AFTER: ${{ github.sha }}
+          run: |
+            set -euo pipefail
+            emit() { echo "$2"; echo "build=$1" >> "$GITHUB_OUTPUT"; exit 0; }
+            case "${GITHUB_REF}" in
+              refs/tags/*) emit true "tag push: building every image" ;;
+            esac
+            if [ -z "${BEFORE:-}" ] || [ "${BEFORE}" = "0000000000000000000000000000000000000000" ] \
+               || ! git cat-file -e "${BEFORE}^{commit}" 2>/dev/null; then
+              emit true "no resolvable base commit: building every image"
+            fi
+            files="$(git diff --name-only "${BEFORE}" "${AFTER}")"
+            echo "changed files:"; echo "${files:-<none>}"
+            [ -z "${files}" ] && emit true "empty diff: building every image"
+            if echo "${files}" | grep -qvE '^(gallery/|docs/|examples/)|\.md$'; then
+              emit true "push touches image-visible content: building"
+            fi
+            emit false "only gallery/docs/markdown changed: images identical, skipping"
+
+    hipblas-jobs:
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}
@@ -47,7 +92,8 @@
               ubuntu-codename: 'noble'
 
     core-image-build:
-      if: github.repository == 'mudler/LocalAI'
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}
@@ -113,7 +159,7 @@
               tag-latest: 'auto'
               tag-suffix: '-gpu-nvidia-cuda-13'
               runs-on: 'ubuntu-latest'
-              base-image: "ubuntu:22.04"
+              base-image: "ubuntu:24.04"
               skip-drivers: 'false'
               makeflags: "--jobs=4 --output-sync=target"
               ubuntu-version: '2404'
@@ -155,8 +201,8 @@
       # merge whenever any matrix cell of the parent build fails or is
       # cancelled. Same fix as backend.yml's merge jobs — we still want to
       # publish the manifest list for tag-suffixes whose legs all succeeded.
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -168,8 +214,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-vulkan-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -187,8 +233,8 @@
     # Each merge job needs only its parent build matrix and is filtered by
     # tag-suffix in image_merge.yml's artifact-download pattern.
     gpu-nvidia-cuda-12-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -200,8 +246,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-nvidia-cuda-13-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -213,8 +259,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-intel-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -226,8 +272,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-hipblas-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: hipblas-jobs
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, hipblas-jobs]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -239,8 +285,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     nvidia-l4t-arm64-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: gh-runner
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, gh-runner]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -252,8 +298,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     nvidia-l4t-arm64-cuda-13-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: gh-runner
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, gh-runner]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -265,7 +311,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gh-runner:
-      if: github.repository == 'mudler/LocalAI'
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}

.github/workflows/lint.yml

@@ -8,6 +8,9 @@ on:
       - 'examples/**'
       - 'README.md'
       - '**/*.md'
+      # golangci-lint runs new-from-merge-base, so a diff with no touched Go
+      # lines can only ever be a no-op. See .agents/ci-caching.md.
+      - 'gallery/**'
   push:
     branches:
       - master
@@ -18,8 +21,41 @@ concurrency:
 
 jobs:
   golangci-lint:
+    # Self-hosted for PUSH only, and only in the canonical repo.
+    #
+    # This workflow also runs on pull_request, which for a fork PR means
+    # executing untrusted contributor code. That must never land on a
+    # self-hosted runner, so anything that is not a push to mudler/LocalAI stays
+    # on the ephemeral hosted pool. Pushes to master are trusted code that has
+    # already been reviewed and merged.
+    #
+    # Why at all: the hosted pool is shared account-wide and starved for 35
+    # straight minutes on 2026-07-31 while arc-runner-set kept completing jobs.
+    # Lint is small and runs on every commit, so it is a good candidate to move
+    # off the contended pool.
+    # REVERTED to hosted: the arc-runner-set image has git, curl, unzip, tar,
+    # ldd and python3, but NOT make (nor gcc). Measured on run 30637392862,
+    # where the preflight below named both. Re-route here once the runner image
+    # ships a C toolchain and make; the preflight stays so the next attempt
+    # fails by name in one second instead of opaquely mid-build.
     runs-on: ubuntu-latest
     steps:
+      - name: Preflight - required host tools
+        # The hosted images ship these; a self-hosted container image may not.
+        # Check up front so a missing tool reports itself by name instead of
+        # surfacing as an opaque failure inside `make protogen-go` (which needs
+        # curl + unzip for protoc) or `make lint`.
+        run: |
+          missing=""
+          for t in git curl unzip make tar; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
       - uses: actions/checkout@v7
         with:
           # Full history so golangci-lint's new-from-merge-base can reach
@@ -46,3 +82,51 @@ jobs:
           touch core/http/react-ui/dist/index.html
       - name: lint
         run: make lint
+
+  build-scripts:
+    # The image packaging scripts encode invariants that only surface inside a
+    # container build (a missing transitive dep, a partial cuDNN family). Their
+    # shell tests need nothing but bash + gcc + ldd, so run them on every PR
+    # rather than waiting on a multi-GB cross-arch backend image build.
+    #
+    # Push-only self-hosted routing, same fork-safety reasoning as
+    # golangci-lint above.
+    # REVERTED to hosted: the arc-runner-set image has git, curl, unzip, tar,
+    # ldd and python3, but NOT make (nor gcc). Measured on run 30637392862,
+    # where the preflight below named both. Re-route here once the runner image
+    # ships a C toolchain and make; the preflight stays so the next attempt
+    # fails by name in one second instead of opaquely mid-build.
+    runs-on: ubuntu-latest
+    steps:
+      - name: Preflight - required host tools
+        # This job additionally needs a C toolchain: the packaging-script tests
+        # compile a throwaway binary and inspect it with ldd.
+        run: |
+          missing=""
+          for t in git make gcc ldd python3; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
+      - uses: actions/checkout@v7
+      - name: run packaging script tests
+        run: make test-build-scripts
+
+      # The backend matrix path filter fails silently: a miss emits an empty
+      # matrix, every job goes green, and the change reaches no image (#10946).
+      # Its tests need only node, so they ride along with this job.
+      - uses: actions/setup-node@v7
+        with:
+          node-version: '20'
+      - name: run CI script tests
+        run: make test-ci-scripts
+
+      # The shared python backend helpers (Options[] parsing, engine-arg
+      # mapping, model reference resolution) are stdlib-only, so their tests
+      # ride along here instead of waiting on a multi-GB backend image build.
+      - name: run shared python backend helper tests
+        run: make test-python-helpers

.github/workflows/realtime-conformance.yml

@@ -0,0 +1,69 @@
+---
+name: 'realtime-conformance'
+
+# Verifies the realtime state-machine implementations conform to their formal
+# designs (docs/design/realtime-state-machines.md, formal-verification/). BOTH
+# layers are enforced and the gate is fail-closed: the Go conformance layer
+# (respcoord + turncoord transition/rapid tests under -race) AND the FizzBee model check of
+# the authoritative specs. FizzBee is pinned + checksum-verified
+# (formal-verification/fizzbee.sha256), so a failed install fails the job rather
+# than silently skipping verification.
+
+on:
+  pull_request:
+    paths:
+      - 'core/http/endpoints/openai/coordinator/**'
+      - 'core/http/endpoints/openai/respcoord/**'
+      - 'core/http/endpoints/openai/turncoord/**'
+      - 'core/http/endpoints/openai/conncoord/**'
+      - 'core/http/endpoints/openai/compactcoord/**'
+      - 'core/http/endpoints/openai/ttscoord/**'
+      - 'formal-verification/**'
+      - 'scripts/realtime-conformance.sh'
+      - 'scripts/install-fizzbee.sh'
+      - '.github/workflows/realtime-conformance.yml'
+  push:
+    branches:
+      - master
+    paths:
+      - 'core/http/endpoints/openai/coordinator/**'
+      - 'core/http/endpoints/openai/respcoord/**'
+      - 'core/http/endpoints/openai/turncoord/**'
+      - 'core/http/endpoints/openai/conncoord/**'
+      - 'core/http/endpoints/openai/compactcoord/**'
+      - 'core/http/endpoints/openai/ttscoord/**'
+      - 'formal-verification/**'
+      - 'scripts/realtime-conformance.sh'
+
+concurrency:
+  group: realtime-conformance-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  conformance:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        go-version: ['1.26.x']
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+      - name: Setup Go ${{ matrix.go-version }}
+        uses: actions/setup-go@v5
+        with:
+          go-version: ${{ matrix.go-version }}
+          cache: false
+      - name: Cache FizzBee
+        uses: actions/cache@v6
+        with:
+          path: .tools/fizzbee
+          key: fizzbee-v0.5.2-${{ runner.os }}-${{ hashFiles('formal-verification/fizzbee.sha256') }}
+      - name: Install FizzBee (pinned, checksum-verified)
+        # No `|| true`: a failed/forged download must fail the job, not silently
+        # drop the design verification. install-fizzbee.sh is a no-op if the
+        # cached binary is already present and valid.
+        run: ./scripts/install-fizzbee.sh
+      - name: Run conformance gate (fail-closed)
+        # No skip env: both the Go conformance and the FizzBee model check are
+        # required. The gate auto-detects .tools/fizzbee/fizz.
+        run: make test-realtime-conformance

.github/workflows/refresh-site-counters.yml

@@ -0,0 +1,44 @@
+name: Refresh site counters
+
+# The landing page shows a star count, a contributor count and a release
+# count. They were typed in by hand, so they drifted the moment somebody
+# forgot. This pulls the real numbers once a week and commits them only when
+# they have actually moved, which in turn triggers the usual Pages deploy.
+
+on:
+  schedule:
+    # Mondays, 06:17 UTC. Off the hour on purpose, since the scheduler queues
+    # everything that asks for :00 and drops what it cannot run.
+    - cron: '17 6 * * 1'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  group: refresh-site-counters
+  cancel-in-progress: false
+
+jobs:
+  refresh:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Read the counts off the GitHub API
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: ./.github/ci/refresh-site-counters.sh
+
+      - name: Commit only if something moved
+        run: |
+          if git diff --quiet -- website/data/stats.yaml; then
+            echo "counters unchanged, nothing to commit"
+            exit 0
+          fi
+          git diff --unified=0 -- website/data/stats.yaml
+          git config user.name  "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git add website/data/stats.yaml
+          git commit -m "chore(website): refresh the counters"
+          git push

.github/workflows/release.yaml

@@ -24,6 +24,11 @@ jobs:
           args: release --clean
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          MACOS_SIGN_P12: ${{ secrets.MACOS_CERTIFICATE }}
+          MACOS_SIGN_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PWD }}
+          MACOS_NOTARY_KEY: ${{ secrets.MACOS_NOTARY_KEY }}
+          MACOS_NOTARY_KEY_ID: ${{ secrets.MACOS_NOTARY_KEY_ID }}
+          MACOS_NOTARY_ISSUER_ID: ${{ secrets.MACOS_NOTARY_ISSUER_ID }}
   launcher-build-darwin:
     runs-on: macos-latest
     steps:
@@ -35,9 +40,19 @@ jobs:
         uses: actions/setup-go@v5
         with:
           go-version: 1.23
-      - name: Build launcher for macOS ARM64
-        run: |
-          make build-launcher-darwin
+      - name: Import signing certificate
+        env:
+          MACOS_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE }}
+          MACOS_CERTIFICATE_PWD: ${{ secrets.MACOS_CERTIFICATE_PWD }}
+          MACOS_CI_KEYCHAIN_PWD: ${{ secrets.MACOS_CI_KEYCHAIN_PWD }}
+        run: bash contrib/macos/sign-and-notarize.sh import-cert
+      - name: Build, sign and notarize the DMG
+        env:
+          MACOS_SIGN_IDENTITY: ${{ secrets.MACOS_SIGN_IDENTITY }}
+          MACOS_NOTARY_KEY: ${{ secrets.MACOS_NOTARY_KEY }}
+          MACOS_NOTARY_KEY_ID: ${{ secrets.MACOS_NOTARY_KEY_ID }}
+          MACOS_NOTARY_ISSUER_ID: ${{ secrets.MACOS_NOTARY_ISSUER_ID }}
+        run: make release-launcher-darwin
       - name: Upload DMG to Release
         uses: softprops/action-gh-release@v3
         with:

.github/workflows/secscan.yaml

@@ -7,6 +7,19 @@ on:
   schedule:
   - cron: '0 0 * * 0'
 
+# `push:` is deliberately unfiltered, so this fires on every push to every
+# branch and there is no pull_request event to key on -- the usual
+# `github.event.pull_request.number || github.sha` idiom used elsewhere would
+# key on the unique-per-commit sha and dedup nothing. Group on the ref instead
+# so successive pushes to the same feature branch supersede one another.
+#
+# Cancelling is safe here: the only output is a SARIF upload, and code scanning
+# tracks the latest result per ref, so a superseded scan has nothing to lose.
+# master is excluded anyway -- every commit on master gets its own scan.
+concurrency:
+  group: ci-secscan-${{ github.ref }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.ref != 'refs/heads/master' }}
+
 jobs:
   tests:
     runs-on: ubuntu-latest
@@ -15,9 +28,9 @@ jobs:
     steps:
       - name: Checkout Source
         uses: actions/checkout@v7
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
       - name: Run Gosec Security Scanner
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
         uses: securego/gosec@v2.27.1
         with:
           # we let the report trigger content trigger a failure using the GitHub Security features.
@@ -26,7 +39,7 @@ jobs:
           # noise, G104 unhandled errors) are inherent to that upstream code, not ours to rewrite.
           args: '-no-fail -exclude-dir=backend/go/supertonic -fmt sarif -out results.sarif ./...'
       - name: Upload SARIF file
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
         uses: github/codeql-action/upload-sarif@v4
         with:
           # Path to SARIF file relative to the root of the repository

.github/workflows/stalebot.yml

@@ -11,7 +11,7 @@ jobs:
     if: github.repository == 'mudler/LocalAI'
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v9
+      - uses: actions/stale@1e223db275d687790206a7acac4d1a11bd6fe629 # v9
         with:
           stale-issue-message: 'This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.'
           stale-pr-message: 'This PR is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 10 days.'

.github/workflows/test-extra.yml

@@ -37,6 +37,8 @@ jobs:
       sglang: ${{ steps.detect.outputs.sglang }}
       acestep-cpp: ${{ steps.detect.outputs.acestep-cpp }}
       qwen3-tts-cpp: ${{ steps.detect.outputs.qwen3-tts-cpp }}
+      magpie-tts-cpp: ${{ steps.detect.outputs.magpie-tts-cpp }}
+      trellis2cpp: ${{ steps.detect.outputs.trellis2cpp }}
       rfdetr-cpp: ${{ steps.detect.outputs.rfdetr-cpp }}
       locate-anything-cpp: ${{ steps.detect.outputs.locate-anything-cpp }}
       vibevoice-cpp: ${{ steps.detect.outputs.vibevoice-cpp }}
@@ -587,7 +589,7 @@ jobs:
         with:
           go-version: '1.25.4'
       - name: Setup Node.js
-        uses: actions/setup-node@v6
+        uses: actions/setup-node@v7
         with:
           node-version: '22'
       - name: Build sherpa-onnx backend image and run realtime e2e tests
@@ -866,6 +868,38 @@ jobs:
       - name: Test qwen3-tts-cpp
         run: |
           make --jobs=5 --output-sync=target -C backend/go/qwen3-tts-cpp test
+  tests-magpie-tts-cpp:
+    needs: detect-changes
+    if: needs.detect-changes.outputs.magpie-tts-cpp == 'true' || needs.detect-changes.outputs.run-all == 'true'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          submodules: true
+      - name: Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential cmake curl libopenblas-dev ffmpeg
+      - name: Setup Go
+        uses: actions/setup-go@v5
+      - name: Display Go version
+        run: go version
+      - name: Proto Dependencies
+        run: |
+          # Install protoc
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
+          unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Build magpie-tts-cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/magpie-tts-cpp
+      - name: Test magpie-tts-cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/magpie-tts-cpp test
   # Per-backend smoke for rfdetr-cpp: builds the .so + Go binary and runs
   # `make -C backend/go/rfdetr-cpp test`. test.sh fetches the small (~20 MB)
   # rfdetr-nano-q8_0 GGUF from the published mudler/rfdetr-cpp-nano HF repo
@@ -902,6 +936,41 @@ jobs:
       - name: Test rfdetr-cpp
         run: |
           make --jobs=5 --output-sync=target -C backend/go/rfdetr-cpp test
+  # Weight-free packaged-backend smoke for trellis2cpp. Starting run.sh loads
+  # libtrellis2 + ggml, resolves the complete C ABI (including remeshing), and
+  # answers gRPC Health without downloading or loading the multi-GB model set.
+  tests-trellis2cpp:
+    needs: detect-changes
+    if: needs.detect-changes.outputs.trellis2cpp == 'true' || needs.detect-changes.outputs.run-all == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          submodules: true
+      - name: Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential cmake curl unzip
+      - name: Setup Go
+        uses: actions/setup-go@v5
+      - name: Display Go version
+        run: go version
+      - name: Proto Dependencies
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
+          unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Build trellis2cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/trellis2cpp
+      - name: Test trellis2cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/trellis2cpp test
   # Per-backend e2e for locate-anything-cpp: builds the .so + Go binary and
   # runs `make -C backend/go/locate-anything-cpp test`. test.sh fetches the
   # locate-anything-q8_0 GGUF (~6.3 GB, NVIDIA LocateAnything-3B) from the
@@ -1008,7 +1077,11 @@ jobs:
   # image + working dir.
   tests-vibevoice-cpp-grpc-transcription:
     needs: detect-changes
-    if: needs.detect-changes.outputs.vibevoice-cpp == 'true' || needs.detect-changes.outputs.run-all == 'true'
+    # Skip on release tag pushes: the ASR Q4_K model is ~10 GB and cannot be
+    # pulled from HF within the inner `go test -timeout 30m` budget on a CI
+    # runner, so every tag build hung and timed out. Still runs on PRs/branch
+    # pushes that touch vibevoice-cpp so regressions are caught off the release path.
+    if: (needs.detect-changes.outputs.vibevoice-cpp == 'true' || needs.detect-changes.outputs.run-all == 'true') && !startsWith(github.ref, 'refs/tags/')
     runs-on: bigger-runner
     timeout-minutes: 150
     steps:

.github/workflows/test.yml

@@ -24,8 +24,13 @@ jobs:
         uses: actions/checkout@v7
         with:
           submodules: true
-      - name: Free disk space
-        uses: ./.github/actions/free-disk-space
+      # No free-disk-space step here on purpose. That action exists to make room
+      # for docker buildx layers, and this job runs no buildx step. It was also
+      # sized for a `make test` that downloaded multi-GB GGUF/whisper fixtures
+      # and built llama-cpp/whisper/stablediffusion-ggml; after the test-suite
+      # reorg it does neither (see the Makefile test target). It cost ~3min of
+      # every run, and its tool-cache:true wipe also forced setup-go and
+      # setup-node to re-download toolchains that ship preinstalled.
       - name: Setup Go ${{ matrix.go-version }}
         uses: actions/setup-go@v5
         with:
@@ -48,7 +53,7 @@ jobs:
           sudo apt-get update
           sudo apt-get install curl ffmpeg libopus-dev
       - name: Setup Node.js
-        uses: actions/setup-node@v6
+        uses: actions/setup-node@v7
         with:
           node-version: '22'
       - name: Build React UI
@@ -100,7 +105,7 @@ jobs:
           brew install protobuf grpc make protoc-gen-go protoc-gen-go-grpc libomp llvm opus ffmpeg
           pip install --user --no-cache-dir grpcio-tools grpcio
       - name: Setup Node.js
-        uses: actions/setup-node@v6
+        uses: actions/setup-node@v7
         with:
           node-version: '22'
       - name: Build React UI
@@ -121,3 +126,19 @@ jobs:
           detached: true
           connect-timeout-seconds: 180
           limit-access-to-actor: true
+
+  # Fast standalone unit tests for the backends' pure C++ helpers - currently the
+  # llama-cpp message reconstruction (backend/cpp/llama-cpp/message_content.h),
+  # which guards the OpenAI chat content normalization (mudler/LocalAI#10524,
+  # #7324, #7528). The runner discovers every *_test.cpp under backend/cpp/, so
+  # new pure-C++ unit tests are picked up with no CI changes. These need only the
+  # C++ stdlib + nlohmann/json, so they run on every PR without the full
+  # llama.cpp + gRPC backend build. (The same suite is also wired as an opt-in
+  # CMake/ctest target, -DLLAMA_GRPC_BUILD_TESTS=ON, for in-backend-build runs.)
+  tests-backend-cpp:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+      - name: Run backend C++ unit tests
+        run: make test-backend-cpp

.github/workflows/tests-e2e.yml

@@ -3,6 +3,14 @@ name: 'E2E Backend Tests'
 
 on:
   pull_request:
+    # The e2e suite drives backends over gRPC directly and reads none of these
+    # paths, so a diff confined to them cannot move it.
+    # See .agents/ci-caching.md.
+    paths-ignore:
+      - 'gallery/**'
+      - 'docs/**'
+      - 'examples/**'
+      - '**/*.md'
   push:
     branches:
       - master
@@ -47,7 +55,7 @@ jobs:
           sudo apt-get update
           sudo apt-get install -y build-essential libopus-dev
       - name: Setup Node.js
-        uses: actions/setup-node@v6
+        uses: actions/setup-node@v7
         with:
           node-version: '22'
       - name: Build React UI

.github/workflows/tests-ui-e2e.yml

@@ -34,7 +34,7 @@ jobs:
           go-version: ${{ matrix.go-version }}
           cache: false
       - name: Setup Node.js
-        uses: actions/setup-node@v6
+        uses: actions/setup-node@v7
         with:
           node-version: '22'
       - name: Setup Bun

.github/workflows/yaml-check.yml

@@ -1,6 +1,11 @@
 name: 'Yamllint GitHub Actions'
 on:
   - pull_request
+
+concurrency:
+  group: ci-yamllint-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
 jobs:
   yamllint:
     name: 'Yamllint'

.gitignore

@@ -30,6 +30,7 @@ LocalAI
 # Go backend packages whose main lives under backend/go/.
 /cloud-proxy
 /local-store
+/valkey-store
 # prevent above rules from omitting the helm chart
 !charts/*
 # prevent above rules from omitting the api/localai folder
@@ -41,7 +42,12 @@ models/*
 test-models/
 test-dir/
 tests/e2e-aio/backends
-mock-backend
+# The mock backend binary built by `make build-mock-backend`. Anchored to its
+# full path: a bare `mock-backend` also matched the *directory* holding the
+# source, so git would not descend into it and adding a file there needed -f.
+# tests/e2e/mock-backend/.gitignore covers the same binary; kept here too so
+# the artifact stays ignored if that scoped file is ever removed.
+/tests/e2e/mock-backend/mock-backend
 
 release/
 
@@ -56,6 +62,11 @@ prepare
 /ggml-metal.metal
 docs/static/gallery.html
 
+# Hugo build output and lock files (docs/ and website/)
+docs/public/
+website/public/
+.hugo_build.lock
+
 # Protobuf generated files
 *.pb.go
 *pb2.py
@@ -94,3 +105,22 @@ core/http/react-ui/test-results/
 
 # SDD / brainstorm scratch (agent-driven development)
 .superpowers/
+
+# Local Apple signing material (never commit)
+.certs/
+
+# Pinned dev tools (e.g. FizzBee for the realtime-conformance gate)
+.tools/
+
+# FizzBee model-check artifacts: the parser emits <spec>.json next to each
+# .fizz and the checker writes run dirs under out/. Both are regenerated by
+# the realtime-conformance gate; only the .fizz sources are authoritative.
+formal-verification/*.json
+formal-verification/out/
+
+# `go build ./.github/ci/apexentries` drops a binary of the package name into
+# whatever directory it runs in, one `git add -A` away from being committed.
+# Both paths are anchored: an unanchored `apexentries` would also match the
+# package directory itself and untrack the source.
+/apexentries
+/.github/ci/apexentries/apexentries

.goreleaser.yaml

@@ -9,7 +9,8 @@ source:
   enabled: true
   name_template: '{{ .ProjectName }}-{{ .Tag }}-source'
 builds:
-  - main: ./cmd/local-ai
+  - id: local-ai
+    main: ./cmd/local-ai
     env:
       - CGO_ENABLED=0
     ldflags:
@@ -35,3 +36,19 @@ snapshot:
   version_template: "{{ .Tag }}-next"
 changelog:
   use: github-native
+# Sign + notarize the macOS server binary via the quill backend (runs on Linux,
+# no macOS runner needed). Disabled automatically when MACOS_SIGN_P12 is unset
+# (forks / PRs), so those builds stay unsigned and green.
+notarize:
+  macos:
+    - enabled: '{{ isEnvSet "MACOS_SIGN_P12" }}'
+      ids:
+        - local-ai
+      sign:
+        certificate: "{{.Env.MACOS_SIGN_P12}}"
+        password: "{{.Env.MACOS_SIGN_PASSWORD}}"
+      notarize:
+        issuer_id: "{{.Env.MACOS_NOTARY_ISSUER_ID}}"
+        key_id: "{{.Env.MACOS_NOTARY_KEY_ID}}"
+        key: "{{.Env.MACOS_NOTARY_KEY}}"
+        wait: true

.impeccable/live/config.json

@@ -0,0 +1,6 @@
+{
+  "files": ["core/http/react-ui/index.html"],
+  "insertBefore": "</body>",
+  "commentSyntax": "html",
+  "cspChecked": true
+}

ADOPTERS.md

@@ -0,0 +1,48 @@
+# Adopters
+
+Organisations running LocalAI, listed by the people who run it.
+
+If your organisation uses LocalAI and you are happy to say so publicly, open a
+pull request adding a row to the table below. That pull request is how we know
+we have permission to list you, which is why we do not add anybody ourselves.
+
+You do not need to be a large company, and you do not need to disclose anything
+sensitive. A sentence on what you use it for is more useful to other readers
+than a logo.
+
+## How to add yourself
+
+1. Add a row to the table, in alphabetical order.
+2. Use your organisation's usual name and a link to your site.
+3. Say briefly what you use LocalAI for, and whether it is in production.
+4. Open the pull request from an account that makes it plausible you speak for
+   the organisation, or say in the description who you are. We may ask.
+
+To be removed, open a pull request deleting your row, or email
+[info@localai.io](mailto:info@localai.io). We will not ask why.
+
+## Who is using LocalAI
+
+<!-- Keep alphabetical. Columns: Organisation | What for | Status -->
+
+| Organisation | What they use it for | Status |
+|---|---|---|
+| _Your organisation here_ | | |
+
+## What this list is not
+
+This is not a list of everyone who has ever starred the repository, and it is
+not a list of the employers of people who have contributed a patch. Both of
+those are easy to scrape and neither means what a logo wall implies.
+
+The website shows two separate things, both of which are checkable without
+anybody's permission:
+
+- **Engineers from these companies have contributed code.** Evidence is the
+  commit history plus the employer on that person's public GitHub profile. It
+  is a claim about a person, not about their employer.
+- **These projects integrate LocalAI.** Evidence is a reference to LocalAI in
+  that project's own repository or documentation.
+
+Those two lists live in [`website/data/ecosystem.yaml`](website/data/ecosystem.yaml).
+This file is the third, stronger thing: organisations that chose to say so.

AGENTS.md

@@ -32,16 +32,19 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 | [.agents/adding-gallery-models.md](.agents/adding-gallery-models.md) | Adding GGUF models from HuggingFace to the model gallery |
 | [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) | LocalAI Assistant chat modality — adding admin tools to the in-process MCP server, editing skill prompts, keeping REST + MCP + skills in sync |
 | [.agents/backend-signing.md](.agents/backend-signing.md) | Backend OCI image signing (keyless cosign + sigstore-go) — producer-side CI setup, consumer-side gallery `verification:` block, strict mode (`LOCALAI_REQUIRE_BACKEND_INTEGRITY`), revocation via `not_before` |
+| [.agents/preparing-a-release.md](.agents/preparing-a-release.md) | Cutting a release: PR labels, `RELEASE_NOTES_vX.Y.Z.md`, the blog post under `website/content/blog/`, and the demo clips under `website/static/media/` |
 
 ## Quick Reference
 
-- **Git hooks & coverage gates**: Run `make install-hooks` once per clone so the pre-commit lint + coverage gates run. **Never bypass them with `git commit --no-verify`, and never lower a coverage baseline or widen a gate's tolerance to turn a red gate green** — the coverage ratchet only moves up. If a change drops coverage, add tests to raise it (e.g. render-smoke specs). See [.agents/building-and-testing.md](.agents/building-and-testing.md).
+- **Coverage gates**: Never lower a coverage baseline or widen a gate's tolerance to turn a red gate green — the coverage ratchet only moves up. If a change drops coverage, add tests to raise it (e.g. render-smoke specs). See [.agents/building-and-testing.md](.agents/building-and-testing.md).
 - **Logging**: Use `github.com/mudler/xlog` (same API as slog)
 - **Go style**: Prefer `any` over `interface{}`
 - **Comments**: Explain *why*, not *what*
-- **Docs**: Update `docs/content/` when adding features or changing config
+- **Docs (docs-with-code rule)**: When you change user-facing behavior (API endpoints, CLI flags, config keys, or features), update the corresponding page under `docs/content/` in the SAME change, not as a follow-up. A user-facing change without a matching docs update is incomplete. See also the documentation conventions in [.agents/coding-style.md](.agents/coding-style.md).
 - **New API endpoints**: LocalAI advertises its capability surface in several independent places — swagger `@Tags`, `/api/instructions` registry, auth `RouteFeatureRegistry`, React UI `capabilities.js`, docs. Read [.agents/api-endpoints-and-auth.md](.agents/api-endpoints-and-auth.md) and follow its checklist — missing any surface means clients, admins, and the UI won't know the endpoint exists.
 - **Admin endpoints → MCP tool**: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in `pkg/mcp/localaitools/`. The LocalAI Assistant chat modality and the standalone `local-ai mcp-server` consume that package; drift between REST and MCP is a real risk. Read [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) — the `TestToolHTTPRouteMappingComplete` test fails until you wire the new tool and update the route map.
+- **Releases ship with a post and clips**: a release is not done at the tag. It needs labelled PRs, `RELEASE_NOTES_vX.Y.Z.md`, a blog post under `website/content/blog/`, and a short demo clip in `website/static/media/` for each notable feature. See [.agents/preparing-a-release.md](.agents/preparing-a-release.md).
 - **Build**: Inspect `Makefile` and `.github/workflows/` — ask the user before running long builds
 - **Backend OS coverage**: a new backend must target every OS it can build for, not just Linux. `.github/backend-matrix.yml` has two matrices — `include:` (Linux) and `includeDarwin:` (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the `includeDarwin` entry + `backend/index.yaml` `metal:` entries, or say in the PR why an OS is unsupported. See the darwin checklist in [.agents/adding-backends.md](.agents/adding-backends.md).
+- **Gallery variant ranking**: a gallery entry can declare `variants` (alternative builds of the same weights), and LocalAI ranks the ones a host can run by engine preference first, size second. A new backend that should be preferred on some hardware must be listed in `engineNamePreferenceRules` in `pkg/system/capabilities.go`; the sibling `backendBuildTagPreferenceRules` speaks build tags rather than engine names, and using the wrong table matches nothing without erroring. See [.agents/adding-backends.md](.agents/adding-backends.md).
 - **UI**: The active UI is the React app in `core/http/react-ui/`. The older Alpine.js/HTML UI in `core/http/static/` is pending deprecation — all new UI work goes in the React UI

CONTRIBUTING.md

@@ -198,7 +198,6 @@ For AI-assisted development, see [`AGENTS.md`](AGENTS.md) (or the equivalent [`C
 
 - Prefer modern Go idioms — for example, use `any` instead of `interface{}`.
 - Use [`golangci-lint`](https://golangci-lint.run) to catch common issues before submitting a PR.
-- Run `make install-hooks` once per clone to enable the pre-commit hook: Go changes run `make lint` + the coverage gate (`make test-coverage-check`); `core/http/react-ui/` changes run the Playwright e2e suite (`make test-ui`). Bypass a single commit with `git commit --no-verify`.
 - Use [`github.com/mudler/xlog`](https://github.com/mudler/xlog) for logging (same API as `slog`). Do not use `fmt.Println` or the standard `log` package for operational logging.
 - Use tab indentation for Go files (as defined in `.editorconfig`).
 
@@ -268,7 +267,7 @@ make test-e2e
 
 ### React UI tests and coverage
 
-The React UI (`core/http/react-ui/`) is covered by Playwright e2e specs, gated by a **monotonic line-coverage ratchet** (`make test-ui-coverage-check`, run in CI and pre-commit). The metric is non-deterministic — a fast local box reads higher than a slow CI runner for the same code — so a small tolerance is unavoidable.
+The React UI (`core/http/react-ui/`) is covered by Playwright e2e specs, gated by a **monotonic line-coverage ratchet** (`make test-ui-coverage-check`, run in CI). The metric is non-deterministic — a fast local box reads higher than a slow CI runner for the same code — so a small tolerance is unavoidable.
 
 **If your change lowers UI coverage, raise it back by adding specs — do not widen the tolerance or hand-lower the baseline.** A *render-smoke* spec (navigate to a page, assert its header is visible) cheaply covers an entire lazy page. See `core/http/react-ui/e2e/page-render-smoke.spec.js` and the full policy in [.agents/building-and-testing.md](.agents/building-and-testing.md#react-ui-coverage).
 

Dockerfile

@@ -12,12 +12,16 @@ ARG APT_MIRROR
 ARG APT_PORTS_MIRROR
 ENV DEBIAN_FRONTEND=noninteractive
 
+# hwdata ships /usr/share/hwdata/pci.ids. Without it, the ghw library we use
+# for hardware detection cannot resolve PCI vendor IDs and fails to enumerate
+# GPUs at all, so the image reports "No GPU detected" (see issue #10941).
 RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mirror \
     APT_MIRROR="${APT_MIRROR}" APT_PORTS_MIRROR="${APT_PORTS_MIRROR}" sh /usr/local/sbin/apt-mirror && \
     apt-get update && \
     apt-get install -y --no-install-recommends \
         ca-certificates curl wget espeak-ng libgomp1 \
-        ffmpeg libopenblas0 libopenblas-dev libopus0 sox && \
+        ffmpeg libopenblas0 libopenblas-dev libopus0 sox \
+        hwdata && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/*
 
@@ -171,6 +175,17 @@ RUN if [ "${BUILD_TYPE}" = "hipblas" ]; then \
     ln -s /opt/rocm-**/lib/llvm/lib/libomp.so /usr/lib/libomp.so \
     ; fi
 
+# ROCm's bundled libdrm_amdgpu is built with a hardcoded fallback lookup path
+# for the ASIC ID table (/opt/amdgpu/share/libdrm/amdgpu.ids), which only exists
+# if AMD's full amdgpu graphics/DKMS stack is installed. This compute-only image
+# doesn't have it, so hipblas/rocBLAS log "No such file or directory" on every
+# model load and can fail to identify the GPU. Point it at the equivalent file
+# Ubuntu's libdrm-common package already ships.
+RUN if [ "${BUILD_TYPE}" = "hipblas" ] && [ -f /usr/share/libdrm/amdgpu.ids ] && [ ! -e /opt/amdgpu/share/libdrm/amdgpu.ids ]; then \
+    mkdir -p /opt/amdgpu/share/libdrm && \
+    ln -s /usr/share/libdrm/amdgpu.ids /opt/amdgpu/share/libdrm/amdgpu.ids \
+    ; fi
+
 RUN expr "${BUILD_TYPE}" = intel && echo "intel" > /run/localai/capability || echo "not intel"
 
 # Cuda
@@ -378,7 +393,12 @@ RUN go install github.com/mikefarah/yq/v4@latest
 # If you cannot find a more suitable place for an addition, this layer is a suitable place for it.
 FROM requirements-drivers
 
-ENV HEALTHCHECK_ENDPOINT=http://localhost:8080/readyz
+# Optional override for the HEALTHCHECK target. Left empty so healthcheck.sh
+# derives the endpoint from the mode the container is actually running — the
+# same image runs `local-ai run` (HTTP on 8080) and `local-ai worker` (HTTP on
+# the gRPC base port minus one), and a hardcoded default marked every worker
+# permanently unhealthy (#10987). Set it to pin an explicit URL.
+ENV HEALTHCHECK_ENDPOINT=""
 
 ARG CUDA_MAJOR_VERSION=12
 ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
@@ -388,6 +408,7 @@ ENV NVIDIA_VISIBLE_DEVICES=all
 WORKDIR /
 
 COPY ./entrypoint.sh .
+COPY ./scripts/build/healthcheck.sh .
 
 # Copy the binary
 COPY --from=builder /build/local-ai ./
@@ -398,9 +419,22 @@ RUN --mount=from=builder,src=/build/,dst=/mnt/build \
 # Make sure the models directory exists
 RUN mkdir -p /models /backends /data
 
-# Define the health check command
-HEALTHCHECK --interval=1m --timeout=10m --retries=10 \
-  CMD curl -f ${HEALTHCHECK_ENDPOINT} || exit 1
+# Define the health check command.
+#
+# --start-period is the knob for slow starts, not --timeout/--retries. Since
+# #10949 a frontend's startup preload materializes HuggingFace artifacts before
+# the HTTP server binds (31 GB observed on a live cluster), so a healthy replica
+# can legitimately fail probes for a long time. Failures inside the start period
+# leave the container `starting` instead of burning retries, and the period ends
+# early on the first success — so a generous value costs a fast-starting
+# container nothing. A process that actually died is handled by the restart
+# policy, not by health.
+#
+# --timeout is a per-probe deadline: 10m meant a wedged probe could hang for ten
+# minutes and stretch detection without bound. A localhost curl that has not
+# answered in 10s is itself the fault being detected.
+HEALTHCHECK --start-period=60m --interval=1m --timeout=10s --retries=3 \
+  CMD /healthcheck.sh
 
 VOLUME /models /backends /configuration /data
 EXPOSE 8080

Makefile

@@ -1,5 +1,5 @@
 # Disable parallel execution for backend builds
-.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
+.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/trellis2cpp backends/trellis2cpp-darwin backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/valkey-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/magpie-tts-cpp backends/vllm-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin backends/audio-cpp backends/audio-cpp-darwin
 
 GOCMD=go
 GOTEST=$(GOCMD) test
@@ -69,7 +69,7 @@ else
 	GORELEASER=$(shell which goreleaser)
 endif
 
-TEST_PATHS?=./api/... ./pkg/... ./core/... ./backend/go/cloud-proxy/... ./backend/go/local-store/...
+TEST_PATHS?=./api/... ./pkg/... ./core/... ./backend/go/cloud-proxy/... ./backend/go/local-store/... ./backend/go/valkey-store/...
 
 ## Coverage output and the committed baseline that CI compares against.
 ## The gate is strict: total coverage must never decrease (no tolerance).
@@ -103,7 +103,7 @@ COVERAGE_E2E_LABELS?=!real-models
 COVERAGE_EXCLUDE_RE?=grpc/proto/.*[.]pb[.]go
 
 
-.PHONY: all test test-coverage test-coverage-baseline test-coverage-check test-ui test-ui-coverage-baseline test-ui-coverage-check install-hooks build vendor lint lint-all
+.PHONY: all test test-coverage test-coverage-baseline test-coverage-check test-backend-cpp test-build-scripts test-ui test-ui-coverage-baseline test-ui-coverage-check build vendor lint lint-all
 
 all: help
 
@@ -172,6 +172,15 @@ build-dev: ## Run LocalAI in dev mode with live reload
 dev-dist:
 	$(GORELEASER) build --snapshot --clean
 
+## PR-time variant of dev-dist: builds only the host platform instead of all
+## three release targets (linux/amd64, linux/arm64, darwin/arm64). The point of
+## running goreleaser on a PR is to catch a broken config or a broken
+## before-hook (protogen-go, react-ui, go mod tidy), and --single-target still
+## exercises every one of those. Nothing consumes a PR's cross-compiled
+## binaries. master pushes and tags still run the full dev-dist/dist.
+dev-dist-single:
+	$(GORELEASER) build --snapshot --clean --single-target
+
 dist:
 	$(GORELEASER) build --clean
 
@@ -201,6 +210,35 @@ test: prepare-test
 	OPUS_SHIM_LIBRARY=$(abspath ./pkg/opus/shim/libopusshim.so) \
 	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) --fail-fast -v -r $(TEST_PATHS)
 
+## Compiles and runs the standalone C++ unit tests for the backends (pure
+## helpers that depend only on the stdlib + nlohmann/json, no full backend
+## build). Discovers every *_test.cpp under backend/cpp/ - see
+## backend/cpp/run-unit-tests.sh. Set NLOHMANN_INCLUDE to skip the header fetch.
+test-backend-cpp:
+	bash backend/cpp/run-unit-tests.sh
+
+## Runs the shell-level regression tests for the image packaging scripts
+## (scripts/build/*_test.sh). These guard invariants that only ever break
+## inside a container build - a missing transitive dep, a partial cuDNN
+## family - and that no Go test can observe. Needs only bash + gcc + ldd.
+test-build-scripts:
+	@set -e; for t in scripts/build/*_test.sh; do echo "== $$t"; bash "$$t"; done
+
+## Runs the unit tests for the CI helper scripts under scripts/lib/. Currently
+## the backend matrix path filter, whose failure mode is invisible in CI: it
+## emits an empty matrix, every job goes green, and the change ships to no
+## image at all (see PR #10946). Plain `node --test`, no dependencies.
+test-ci-scripts:
+	@set -e; for t in scripts/lib/*_test.mjs; do echo "== $$t"; node --test "$$t"; done
+
+## Runs the unit tests for the shared python backend helpers. These modules are
+## pure stdlib on purpose so they run without any backend venv; the list is
+## explicit because their siblings (model_identity_test) import grpc and the
+## generated protobufs, which only exist inside a built backend.
+PYTHON_HELPER_TESTS?=python_utils_test vllm_utils_test model_utils_test mlx_utils_test parent_watch_test
+test-python-helpers:
+	cd backend/python/common && python3 -m unittest $(PYTHON_HELPER_TESTS)
+
 ## Runs the core suite ($(TEST_PATHS)) with statement-coverage instrumentation
 ## and writes a merged profile to $(COVERAGE_PROFILE). Deliberately omits
 ## --fail-fast so a single failure doesn't truncate the coverage number, and
@@ -248,8 +286,7 @@ LINT_EXCLUDE_DIRS_RE=/(backend/go/(piper|silero-vad|llm)|cmd/launcher)(/|$$)
 
 ## Set LINT_NEW_FROM to a git ref to override .golangci.yml's
 ## new-from-merge-base (origin/master). Useful from a fork clone where
-## origin/master is stale relative to the canonical repo — the pre-commit
-## hook passes the resolved upstream ref here so local lint matches CI.
+## origin/master is stale relative to the canonical repo.
 LINT_NEW_FROM?=
 lint:
 	@command -v golangci-lint >/dev/null 2>&1 || { \
@@ -268,17 +305,6 @@ lint-all:
 	}
 	golangci-lint run --new=false --new-from-merge-base= --new-from-rev= $$(go list -e -f '{{.Dir}}' ./... | grep -vE '$(LINT_EXCLUDE_DIRS_RE)')
 
-########################################################
-## Git hooks
-########################################################
-## Points git at the versioned .githooks/ directory so the pre-commit hook
-## (lint + coverage gate) runs locally. Run once per clone. Undo with:
-## `git config --unset core.hooksPath`. Skip a single commit with
-## `git commit --no-verify`.
-install-hooks:
-	git config core.hooksPath .githooks
-	@echo 'Installed git hooks: core.hooksPath -> .githooks (pre-commit runs lint + test-coverage-check on Go changes)'
-
 ########################################################
 ## E2E AIO tests (uses standard image with pre-configured models)
 ########################################################
@@ -377,6 +403,15 @@ test-stores: backends/local-store
 	BACKENDS_PATH=$(abspath ./)/backends \
 	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) -v -r tests/integration
 
+## Valkey-backed vector-store integration. Requires a running Valkey Search
+## server (valkey/valkey-bundle:9.1.0) reachable at $$VALKEY_ADDR — the suite
+## skips itself when VALKEY_ADDR is unset. Builds the backend on demand and
+## points the model loader at it via BACKENDS_PATH. Label-filtered to the
+## valkey specs so it does not also run the in-memory local-store suite.
+test-valkey-store: backends/valkey-store
+	BACKENDS_PATH=$(abspath ./)/backends \
+	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) --label-filter='valkey' -v -r tests/integration
+
 test-opus:
 	@echo 'Running opus backend tests'
 	$(MAKE) -C backend/go/opus libopusshim.so
@@ -398,6 +433,23 @@ test-realtime: build-mock-backend
 	@echo 'Running realtime e2e tests (mock backend)'
 	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --label-filter="Realtime && !real-models" --flake-attempts $(TEST_FLAKES) -v -r ./tests/e2e
 
+# Verify the realtime state-machine implementations conform to their formal
+# designs (Go transition/rapid tests under -race + FizzBee model check of the
+# authoritative specs). See docs/design/realtime-state-machines.md (Part 6) and
+# docs/design/specs/README.md.
+test-realtime-conformance:
+	GOCMD=$(GOCMD) ./scripts/realtime-conformance.sh
+
+# Verify the shared model-loader shutdown behavior independently of any API
+# modality (focused loader/gRPC/distributed/worker tests under -race + FizzBee).
+test-model-lifecycle-conformance:
+	GOCMD=$(GOCMD) ./scripts/model-lifecycle-conformance.sh
+
+# Install the pinned, checksum-verified FizzBee model checker (into .tools/,
+# gitignored) used by the conformance targets. Idempotent; no-op if present.
+install-fizzbee:
+	./scripts/install-fizzbee.sh
+
 # Container-based real-model realtime testing. Build env vars / pipeline
 # definition kept here so test-realtime-models-docker can drive a fully wired
 # pipeline (VAD + STT + LLM + TTS) from inside a containerised runner.
@@ -546,6 +598,7 @@ prepare-test-extra: protogen-python
 	$(MAKE) -C backend/python/chatterbox
 	$(MAKE) -C backend/python/vllm
 	$(MAKE) -C backend/python/vllm-omni
+	$(MAKE) -C backend/python/longcat-video
 	$(MAKE) -C backend/python/sglang
 	$(MAKE) -C backend/python/vibevoice
 	$(MAKE) -C backend/python/liquid-audio
@@ -567,6 +620,8 @@ prepare-test-extra: protogen-python
 	$(MAKE) -C backend/rust/kokoros kokoros-grpc
 	$(MAKE) -C backend/go/rfdetr-cpp
 	$(MAKE) -C backend/go/locate-anything-cpp
+	$(MAKE) -C backend/go/trellis2cpp
+	$(MAKE) -C backend/go/valkey-store
 
 test-extra: prepare-test-extra
 	$(MAKE) -C backend/python/transformers test
@@ -575,6 +630,7 @@ test-extra: prepare-test-extra
 	$(MAKE) -C backend/python/chatterbox test
 	$(MAKE) -C backend/python/vllm test
 	$(MAKE) -C backend/python/vllm-omni test
+	$(MAKE) -C backend/python/longcat-video test
 	$(MAKE) -C backend/python/vibevoice test
 	$(MAKE) -C backend/python/liquid-audio test
 	$(MAKE) -C backend/python/moonshine test
@@ -597,6 +653,9 @@ test-extra: prepare-test-extra
 	$(MAKE) -C backend/go/locate-anything-cpp test
 	$(MAKE) -C backend/go/depth-anything-cpp test
 	$(MAKE) -C backend/go/supertonic test
+	$(MAKE) -C backend/go/vllm-cpp test
+	$(MAKE) -C backend/go/trellis2cpp test
+	$(MAKE) -C backend/go/valkey-store test
 
 ##
 ## End-to-end gRPC tests that exercise a built backend container image.
@@ -626,6 +685,9 @@ test-extra: prepare-test-extra
 ## suite against it.
 ##
 BACKEND_TEST_MODEL_URL?=https://huggingface.co/Qwen/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q8_0.gguf
+## Suite timeout for `go test`. Wrappers whose model download alone can eat
+## most of the default (multi-GB models on a slow HF CDN day) override this.
+BACKEND_TEST_TIMEOUT?=30m
 
 ## Generic target — runs the suite against whatever BACKEND_IMAGE points at.
 ## Depends on protogen-go so pkg/grpc/proto is generated before `go test`.
@@ -653,7 +715,7 @@ test-extra-backend: protogen-go
 	BACKEND_TEST_FACE_IMAGE_3_URL="$$BACKEND_TEST_FACE_IMAGE_3_URL" \
 	BACKEND_TEST_FACE_IMAGE_3_FILE="$$BACKEND_TEST_FACE_IMAGE_3_FILE" \
 	BACKEND_TEST_VERIFY_DISTANCE_CEILING="$$BACKEND_TEST_VERIFY_DISTANCE_CEILING" \
-	go test -v -timeout 30m ./tests/e2e-backends/...
+	go test -v -timeout $(BACKEND_TEST_TIMEOUT) ./tests/e2e-backends/...
 
 ## Convenience wrappers: build the image, then exercise it.
 test-extra-backend-llama-cpp: docker-build-llama-cpp
@@ -676,6 +738,16 @@ test-extra-backend-turboquant: docker-build-turboquant
 	BACKEND_TEST_CACHE_TYPE_V=turbo3 \
 	$(MAKE) test-extra-backend
 
+## bonsai: exercises the llama.cpp-fork backend with a real Q1_0 (1-bit) model —
+## the PrismML Bonsai-8B GGUF, whose weight quant is *only* decodable by the fork's
+## Q1_0 kernels. Loading it is what makes this backend distinct from stock llama-cpp;
+## a standard-quant model would only test the upstream code path the llama-cpp backend
+## already covers.
+test-extra-backend-bonsai: docker-build-bonsai
+	BACKEND_IMAGE=local-ai-backend:bonsai \
+	BACKEND_TEST_MODEL_URL=https://huggingface.co/prism-ml/Bonsai-8B-gguf/resolve/main/Bonsai-8B-Q1_0.gguf \
+	$(MAKE) test-extra-backend
+
 ## Audio transcription wrapper for the llama-cpp backend.
 ## Drives the new AudioTranscription / AudioTranscriptionStream RPCs against
 ## ggml-org/Qwen3-ASR-0.6B-GGUF (a small ASR model that requires its mmproj
@@ -993,6 +1065,7 @@ test-extra-backend-vibevoice-cpp-tts: docker-build-vibevoice-cpp
 ## post-image disk budget.
 test-extra-backend-vibevoice-cpp-transcription: docker-build-vibevoice-cpp
 	BACKEND_IMAGE=local-ai-backend:vibevoice-cpp \
+	BACKEND_TEST_TIMEOUT=120m \
 	BACKEND_TEST_MODEL_URL='https://huggingface.co/mudler/vibevoice.cpp-models/resolve/main/vibevoice-asr-q4_k.gguf#vibevoice-asr-q4_k.gguf' \
 	BACKEND_TEST_EXTRA_FILES='https://huggingface.co/mudler/vibevoice.cpp-models/resolve/main/tokenizer.gguf#tokenizer.gguf' \
 	BACKEND_TEST_AUDIO_URL=https://github.com/ggml-org/whisper.cpp/raw/master/samples/jfk.wav \
@@ -1020,7 +1093,19 @@ test-extra-backend-whisper-transcription: docker-build-whisper
 ## is reachable.
 test-extra-backend-parakeet-cpp-transcription: docker-build-parakeet-cpp
 	BACKEND_IMAGE=local-ai-backend:parakeet-cpp \
-	BACKEND_TEST_MODEL_URL=https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/tdt_ctc-110m-f16.gguf \
+	BACKEND_TEST_MODEL_URL=https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/realtime_eou_120m-v1-f16.gguf \
+	BACKEND_TEST_AUDIO_URL=https://github.com/ggml-org/whisper.cpp/raw/master/samples/jfk.wav \
+	BACKEND_TEST_CAPS=health,load,transcription \
+	$(MAKE) test-extra-backend
+
+## Audio transcription wrapper for the moss-transcribe-cpp (moss-transcribe.cpp
+## ggml port) backend. Mirrors test-extra-backend-parakeet-cpp-transcription:
+## drives the AudioTranscription RPC against a published MOSS GGUF using the JFK
+## 11s clip from whisper.cpp's CI samples. Not part of the default test suite -
+## run explicitly once the pinned model URL is reachable.
+test-extra-backend-moss-transcribe-cpp-transcription: docker-build-moss-transcribe-cpp
+	BACKEND_IMAGE=local-ai-backend:moss-transcribe-cpp \
+	BACKEND_TEST_MODEL_URL=https://huggingface.co/mudler/moss-transcribe.cpp-gguf/resolve/main/moss-transcribe-q5_k.gguf \
 	BACKEND_TEST_AUDIO_URL=https://github.com/ggml-org/whisper.cpp/raw/master/samples/jfk.wav \
 	BACKEND_TEST_CAPS=health,load,transcription \
 	$(MAKE) test-extra-backend
@@ -1133,6 +1218,10 @@ backends/privacy-filter-darwin: build
 	bash ./scripts/build/privacy-filter-darwin.sh
 	./local-ai backends install "ocifile://$(abspath ./backend-images/privacy-filter.tar)"
 
+backends/audio-cpp-darwin: build
+	bash ./scripts/build/audio-cpp-darwin.sh
+	./local-ai backends install "ocifile://$(abspath ./backend-images/audio-cpp.tar)"
+
 build-darwin-python-backend: build
 	bash ./scripts/build/python-darwin.sh
 
@@ -1163,6 +1252,10 @@ backends/stablediffusion-ggml-darwin:
 	BACKEND=stablediffusion-ggml BUILD_TYPE=metal $(MAKE) build-darwin-go-backend
 	./local-ai backends install "ocifile://$(abspath ./backend-images/stablediffusion-ggml.tar)"
 
+backends/trellis2cpp-darwin:
+	BACKEND=trellis2cpp BUILD_TYPE=metal $(MAKE) build-darwin-go-backend
+	./local-ai backends install "ocifile://$(abspath ./backend-images/trellis2cpp.tar)"
+
 backend-images:
 	mkdir -p backend-images
 
@@ -1174,6 +1267,10 @@ BACKEND_IK_LLAMA_CPP = ik-llama-cpp|ik-llama-cpp|.|false|false
 # turboquant is a llama.cpp fork with TurboQuant KV-cache quantization.
 # Reuses backend/cpp/llama-cpp grpc-server sources via a thin wrapper Makefile.
 BACKEND_TURBOQUANT = turboquant|turboquant|.|false|false
+# bonsai is a llama.cpp fork (PrismML) adding the Q1_0 (1-bit) and Q2_0 (ternary)
+# weight-quant kernels the Bonsai / Ternary-Bonsai models ship in. Reuses
+# backend/cpp/llama-cpp grpc-server sources via a thin wrapper Makefile.
+BACKEND_BONSAI = bonsai|bonsai|.|false|false
 # ds4 is antirez/ds4, a DeepSeek V4 Flash-specific inference engine.
 # Single-model; hardware-only validation lives at tests/e2e-backends/
 # (BACKEND_BINARY mode); see docs/superpowers/plans/2026-05-11-ds4-backend.md.
@@ -1182,21 +1279,32 @@ BACKEND_DS4 = ds4|ds4|.|false|false
 # openai-privacy-filter PII/NER token classifier) — the TokenClassify RPC for
 # the PII redactor tier, on stock ggml with no llama.cpp carry-patches.
 BACKEND_PRIVACY_FILTER = privacy-filter|privacy-filter|.|false|false
+# audio-cpp wraps 0xShug0/audio.cpp, a multi-family ggml audio inference engine
+# (TTS, ASR, VAD, diarization, source separation, music generation). Builds
+# against apt gRPC/protobuf rather than a prebuilt base-grpc image; the reason
+# is on the audio-cpp block in .github/backend-matrix.yml.
+BACKEND_AUDIO_CPP = audio-cpp|audio-cpp|.|false|false
 
 # Golang backends
 BACKEND_PIPER = piper|golang|.|false|true
 BACKEND_LOCAL_STORE = local-store|golang|.|false|true
+BACKEND_VALKEY_STORE = valkey-store|golang|.|false|true
 BACKEND_CLOUD_PROXY = cloud-proxy|golang|.|false|true
 BACKEND_HUGGINGFACE = huggingface|golang|.|false|true
 BACKEND_SILERO_VAD = silero-vad|golang|.|false|true
 BACKEND_STABLEDIFFUSION_GGML = stablediffusion-ggml|golang|.|--progress=plain|true
+BACKEND_TRELLIS2CPP = trellis2cpp|golang|.|--progress=plain|true
 BACKEND_WHISPER = whisper|golang|.|false|true
 BACKEND_CRISPASR = crispasr|golang|.|false|true
 BACKEND_PARAKEET_CPP = parakeet-cpp|golang|.|false|true
+BACKEND_MOSS_TRANSCRIBE_CPP = moss-transcribe-cpp|golang|.|false|true
 BACKEND_DEPTH_ANYTHING_CPP = depth-anything-cpp|golang|.|false|true
 BACKEND_VOXTRAL = voxtral|golang|.|false|true
 BACKEND_ACESTEP_CPP = acestep-cpp|golang|.|false|true
 BACKEND_QWEN3_TTS_CPP = qwen3-tts-cpp|golang|.|false|true
+BACKEND_MOSS_TTS_CPP = moss-tts-cpp|golang|.|false|true
+BACKEND_MAGPIE_TTS_CPP = magpie-tts-cpp|golang|.|false|true
+BACKEND_VLLM_CPP = vllm-cpp|golang|.|false|true
 BACKEND_OMNIVOICE_CPP = omnivoice-cpp|golang|.|false|true
 BACKEND_VIBEVOICE_CPP = vibevoice-cpp|golang|.|false|true
 BACKEND_LOCALVQE = localvqe|golang|.|false|true
@@ -1218,6 +1326,7 @@ BACKEND_NEUTTS = neutts|python|.|false|true
 BACKEND_KOKORO = kokoro|python|.|false|true
 BACKEND_VLLM = vllm|python|.|false|true
 BACKEND_VLLM_OMNI = vllm-omni|python|.|false|true
+BACKEND_LONGCAT_VIDEO = longcat-video|python|.|--progress=plain|true
 BACKEND_SGLANG = sglang|python|.|false|true
 BACKEND_DIFFUSERS = diffusers|python|.|--progress=plain|true
 BACKEND_CHATTERBOX = chatterbox|python|.|false|true
@@ -1275,17 +1384,22 @@ endef
 $(eval $(call generate-docker-build-target,$(BACKEND_LLAMA_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_IK_LLAMA_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_TURBOQUANT)))
+$(eval $(call generate-docker-build-target,$(BACKEND_BONSAI)))
 $(eval $(call generate-docker-build-target,$(BACKEND_DS4)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PRIVACY_FILTER)))
+$(eval $(call generate-docker-build-target,$(BACKEND_AUDIO_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PIPER)))
 $(eval $(call generate-docker-build-target,$(BACKEND_LOCAL_STORE)))
+$(eval $(call generate-docker-build-target,$(BACKEND_VALKEY_STORE)))
 $(eval $(call generate-docker-build-target,$(BACKEND_CLOUD_PROXY)))
 $(eval $(call generate-docker-build-target,$(BACKEND_HUGGINGFACE)))
 $(eval $(call generate-docker-build-target,$(BACKEND_SILERO_VAD)))
 $(eval $(call generate-docker-build-target,$(BACKEND_STABLEDIFFUSION_GGML)))
+$(eval $(call generate-docker-build-target,$(BACKEND_TRELLIS2CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_WHISPER)))
 $(eval $(call generate-docker-build-target,$(BACKEND_CRISPASR)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PARAKEET_CPP)))
+$(eval $(call generate-docker-build-target,$(BACKEND_MOSS_TRANSCRIBE_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_DEPTH_ANYTHING_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_VOXTRAL)))
 $(eval $(call generate-docker-build-target,$(BACKEND_OPUS)))
@@ -1302,6 +1416,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_NEUTTS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_KOKORO)))
 $(eval $(call generate-docker-build-target,$(BACKEND_VLLM)))
 $(eval $(call generate-docker-build-target,$(BACKEND_VLLM_OMNI)))
+$(eval $(call generate-docker-build-target,$(BACKEND_LONGCAT_VIDEO)))
 $(eval $(call generate-docker-build-target,$(BACKEND_SGLANG)))
 $(eval $(call generate-docker-build-target,$(BACKEND_DIFFUSERS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_CHATTERBOX)))
@@ -1319,6 +1434,9 @@ $(eval $(call generate-docker-build-target,$(BACKEND_WHISPERX)))
 $(eval $(call generate-docker-build-target,$(BACKEND_ACE_STEP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_ACESTEP_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_QWEN3_TTS_CPP)))
+$(eval $(call generate-docker-build-target,$(BACKEND_MOSS_TTS_CPP)))
+$(eval $(call generate-docker-build-target,$(BACKEND_MAGPIE_TTS_CPP)))
+$(eval $(call generate-docker-build-target,$(BACKEND_VLLM_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_OMNIVOICE_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_VIBEVOICE_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_LOCALVQE)))
@@ -1338,7 +1456,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_SUPERTONIC)))
 docker-save-%: backend-images
 	docker save local-ai-backend:$* -o backend-images/$*.tar
 
-docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-privacy-filter
+docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-magpie-tts-cpp docker-build-vllm-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter docker-build-trellis2cpp docker-build-valkey-store docker-build-audio-cpp
 
 ########################################################
 ### Mock Backend for E2E Tests
@@ -1373,7 +1491,7 @@ test-ui-e2e: build-ui-test-server
 UI_TEST_WORKERS ?=
 PLAYWRIGHT_WORKERS_FLAG = $(if $(UI_TEST_WORKERS),--workers=$(UI_TEST_WORKERS),)
 
-## Fast Playwright e2e run used by the pre-commit hook on React UI changes.
+## Fast Playwright e2e run for local React UI validation.
 ## Force-rebuilds the (non-instrumented) dist so the suite tests the working
 ## tree — not a stale dist the `react-ui` skip-guard would leave — re-embeds
 ## it into ui-test-server, and runs the specs. Uses the nix-provided browser
@@ -1430,7 +1548,12 @@ swagger:
 gen-assets:
 	$(GOCMD) run core/dependencies_manager/manager.go webui_static.yaml core/http/static/assets
 
-## Documentation
+## Documentation and website
+# The published site is two Hugo sites: website/ owns the root, docs/ is nested
+# under /docs/. Serve them separately while editing; use `make site` to get the
+# merged tree (including the legacy URL redirects) that GitHub Pages deploys.
+SITE_BASE_URL?=http://localhost:8000
+
 docs/layouts/_default:
 	mkdir -p docs/layouts/_default
 
@@ -1442,24 +1565,66 @@ docs/public: docs/layouts/_default docs/static/gallery.html
 
 docs-clean:
 	rm -rf docs/public
+	rm -rf website/public
 	rm -rf docs/static/gallery.html
 
 .PHONY: docs
 docs: docs/static/gallery.html
 	cd docs && hugo serve
 
+.PHONY: website
+website:
+	cd website && hugo serve
+
+.PHONY: site
+site: docs/static/gallery.html
+	rm -rf website/public docs/public
+	cd website && hugo --minify --baseURL "$(SITE_BASE_URL)/"
+	cd docs && hugo --minify --baseURL "$(SITE_BASE_URL)/docs/"
+	mkdir -p website/public/docs
+	cp -R docs/public/. website/public/docs/
+	./.github/ci/gen-redirects.sh website/public "$(SITE_BASE_URL)/"
+
+.PHONY: site-serve
+site-serve: site
+	cd website/public && python3 -m http.server 8000
+
 ########################################################
 ## Platform-specific builds
 ########################################################
 
 ## fyne cross-platform build
-build-launcher-darwin: build-launcher
-	go run github.com/tiagomelo/macos-dmg-creator/cmd/createdmg@latest \
-	--appName "LocalAI" \
-	--appBinaryPath "$(LAUNCHER_BINARY_NAME)" \
-	--bundleIdentifier "com.localai.launcher" \
-	--iconPath "core/http/static/logo.png" \
-	--outputDir "dist/"
+# Build LocalAI.app from the launcher via fyne (metadata read from cmd/launcher/FyneApp.toml).
+# Signing happens via contrib/macos/sign-and-notarize.sh, which is a no-op when the signing
+# secrets are unset, so unsigned local/fork builds keep working.
+build-launcher-darwin:
+	rm -rf dist/LocalAI.app cmd/launcher/LocalAI.app
+	mkdir -p dist
+	cd cmd/launcher && go run fyne.io/tools/cmd/fyne@latest package -os darwin -icon ../../core/http/static/logo.png --executable $(LAUNCHER_BINARY_NAME)
+	mv cmd/launcher/LocalAI.app dist/LocalAI.app
+	bash contrib/macos/sign-and-notarize.sh sign dist/LocalAI.app
+
+# Notarize + staple the .app itself, then wrap it into a drag-to-Applications
+# DMG via hdiutil and sign the DMG. The app is stapled BEFORE packaging so the
+# bundle carries its own ticket and verifies offline (a dmg-only staple leaves
+# the app relying on an online Gatekeeper check, which fails offline / once the
+# app is copied out of the dmg). No-op without notary secrets.
+dmg-launcher-darwin: build-launcher-darwin
+	bash contrib/macos/sign-and-notarize.sh notarize-app dist/LocalAI.app
+	rm -rf dist/dmg dist/LocalAI.dmg
+	mkdir -p dist/dmg
+	cp -R dist/LocalAI.app dist/dmg/LocalAI.app
+	ln -s /Applications dist/dmg/Applications
+	hdiutil create -volname "LocalAI" -srcfolder dist/dmg -ov -format UDZO dist/LocalAI.dmg
+	bash contrib/macos/sign-and-notarize.sh sign dist/LocalAI.dmg
+
+# Submit the DMG to Apple notarization and staple the ticket (no-op without notary secrets).
+notarize-launcher-darwin: dmg-launcher-darwin
+	bash contrib/macos/sign-and-notarize.sh notarize dist/LocalAI.dmg
+
+# Single entrypoint for CI: build -> sign app -> notarize+staple app -> dmg -> sign dmg -> notarize+staple dmg.
+release-launcher-darwin: notarize-launcher-darwin
+	@echo "dist/LocalAI.dmg is ready"
 
 build-launcher-linux:
-	cd cmd/launcher && go run fyne.io/tools/cmd/fyne@latest package -os linux -icon ../../core/http/static/logo.png --executable $(LAUNCHER_BINARY_NAME)-linux && mv launcher.tar.xz ../../$(LAUNCHER_BINARY_NAME)-linux.tar.xz
+	cd cmd/launcher && go run fyne.io/tools/cmd/fyne@latest package -os linux -icon ../../core/http/static/logo.png --executable $(LAUNCHER_BINARY_NAME)-linux && mv LocalAI.tar.xz ../../$(LAUNCHER_BINARY_NAME)-linux.tar.xz

README.md

@@ -177,6 +177,7 @@ For more details, see the [Getting Started guide](https://localai.io/basics/gett
 
 ## Latest News
 
+- **June 2026**: New native biometric backends from the LocalAI team: [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) for speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion) and [face-detect.cpp](https://github.com/mudler/face-detect.cpp) for face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace). Both are from-scratch C++/ggml engines with no Python or onnxruntime at inference, self-contained GGUF weights, bit-exact parity with the reference, and GPU cuDNN parity, replacing the heavier Python `insightface` and `speaker-recognition` backends ([PR #10441](https://github.com/mudler/LocalAI/pull/10441)).
 - **June 2026**: New [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) (a tiny Go client for the Realtime API with a full talk-back voice loop and tool calling), plus [streaming of the realtime LLM / TTS / transcription pipeline stages](https://github.com/mudler/LocalAI/pull/10176) and [configurable WebRTC ICE candidates](https://github.com/mudler/LocalAI/pull/10231).
 - **June 2026**: Big speech push: the [parakeet.cpp](https://github.com/mudler/parakeet.cpp) ASR engine gains [NeMo-faithful segment timestamps](https://github.com/mudler/LocalAI/pull/10207), a [multilingual streaming Nemotron-3.5 model](https://github.com/mudler/LocalAI/pull/10199), [dynamic batching for concurrent transcription](https://github.com/mudler/LocalAI/pull/10112) and [CUDA graphs](https://github.com/mudler/LocalAI/pull/10273); the new [CrispASR backend](https://github.com/mudler/LocalAI/pull/10099) adds multi-architecture ASR + TTS, and [60 Piper TTS voices across 42 languages](https://github.com/mudler/LocalAI/pull/10296) land in the gallery (plus [per-request TTS instructions and params](https://github.com/mudler/LocalAI/pull/10172)).
 - **June 2026**: New backends and models: [locate-anything.cpp](https://github.com/mudler/LocalAI/pull/10264) for open-vocabulary object detection via ggml, [Ideogram4 image generation](https://github.com/mudler/LocalAI/pull/10201) in stablediffusion-ggml, [llama.cpp video input](https://github.com/mudler/LocalAI/pull/10216), and the [Gemma 4 QAT family with MTP speculative-decoding pairs](https://github.com/mudler/LocalAI/pull/10215). Plus an [interactive CLI chat mode](https://github.com/mudler/LocalAI/pull/10226) and [RAG source citations in agent responses](https://github.com/mudler/LocalAI/pull/10228).
@@ -230,13 +231,21 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native
 
 | Backend | What it does |
 |---------|-------------|
+| [vllm.cpp](https://github.com/mudler/vllm.cpp) | From-scratch C++20 port of vLLM for text generation: paged KV cache, continuous batching, prefix caching, safetensors + GGUF loading, engine-enforced structured output, on CPU, CUDA, Metal and Vulkan |
 | [parakeet.cpp](https://github.com/mudler/parakeet.cpp) | C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription |
-| [ced.cpp](https://github.com/mudler/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
-| [voxtral.c](https://github.com/mudler/voxtral.c) | Voxtral Realtime 4B speech-to-text in pure C |
+| [moss-transcribe.cpp](https://github.com/localai-org/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass |
+| [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | C++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec |
+| [magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp) | C++/GGML port of NVIDIA's Magpie TTS Multilingual 357M: 22.05 kHz mono text-to-speech in 5 voices and 9+ languages, with the NanoCodec neural codec and tokenizer/G2P embedded in a single GGUF |
+| [ced.cpp](https://github.com/localai-org/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
+| [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
+| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Mistral Voxtral-4B-TTS text-to-speech in pure C: 20 preset voices across 9 languages, 24 kHz WAV output, no dependencies beyond libc |
 | [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization |
-| [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
+| [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
 | [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) |
 | [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose estimation |
+| [face-detect.cpp](https://github.com/mudler/face-detect.cpp) | Face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace), replacing the Python insightface backend |
+| [free-splatter.cpp](https://github.com/localai-org/free-splatter.cpp) | Pose-free 3D reconstruction (FreeSplatter): turns a handful of plain photos into 3D Gaussians, no camera poses or GPU required |
+| [trellis2.cpp](https://github.com/localai-org/trellis2cpp) | C++/GGML port of Microsoft TRELLIS.2: single-image to textured 3D mesh (GLB with PBR materials) |
 | [privacy-filter.cpp](https://github.com/localai-org/privacy-filter.cpp) | Standalone GGML PII/NER token-classification engine powering LocalAI's PII redaction tier |
 | [LocalVQE](https://github.com/localai-org/LocalVQE) | Joint acoustic echo cancellation, noise suppression, and dereverberation |
 | [local-store](https://github.com/mudler/LocalAI) | Local-first vector database for embeddings (shipped in-tree) |

backend/Dockerfile.audio-cpp

@@ -0,0 +1,120 @@
+ARG BASE_IMAGE=ubuntu:24.04
+ARG APT_MIRROR=""
+ARG APT_PORTS_MIRROR=""
+
+# audio-cpp: 0xShug0/audio.cpp, a ggml audio inference framework covering TTS,
+# ASR, VAD, diarization, source separation and music generation, wrapped as a
+# LocalAI gRPC backend.
+#
+# BASE_IMAGE is ubuntu:24.04 for cpu and vulkan builds, or
+# nvidia/cuda:<ver>-devel-ubuntu24.04 for cublas builds; both ship apt and
+# Ubuntu Noble packages, and the CUDA base additionally provides
+# /usr/local/cuda. BUILD_TYPE selects the engine backend in the Makefile:
+# "" = portable CPU with all ggml CPU variants, "cublas" ->
+# -DENGINE_ENABLE_CUDA=ON, "vulkan" -> -DENGINE_ENABLE_VULKAN=ON. Darwin
+# (Metal) builds bypass this Dockerfile entirely.
+#
+# Upstream needs GCC 13 or newer, which ubuntu:24.04 and the CUDA 12/13
+# devel-ubuntu24.04 images all provide.
+#
+# THIS BACKEND CANNOT USE .docker/install-base-deps.sh OR THE PREBUILT
+# quay.io/go-skynet/ci-cache:base-grpc-* IMAGES, AND THAT IS NOT A STYLE CHOICE.
+#
+# Both supply gRPC v1.65 built from source at /opt/grpc, which downstream
+# Dockerfiles copy to /usr/local. That gRPC vendors protobuf v26, and protobuf
+# has depended on abseil since v22: google/protobuf/message_lite.h includes
+# absl/strings/cord.h. audio.cpp links sentencepiece, and our CMakeLists sets
+# SPM_PROTOBUF_PROVIDER=package so sentencepiece uses the same protobuf the
+# generated backend.pb.cc was built against (the alternative broke every
+# nested-message parse; the full account is in backend/cpp/audio-cpp/CMakeLists.txt).
+# That makes sentencepiece's init.h include the external message_lite.h while it
+# still includes its own vendored mini-abseil from third_party/absl. The vendored
+# copy declares `namespace absl { namespace internal { ... } }` and real abseil
+# declares `namespace absl { inline namespace lts_20240116 { namespace internal
+# { ... } } }`, so every `absl::internal::` reference becomes ambiguous and the
+# compile dies in absl/base/casts.h. Verified, not theorised: building this image
+# against the base-grpc-amd64 prebuilt fails at
+# sentencepiece-static/error.cc.o with "reference to 'internal' is ambiguous".
+#
+# Ubuntu Noble's apt protobuf is 3.21.12, which predates the abseil dependency,
+# so message_lite.h pulls in no abseil and the vendored copy is the only one in
+# scope. That is also the exact protobuf/gRPC pair every unit and end-to-end run
+# of this backend has been verified against. Keep it: a from-source gRPC here
+# does not buy a faster build, it buys a broken one.
+#
+# The install-base-deps path is additionally unsafe because it drops protoc 27.1
+# into /usr/local/bin, which shadows apt's protoc on PATH and would generate
+# protobuf-27 sources to be compiled against 3.21 headers.
+FROM ${BASE_IMAGE} AS builder
+ARG BUILD_TYPE
+ARG TARGETARCH
+ARG TARGETVARIANT
+ARG APT_MIRROR
+ARG APT_PORTS_MIRROR
+# Selects the CUDA architecture list in backend/cpp/audio-cpp/Makefile. It has
+# to be forwarded: upstream compiles engine_runtime for `native` when
+# CMAKE_CUDA_ARCHITECTURES is unset, and no CI runner has a GPU to enumerate.
+# The value is the same cuda-major-version the matrix entry declares.
+ARG CUDA_MAJOR_VERSION
+
+ENV BUILD_TYPE=${BUILD_TYPE} \
+    CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} \
+    APT_MIRROR=${APT_MIRROR} \
+    APT_PORTS_MIRROR=${APT_PORTS_MIRROR} \
+    DEBIAN_FRONTEND=noninteractive \
+    PATH=/usr/local/cuda/bin:${PATH}
+
+WORKDIR /build
+
+# gRPC/protobuf from apt, deliberately; see the block above. libgrpc++-dev ships
+# a CMake config so find_package(gRPC CONFIG) resolves, and libprotobuf-dev
+# lands in the layout CMake's FindProtobuf module expects, which matters because
+# sentencepiece runs a bare find_package(Protobuf REQUIRED) with no CONFIG
+# fallback of its own.
+#
+# BUILD_TYPE=vulkan additionally needs the loader headers and glslc; both are in
+# Noble. The CUDA toolkit for BUILD_TYPE=cublas comes from BASE_IMAGE.
+RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mirror \
+    sh /usr/local/sbin/apt-mirror && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends \
+        git cmake build-essential pkg-config ca-certificates \
+        libgrpc++-dev libprotobuf-dev protobuf-compiler protobuf-compiler-grpc && \
+    if [ "${BUILD_TYPE}" = "vulkan" ]; then \
+        apt-get install -y --no-install-recommends libvulkan-dev glslc; \
+    fi && \
+    if [ "${TARGETARCH}" = "arm64" ]; then \
+        apt-get install -y --no-install-recommends gcc-14 g++-14; \
+    fi && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+COPY . /LocalAI
+
+# gcc-14 on arm64, for the same reason llama-cpp does it in
+# .docker/llama-cpp-compile.sh: ggml's CPU_ALL_VARIANTS table includes armv9.2
+# variants built with -march=...+sme, and Noble's default gcc-13 rejects that
+# feature modifier outright ("invalid feature modifier 'sme'"). Every variant in
+# the table has to COMPILE even though a host only ever dlopens the one its own
+# CPU supports, so one unbuildable variant fails the whole image.
+#
+# ON EVERY arm64 BUILD_TYPE, which is where this differs from llama-cpp's script
+# and why that difference is spelled out rather than assumed. llama-cpp only
+# needs gcc-14 for its pure-CPU image because its GPU builds run
+# llama-cpp-fallback, which has no variant table at all. This backend's Makefile
+# sets ENGINE_ENABLE_CPU_ALL_VARIANTS for every non-Darwin build, GPU included,
+# so an arm64 GPU image would hit the identical compile error. Gating this on an
+# empty BUILD_TYPE would leave that trap armed for the first arm64 GPU entry
+# added to the matrix, which today has none.
+RUN --mount=type=cache,target=/root/.ccache,id=audio-cpp-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
+    if [ "${TARGETARCH}" = "arm64" ]; then \
+        export CC=gcc-14 CXX=g++-14; \
+    fi && \
+    make -C /LocalAI/backend/cpp/audio-cpp BUILD_TYPE=${BUILD_TYPE} \
+        CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} NATIVE=false grpc-server package
+
+# The package directory is the whole image: run.sh, grpc-server, the dlopened
+# ggml CPU variants, the bundled loader and its library closure, and the
+# bundled silero_vad / marblenet_vad assets. Nothing else exists at run time.
+FROM scratch
+COPY --from=builder /LocalAI/backend/cpp/audio-cpp/package/. ./

backend/Dockerfile.bonsai

@@ -0,0 +1,160 @@
+ARG BASE_IMAGE=ubuntu:24.04
+# BUILDER_BASE_IMAGE defaults to BASE_IMAGE so the Dockerfile parses even
+# when no prebuilt base is supplied. The builder-prebuilt stage is only
+# entered when BUILDER_TARGET=builder-prebuilt, so a "wrong" fallback
+# content here is harmless — BuildKit prunes the unreferenced builder.
+ARG BUILDER_BASE_IMAGE=${BASE_IMAGE}
+# BUILDER_TARGET selects which builder stage the final scratch image copies
+# package output from. Declared at global scope (before any FROM) so it's
+# usable in `FROM ${BUILDER_TARGET}` below. Default keeps local
+# `make backends/bonsai` on the from-source path.
+ARG BUILDER_TARGET=builder-fromsource
+ARG APT_MIRROR=""
+ARG APT_PORTS_MIRROR=""
+
+
+# ============================================================================
+# Stage: builder-fromsource — self-contained build path.
+# Runs .docker/install-base-deps.sh (apt deps + cmake + protoc + gRPC +
+# conditional CUDA/ROCm/Vulkan), copies /opt/grpc to /usr/local, then
+# compiles the variant. Used when BUILDER_TARGET=builder-fromsource (the
+# default; local `make backends/bonsai`).
+#
+# The install script is the same one that backend/Dockerfile.base-grpc-builder
+# runs, so the result is bit-equivalent to the prebuilt-base path
+# (builder-prebuilt below).
+# ============================================================================
+FROM ${BASE_IMAGE} AS builder-fromsource
+ARG BUILD_TYPE
+ARG CUDA_MAJOR_VERSION
+ARG CUDA_MINOR_VERSION
+ARG CMAKE_FROM_SOURCE=false
+# CUDA Toolkit 13.x compatibility: CMake 3.31.9+ fixes toolchain detection/arch table issues
+ARG CMAKE_VERSION=3.31.10
+ARG GRPC_VERSION=v1.65.0
+ARG GRPC_MAKEFLAGS="-j4 -Otarget"
+ARG SKIP_DRIVERS=false
+ARG TARGETARCH
+ARG TARGETVARIANT
+ARG GO_VERSION=1.25.4
+ARG UBUNTU_VERSION=2404
+ARG APT_MIRROR
+ARG APT_PORTS_MIRROR
+ARG AMDGPU_TARGETS=""
+ARG BACKEND=rerankers
+# CUDA target archs, e.g. --build-arg CUDA_DOCKER_ARCH='75;86;89;120'
+ARG CUDA_DOCKER_ARCH
+ARG CMAKE_ARGS
+
+ENV BUILD_TYPE=${BUILD_TYPE} \
+    CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} \
+    CUDA_MINOR_VERSION=${CUDA_MINOR_VERSION} \
+    CMAKE_FROM_SOURCE=${CMAKE_FROM_SOURCE} \
+    CMAKE_VERSION=${CMAKE_VERSION} \
+    GRPC_VERSION=${GRPC_VERSION} \
+    GRPC_MAKEFLAGS=${GRPC_MAKEFLAGS} \
+    SKIP_DRIVERS=${SKIP_DRIVERS} \
+    TARGETARCH=${TARGETARCH} \
+    UBUNTU_VERSION=${UBUNTU_VERSION} \
+    APT_MIRROR=${APT_MIRROR} \
+    APT_PORTS_MIRROR=${APT_PORTS_MIRROR} \
+    AMDGPU_TARGETS=${AMDGPU_TARGETS} \
+    CUDA_DOCKER_ARCH=${CUDA_DOCKER_ARCH} \
+    CMAKE_ARGS=${CMAKE_ARGS} \
+    DEBIAN_FRONTEND=noninteractive
+
+# CUDA on PATH (no-op when CUDA isn't installed)
+ENV PATH=/usr/local/cuda/bin:${PATH}
+# HipBLAS / ROCm on PATH (no-op when ROCm isn't installed)
+ENV PATH=/opt/rocm/bin:${PATH}
+
+WORKDIR /build
+
+# Install everything via the shared script — the same one that
+# backend/Dockerfile.base-grpc-builder runs, so the prebuilt CI base and
+# this from-source path are bit-equivalent.
+RUN --mount=type=bind,source=.docker/install-base-deps.sh,target=/usr/local/sbin/install-base-deps \
+    --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mirror \
+    bash /usr/local/sbin/install-base-deps
+
+# Mirror builder-prebuilt: copy gRPC from /opt/grpc to /usr/local so
+# CMake's find_package finds it at the canonical prefix the Makefile expects.
+RUN cp -a /opt/grpc/. /usr/local/
+
+COPY . /LocalAI
+
+# BuildKit cache mount for ccache. See Dockerfile.llama-cpp (commit 9228e5b4)
+# for rationale. bonsai is a llama.cpp fork that reuses
+# backend/cpp/llama-cpp source via a thin wrapper Makefile, so MOST TUs
+# are content-identical to the upstream llama-cpp build. Sharing a cache
+# id with llama-cpp could give cross-fork hits — but for now keep them
+# separate so a regression in one doesn't poison the other. Revisit
+# sharing after measuring the actual hit rate.
+#
+# The compile body is shared with builder-prebuilt via .docker/bonsai-compile.sh.
+RUN --mount=type=bind,source=.docker/bonsai-compile.sh,target=/usr/local/sbin/compile.sh \
+    --mount=type=cache,target=/root/.ccache,id=bonsai-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
+    bash /usr/local/sbin/compile.sh
+
+
+# Copy libraries using a script to handle architecture differences
+RUN make -BC /LocalAI/backend/cpp/bonsai package
+
+
+# ============================================================================
+# Stage: builder-prebuilt — uses the pre-built base from
+# quay.io/go-skynet/ci-cache:base-grpc-* (built by .github/workflows/base-images.yml).
+# That image already has gRPC at /opt/grpc + apt deps + CUDA/ROCm/Vulkan
+# pre-installed, so we just copy gRPC to /usr/local and compile. Used when
+# BUILDER_TARGET=builder-prebuilt (CI when the matrix entry sets
+# builder-base-image).
+# ============================================================================
+FROM ${BUILDER_BASE_IMAGE} AS builder-prebuilt
+
+ARG BUILD_TYPE
+ENV BUILD_TYPE=${BUILD_TYPE}
+ARG CUDA_DOCKER_ARCH
+ENV CUDA_DOCKER_ARCH=${CUDA_DOCKER_ARCH}
+ARG CMAKE_ARGS
+ENV CMAKE_ARGS=${CMAKE_ARGS}
+# AMDGPU_TARGETS must be forwarded into the env here too — backend/cpp/llama-cpp/Makefile
+# (which the bonsai Makefile reuses via a sibling build dir) errors out when the var
+# is empty on a hipblas build, and the prebuilt path is what CI exercises most of the
+# time. The builder-fromsource stage above already does this; mirror it here.
+ARG AMDGPU_TARGETS
+ENV AMDGPU_TARGETS=${AMDGPU_TARGETS}
+ARG TARGETARCH
+ARG TARGETVARIANT
+
+# The base-grpc-* image installs gRPC to /opt/grpc but doesn't copy it to
+# /usr/local. Mirror what the from-source path does so the compile step
+# can find gRPC at the canonical prefix the Makefile expects.
+RUN cp -a /opt/grpc/. /usr/local/
+
+COPY . /LocalAI
+
+RUN --mount=type=bind,source=.docker/bonsai-compile.sh,target=/usr/local/sbin/compile.sh \
+    --mount=type=cache,target=/root/.ccache,id=bonsai-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
+    bash /usr/local/sbin/compile.sh
+
+RUN make -BC /LocalAI/backend/cpp/bonsai package
+
+
+# ============================================================================
+# Final stage — copies package output from one of the two builders.
+# BUILDER_TARGET selects which one. BuildKit prunes the unreferenced builder.
+#
+# BuildKit doesn't support variable expansion in `COPY --from=` directly,
+# so we resolve the ARG by aliasing the chosen builder to a fixed stage
+# name via `FROM ${BUILDER_TARGET} AS builder` and then COPY --from=builder.
+# BUILDER_TARGET itself is declared as a global ARG at the top of this
+# file (required for use in FROM), so we just re-import it into this
+# stage's scope before the FROM directive.
+# ============================================================================
+FROM ${BUILDER_TARGET} AS builder
+
+FROM scratch
+
+
+# Copy all available binaries (the build process only creates the appropriate ones for the target architecture)
+COPY --from=builder /LocalAI/backend/cpp/bonsai/package/. ./

backend/Dockerfile.golang

@@ -137,7 +137,7 @@ RUN <<EOT bash
             libcusolver-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}
         if [ "${CUDA_MAJOR_VERSION}" = "13" ] && [ "arm64" = "$TARGETARCH" ]; then
             apt-get install -y --no-install-recommends \
-            libcufile-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libcudnn9-cuda-${CUDA_MAJOR_VERSION} cuda-cupti-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libnvjitlink-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}
+            libcufile-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libcudnn9-cuda-${CUDA_MAJOR_VERSION} libcudnn9-dev-cuda-${CUDA_MAJOR_VERSION} cuda-cupti-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libnvjitlink-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}
         fi
         apt-get clean && \
         rm -rf /var/lib/apt/lists/*
@@ -221,10 +221,75 @@ RUN if [ "${BACKEND}" = "crispasr" ]; then \
     apt-get clean && rm -rf /var/lib/apt/lists/*; \
 fi
 
-COPY . /LocalAI
+# sherpa-onnx links onnxruntime's CUDA execution provider, and
+# libonnxruntime_providers_cuda.so has cuDNN as a hard DT_NEEDED. The
+# onnxruntime GPU tarball does not ship cuDNN itself, so without this the
+# builder has none (the arm64 + CUDA 13 branch above is the only other place
+# that installs it) and package-gpu-libs.sh correctly refuses to produce a
+# package that references cuDNN with no cuDNN available to it.
+#
+# Installed per-backend rather than for every cublas build: the auto-detection
+# in package-gpu-libs.sh bundles only what a package actually references, so
+# the ggml backends would not grow either way, but they would all pay ~1.1 GB
+# of builder layer and registry cache for a library they never call.
+#
+# Runtime package only, no -dev: sherpa-onnx consumes onnxruntime's prebuilt
+# CUDA provider and never compiles against cuDNN headers. libcudnn9-cuda-N
+# carries the dispatcher plus all seven dlopen()ed sublibraries, which is what
+# complete_cudnn_family needs to assemble a whole bundle.
+RUN <<EOT bash
+    if [ "${BACKEND}" = "sherpa-onnx" ] && [ "${BUILD_TYPE}" = "cublas" ] && [ "${SKIP_DRIVERS}" = "false" ]; then
+        apt-get update && \
+        apt-get install -y --no-install-recommends \
+            libcudnn9-cuda-${CUDA_MAJOR_VERSION} && \
+        ldconfig && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists/*
+    fi
+EOT
 
 RUN git config --global --add safe.directory /LocalAI
 
+# Prebuild the native engine from a layer that depends on this backend's own
+# directory and nothing else.
+#
+# The expensive part of a C++ backend build is the engine: each of these
+# Makefiles clones an upstream repo at a pinned SHA and compiles it once per
+# SIMD variant (depth-anything-cpp builds four: avx, avx2, avx512, fallback),
+# and those variant targets depend only on the clone. They cannot observe a
+# change anywhere else in the LocalAI tree. Building them below `COPY . /LocalAI`
+# threw that away: any Go-side edit invalidated the layer and recompiled C++ that
+# had not changed. Measured on 2026-07-30, that is a 100+ minute rebuild for the
+# larger engines.
+#
+# Copying only this backend's directory first keeps the compile in a layer that
+# survives any change elsewhere in the tree, so `cache-from: type=registry`
+# restores it. That covers the expensive cases directly: a shared-build-input or
+# backend.proto change, the weekly full-matrix cron and a tag push all rebuild
+# every backend while touching none of their directories. This is the mechanism
+# behind base-grpc-* applied one level down; unlike a --mount=type=cache it is a
+# real layer, which is what actually survives to the registry.
+#
+# The whole directory rather than just the Makefile: the CMake targets also need
+# CMakeLists.txt, and the file list differs per backend. The cost is that editing
+# this backend's Go sources also invalidates the engine layer.
+#
+# Backends whose Makefile has no `engine` target are unaffected: the guard skips
+# the prebuild and their engine still compiles in the `build` step below.
+COPY backend/go/${BACKEND}/ /LocalAI/backend/go/${BACKEND}/
+RUN cd /LocalAI/backend/go/${BACKEND} && \
+    if make -n engine >/dev/null 2>&1; then \
+        echo "==> prebuilding engine for ${BACKEND} (cacheable layer)" && \
+        make engine; \
+    else \
+        echo "==> ${BACKEND} has no engine target; it builds with the backend"; \
+    fi
+
+COPY . /LocalAI
+
+# The engine variants built above survive this COPY (they are build outputs, not
+# tracked files) and are newer than the pinned clone, so make treats them as up
+# to date and goes straight to the Go binary.
 RUN cd /LocalAI && make protogen-go && make -C /LocalAI/backend/go/${BACKEND} build
 
 FROM scratch

backend/Dockerfile.llama-cpp

@@ -111,6 +111,10 @@ RUN make -BC /LocalAI/backend/cpp/llama-cpp package
 # ============================================================================
 FROM ${BUILDER_BASE_IMAGE} AS builder-prebuilt
 
+ARG APT_MIRROR
+ENV APT_MIRROR=${APT_MIRROR}
+ARG APT_PORTS_MIRROR
+ENV APT_PORTS_MIRROR=${APT_PORTS_MIRROR}
 ARG BUILD_TYPE
 ENV BUILD_TYPE=${BUILD_TYPE}
 ARG CUDA_DOCKER_ARCH

backend/Dockerfile.python

@@ -224,7 +224,11 @@ ARG DEPS_REFRESH=initial
 
 RUN cd /${BACKEND} && PORTABLE_PYTHON=true make
 
-# Package GPU libraries into the backend's lib directory
+# Package GPU libraries into the backend's lib directory.
+#
+# Must stay after the venv is built above: package-gpu-libs.sh inspects
+# /${BACKEND}/venv to decide whether this backend already carries a complete
+# cuDNN from pip, and bundles one only when it does not (issue #10905).
 RUN mkdir -p /${BACKEND}/lib && \
     TARGET_LIB_DIR="/${BACKEND}/lib" BUILD_TYPE="${BUILD_TYPE}" CUDA_MAJOR_VERSION="${CUDA_MAJOR_VERSION}" \
     bash /package-gpu-libs.sh "/${BACKEND}/lib"

backend/README.md

@@ -46,6 +46,7 @@ The backend system provides language-specific Dockerfiles that handle the build
 - **vllm**: High-performance LLM inference
 - **mlx**: Apple Silicon optimization
 - **diffusers**: Stable Diffusion models
+- **longcat-video**: CUDA text/image-to-video and speech-driven avatar generation
 - **Audio**: coqui, faster-whisper, kitten-tts
 - **Vision**: mlx-vlm, rfdetr
 - **Specialized**: rerankers, chatterbox, kokoro
@@ -55,6 +56,7 @@ The backend system provides language-specific Dockerfiles that handle the build
 - **stablediffusion-ggml**: Stable Diffusion in Go with GGML Cpp backend
 - **piper**: Text-to-speech synthesis Golang with C bindings using rhaspy/piper
 - **local-store**: Vector storage backend
+- **valkey-store**: Durable vector storage backend backed by Valkey Search (FT.*)
 
 #### C++ Backends (`cpp/`)
 - **llama-cpp**: Llama.cpp integration

backend/backend.proto

@@ -16,12 +16,26 @@ service Backend {
   rpc Embedding(PredictOptions) returns (EmbeddingResult) {}
   rpc GenerateImage(GenerateImageRequest) returns (Result) {}
   rpc GenerateVideo(GenerateVideoRequest) returns (Result) {}
+  rpc Generate3D(Generate3DRequest) returns (Result) {}
   rpc AudioTranscription(TranscriptRequest) returns (TranscriptResult) {}
   rpc AudioTranscriptionStream(TranscriptRequest) returns (stream TranscriptStreamResponse) {}
+  // AudioTranscriptionLive is the bidirectional live-microphone ASR RPC. The
+  // first message MUST carry a Config; subsequent messages carry Audio frames
+  // (mono float PCM at config.sample_rate, 16 kHz default). After a
+  // successful open the backend replies with a single ready ack
+  // (TranscriptLiveResponse{ready:true}); backends or models without
+  // cache-aware streaming support return UNIMPLEMENTED instead. Newly
+  // finalized text streams back as deltas; eou=true marks the model's
+  // end-of-utterance token. One stream spans many utterances (the decoder
+  // resets itself after each EOU). Closing the send side finalizes: the
+  // backend flushes the decoder tail and emits a terminal message carrying
+  // final_result. A second Config mid-stream resets the decode session.
+  rpc AudioTranscriptionLive(stream TranscriptLiveRequest) returns (stream TranscriptLiveResponse) {}
   rpc TTS(TTSRequest) returns (Result) {}
   rpc TTSStream(TTSRequest) returns (stream Reply) {}
   rpc SoundGeneration(SoundGenerationRequest) returns (Result) {}
   rpc TokenizeString(PredictOptions) returns (TokenizationResponse) {}
+  rpc Detokenize(DetokenizeRequest) returns (DetokenizeResponse) {}
   rpc Status(HealthMessage) returns (StatusResponse) {}
   rpc Detect(DetectOptions) returns (DetectResponse) {}
   // SoundDetection runs an audio-tagging / sound-event-classification model
@@ -124,6 +138,10 @@ message MetricsResponse {
 message TokenClassifyRequest {
   string text = 1;
   float threshold = 2;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 3;
 }
 
 // TokenClassifyEntity is one detected entity span. Byte offsets are
@@ -161,6 +179,17 @@ message ScoreRequest {
   // candidates differ in length and the consumer wants a per-token
   // measure comparable across them (PMI-style scoring).
   bool length_normalize = 4;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 5;
+  // Byte length of the prompt prefix that stays identical across
+  // repeated scoring calls (e.g. a classifier's option-list system
+  // prompt — everything before the per-turn probe text). Backends that
+  // snapshot state (hybrid/recurrent models cannot rewind otherwise)
+  // use it to place a reuse point exactly at the boundary, so the next
+  // call re-processes only the tokens after it. 0 means unknown.
+  int32 stable_prefix_len = 6;
 }
 
 // CandidateScore is one row in the ScoreResponse, matching by index
@@ -192,6 +221,10 @@ message RerankRequest {
   string query = 1;
   repeated string documents = 2;
   int32 top_n = 3;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 4;
 }
 
 message RerankResult {
@@ -303,6 +336,39 @@ message PredictOptions {
   int32 TopLogprobs = 51;  // Number of top logprobs to return per token (maps to OpenAI top_logprobs parameter)
   map<string, string> Metadata = 52;  // Generic per-request metadata (e.g., enable_thinking)
   float MinP = 53;  // Minimum probability sampling threshold (0.0 = disabled)
+
+  // ModelIdentity names the model this request is for, so a backend can reject
+  // a request that reached it by mistake instead of answering from whatever
+  // model it happens to hold. In distributed mode a worker can recycle a
+  // stopped backend's gRPC port for a different model's backend, and a
+  // liveness-only health probe cannot tell that apart from a valid cached
+  // route (#10952).
+  //
+  // The value is the controller's ModelConfig.Model, the SAME expression that
+  // produces ModelOptions.Model at LoadModel time, so the two are equal by
+  // construction rather than by convention.
+  //
+  // Empty means "no identity supplied": backends MUST skip the check. That
+  // keeps an old controller talking to a new backend working, and covers
+  // callers that legitimately synthesize a PredictOptions internally.
+  //
+  // Do NOT reuse TTSRequest.model or SoundGenerationRequest.model for this
+  // purpose. FileStagingClient already rewrites those to worker-local absolute
+  // paths (core/services/nodes/file_staging_client.go), so in distributed mode
+  // they already differ from the load-time value and comparing them would
+  // reject valid requests. Extending identity to those RPCs needs a separate
+  // field carrying the untranslated value - which is exactly what
+  // TTSRequest.ModelIdentity and SoundGenerationRequest.ModelIdentity are.
+  //
+  // Every other request message that reaches a backend through the distributed
+  // router now carries the same ModelIdentity field, populated from the same
+  // ModelConfig.Model. FileStagingClient rewrites Src/Dst/Voice/Model/
+  // StartImage/EndImage/Audio and never ModelIdentity, so what the backend
+  // compares is always what the controller sent.
+  string ModelIdentity = 54;
+
+  // 24 was never assigned; reserve it so it is not silently reused.
+  reserved 24;
 }
 
 // ToolCallDelta represents an incremental tool call update from the C++ parser.
@@ -436,6 +502,11 @@ message ModelOptions {
   // Proxy carries the cloud-proxy backend's per-model configuration.
   // Empty for non-proxy backends.
   ProxyOptions Proxy = 74;
+
+  // EnableScore reserves backend resources for the Score RPC. It is derived
+  // from the model's explicit `known_usecases: [score]` declaration so models
+  // that never score retain their ordinary serving footprint.
+  bool EnableScore = 75;
 }
 
 // ProxyOptions configures the cloud-proxy backend. UpstreamURL and
@@ -451,6 +522,12 @@ message ProxyOptions {
   string api_key_file = 5;
   string upstream_model = 6;
   int32 request_timeout_seconds = 7;
+  // cache_prompt enables automatic Anthropic prompt-cache breakpoints
+  // (cache_control: ephemeral) on the stable prefix — system, tools, and
+  // the last message block — when translating to the Anthropic provider.
+  // Cuts input cost on repeated/agentic calls (cache read = 0.1x). Only
+  // meaningful for mode=translate + provider=anthropic; ignored otherwise.
+  bool cache_prompt = 8;
 }
 
 message Result {
@@ -472,6 +549,10 @@ message TranscriptRequest {
   float temperature = 8;
   repeated string timestamp_granularities = 9;
   bool stream = 10;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 11;
 }
 
 message TranscriptResult {
@@ -479,6 +560,10 @@ message TranscriptResult {
   string text = 2;
   string language = 3;
   float duration = 4;
+  // True when the decode ended on the model's end-of-utterance special token
+  // (<EOU>/<EOB>, emitted by cache-aware streaming models such as
+  // parakeet_realtime_eou_120m-v1). The marker itself is stripped from text.
+  bool eou = 5;
 }
 
 message TranscriptStreamResponse {
@@ -486,6 +571,34 @@ message TranscriptStreamResponse {
   TranscriptResult final_result = 2;
 }
 
+// === AudioTranscriptionLive messages =====================================
+
+message TranscriptLiveRequest {
+  oneof payload {
+    TranscriptLiveConfig config = 1;
+    TranscriptLiveAudio  audio  = 2;
+  }
+}
+
+message TranscriptLiveConfig {
+  string language = 1;             // "" => model default
+  int32 sample_rate = 2;           // 0 => 16000; backends may reject others
+  map<string, string> params = 3;  // backend-specific tuning
+}
+
+message TranscriptLiveAudio {
+  repeated float pcm = 1;          // mono PCM in [-1,1] at config.sample_rate
+}
+
+message TranscriptLiveResponse {
+  bool ready = 1;                       // open ack: sent once, before any delta
+  string delta = 2;                     // newly-finalized text since previous response
+  bool eou = 3;                         // <EOU> fired during this feed (the user yielded the turn)
+  repeated TranscriptWord words = 4;    // words finalized by this feed (stream-relative ns)
+  TranscriptResult final_result = 5;    // terminal message only, after the send side closes
+  bool eob = 6;                         // <EOB> fired: a backchannel ("uh-huh") ended — NOT a turn boundary
+}
+
 message TranscriptWord {
   int64 start = 1;
   int64 end = 2;
@@ -518,6 +631,10 @@ message GenerateImageRequest {
 
   // Reference images for models that support them (e.g., Flux Kontext)
   repeated string ref_images = 12;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 13;
 }
 
 message GenerateVideoRequest {
@@ -533,6 +650,28 @@ message GenerateVideoRequest {
   float cfg_scale = 10;    // Classifier-free guidance scale
   int32 step = 11;         // Number of inference steps
   string dst = 12;        // Output path for the generated video
+  string audio = 13;      // Path to staged audio for audio-conditioned video
+  // Backend-specific per-request generation parameters. Values are strings
+  // and are validated/coerced by the selected backend.
+  map<string, string> params = 14;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 15;
+}
+
+message Generate3DRequest {
+  string src = 1;          // Path to the staged conditioning image (3D generation is image-conditioned)
+  string dst = 2;          // Output path for the generated binary glTF (.glb) asset
+  int32 seed = 3;          // <=0 lets the backend pick a random seed
+  int32 step = 4;          // Flow sampling steps; <=0 uses the backend default
+  float cfg_scale = 5;     // Classifier-free guidance scale; <=0 uses the backend default
+  int32 texture_steps = 6; // Texture flow sampling steps; <=0 uses the backend default
+  string quality = 7;      // Mesh pipeline: ""|"auto"|"coarse"|"512"|"1024"
+  string background = 8;   // Conditioning-image background handling: ""|"auto"|"keep"|"black"|"white"
+  // Backend-specific per-request generation parameters. Values are strings
+  // and are validated/coerced by the selected backend.
+  map<string, string> params = 9;
 }
 
 message TTSRequest {
@@ -550,10 +689,26 @@ message TTSRequest {
   // (e.g. Chatterbox exaggeration/cfg_weight/temperature). Values are strings and
   // coerced by the backend; unset leaves the backend's configured defaults.
   map<string, string> params = 7;
+  // ModelIdentity is a SEPARATE field from `model` above and carries the
+  // UNTRANSLATED controller-side ModelConfig.Model, so a backend can reject a
+  // request that reached it through a stale distributed route (#10952).
+  //
+  // `model` cannot be reused for this: FileStagingClient.TTS/.TTSStream and the
+  // SoundGeneration path rewrite it into a worker-local absolute path
+  // (core/services/nodes/file_staging_client.go), while the load-time value is
+  // untranslated. In distributed mode - exactly the configuration this guards -
+  // the two already differ, so comparing them would reject valid requests.
+  //
+  // Empty means "no identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 8;
 }
 
 message VADRequest {
   repeated float audio = 1;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 2;
 }
 
 message VADSegment {
@@ -585,6 +740,10 @@ message DiarizeRequest {
   float  min_duration_on = 8;          // discard segments shorter than this (seconds); 0 = backend default
   float  min_duration_off = 9;         // merge gaps shorter than this (seconds); 0 = backend default
   bool   include_text = 10;            // when the backend can emit per-segment transcript for free, ask it to populate `text`
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 11;
 }
 
 message DiarizeSegment {
@@ -619,6 +778,18 @@ message SoundGenerationRequest {
   optional string language = 14;
   optional string timesignature = 15;
   optional bool instrumental = 17;
+  // ModelIdentity is a SEPARATE field from `model` above and carries the
+  // UNTRANSLATED controller-side ModelConfig.Model, so a backend can reject a
+  // request that reached it through a stale distributed route (#10952).
+  //
+  // `model` cannot be reused for this: FileStagingClient.TTS/.TTSStream and the
+  // SoundGeneration path rewrite it into a worker-local absolute path
+  // (core/services/nodes/file_staging_client.go), while the load-time value is
+  // untranslated. In distributed mode - exactly the configuration this guards -
+  // the two already differ, so comparing them would reject valid requests.
+  //
+  // Empty means "no identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 18;
 }
 
 message TokenizationResponse {
@@ -626,6 +797,14 @@ message TokenizationResponse {
   repeated int32 tokens = 2;
 }
 
+message DetokenizeRequest {
+  repeated int32 tokens = 1;
+}
+
+message DetokenizeResponse {
+  string content = 1;
+}
+
 message MemoryUsageData {
   uint64 total = 1;
   map<string, uint64> breakdown = 2;
@@ -658,6 +837,10 @@ message DetectOptions {
   repeated float points = 3;   // Point coordinates as [x1, y1, label1, x2, y2, label2, ...] (label: 1=pos, 0=neg)
   repeated float boxes = 4;    // Box coordinates as [x1, y1, x2, y2, ...]
   float threshold = 5;         // Detection confidence threshold
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 6;
 }
 
 message Detection {
@@ -680,6 +863,10 @@ message SoundDetectionRequest {
   string src = 1;       // audio file path (LocalAI writes the upload to disk)
   int32 top_k = 2;      // number of top tags to return (0 = all classes)
   float threshold = 3;  // optional: drop tags scoring below this
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 4;
 }
 
 message SoundClass {
@@ -704,6 +891,10 @@ message DepthRequest {
   bool include_points = 7;         // back-project to a 3D point cloud (DualDPT)
   float points_conf_thresh = 8;    // keep points with confidence >= this threshold
   repeated string exports = 9;     // requested exports: "glb", "colmap"
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 10;
 }
 
 message DepthResponse {
@@ -735,6 +926,10 @@ message FaceVerifyRequest {
   string img2 = 2;              // base64-encoded image
   float  threshold = 3;         // cosine-distance threshold; 0 = use backend default
   bool   anti_spoofing = 4;     // run MiniFASNet liveness on each image; failed liveness forces verified=false
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 5;
 }
 
 message FaceVerifyResponse {
@@ -756,6 +951,10 @@ message FaceAnalyzeRequest {
   string          img = 1;          // base64-encoded image
   repeated string actions = 2;      // subset of ["age","gender","emotion","race"]; empty = all-supported
   bool            anti_spoofing = 3;
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 4;
 }
 
 message FaceAnalysis {
@@ -788,6 +987,10 @@ message VoiceVerifyRequest {
   string audio2 = 2;            // path to second audio clip
   float  threshold = 3;         // cosine-distance threshold; 0 = use backend default
   bool   anti_spoofing = 4;     // reserved for future AASIST bolt-on
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 5;
 }
 
 message VoiceVerifyResponse {
@@ -802,6 +1005,10 @@ message VoiceVerifyResponse {
 message VoiceAnalyzeRequest {
   string          audio = 1;        // path to audio clip
   repeated string actions = 2;      // subset of ["age","gender","emotion"]; empty = all-supported
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 3;
 }
 
 message VoiceAnalysis {
@@ -820,6 +1027,10 @@ message VoiceAnalyzeResponse {
 
 message VoiceEmbedRequest {
   string audio = 1;              // path to audio clip
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 2;
 }
 
 message VoiceEmbedResponse {
@@ -914,6 +1125,18 @@ message AudioTransformRequest {
   string reference_path = 2;         // optional auxiliary; empty => zero-fill
   string dst = 3;                    // required, output file path
   map<string, string> params = 4;    // backend-specific tuning
+  // ModelIdentity names the model this request is for; see
+  // PredictOptions.ModelIdentity for the full rationale. Empty means "no
+  // identity supplied" and backends MUST skip the check.
+  string ModelIdentity = 5;
+}
+
+// One named output of a transform that produces several from a single run.
+// Source separation is the case that needs it: htdemucs yields drums, bass,
+// other and vocals from one pass over the input.
+message AudioTransformStem {
+  string name = 1;                   // the model's own stem id, e.g. "vocals"
+  string dst = 2;                    // path of the file written for that stem
 }
 
 message AudioTransformResult {
@@ -921,6 +1144,15 @@ message AudioTransformResult {
   int32  sample_rate = 2;
   int32  samples = 3;
   bool   reference_provided = 4;
+  // Every named output the run produced, in the model's own order, including
+  // the one copied into dst. Empty for a transform with a single output.
+  //
+  // It exists because dst carries one file while separation produces several,
+  // and running the model once per stem would cost four full separations of
+  // the same audio. The backend runs once, writes each stem beside dst, and
+  // names them here; without this field the other stems are on disk but no
+  // caller can find them, which is the same as not having produced them.
+  repeated AudioTransformStem stems = 5;
 }
 
 // Bidirectional streaming audio transform. The first message MUST carry a
@@ -1212,4 +1444,3 @@ message ForwardReply {
   repeated ForwardHeader headers = 2;
   bytes body_chunk = 3;
 }
-

backend/cpp/audio-cpp/.gitignore

@@ -0,0 +1,8 @@
+audio.cpp/
+build/
+package/
+grpc-server
+backend.pb.cc
+backend.pb.h
+backend.grpc.pb.cc
+backend.grpc.pb.h

backend/cpp/audio-cpp/CMakeLists.txt

@@ -0,0 +1,331 @@
+cmake_minimum_required(VERSION 3.20)
+project(audio-cpp-grpc-server LANGUAGES C CXX)
+
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(TARGET grpc-server)
+
+set(AUDIO_CPP_DIR "${CMAKE_CURRENT_SOURCE_DIR}/audio.cpp"
+    CACHE PATH "Path to the pinned audio.cpp checkout")
+option(AUDIO_CPP_GRPC_BUILD_TESTS "Build engine-linked ctest binaries" OFF)
+
+if(NOT EXISTS "${AUDIO_CPP_DIR}/CMakeLists.txt")
+    message(FATAL_ERROR
+        "AUDIO_CPP_DIR does not contain an audio.cpp checkout: ${AUDIO_CPP_DIR}. "
+        "Run 'make audio.cpp' first.")
+endif()
+
+if(APPLE)
+    # Homebrew installs protobuf/grpc under a non-default prefix.
+    if(CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "arm64")
+        set(HOMEBREW_DEFAULT_PREFIX "/opt/homebrew")
+    else()
+        set(HOMEBREW_DEFAULT_PREFIX "/usr/local")
+    endif()
+    link_directories("${HOMEBREW_DEFAULT_PREFIX}/lib")
+    include_directories("${HOMEBREW_DEFAULT_PREFIX}/include")
+endif()
+
+find_package(Threads REQUIRED)
+find_package(Protobuf CONFIG QUIET)
+if(NOT Protobuf_FOUND)
+    find_package(Protobuf REQUIRED)
+endif()
+find_package(gRPC CONFIG QUIET)
+if(NOT gRPC_FOUND)
+    # Reached only on distros whose grpc++ packaging ships no CMake config.
+    # Ubuntu's libgrpc-dev does ship one, so this is dead code on LocalAI's own
+    # build distro. Kept for the distros that do not.
+    find_library(GRPCPP_LIB grpc++ REQUIRED)
+    find_library(GRPCPP_REFLECTION_LIB grpc++_reflection REQUIRED)
+    add_library(gRPC::grpc++ INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++ PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_LIB}")
+    add_library(gRPC::grpc++_reflection INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++_reflection PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_REFLECTION_LIB}")
+endif()
+
+find_program(_PROTOC NAMES protoc REQUIRED)
+find_program(_GRPC_CPP_PLUGIN NAMES grpc_cpp_plugin REQUIRED)
+
+get_filename_component(HW_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/../../backend.proto" ABSOLUTE)
+get_filename_component(HW_PROTO_PATH "${HW_PROTO}" PATH)
+
+set(HW_PROTO_SRCS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.cc")
+set(HW_PROTO_HDRS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.h")
+set(HW_GRPC_SRCS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.cc")
+set(HW_GRPC_HDRS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.h")
+
+add_custom_command(
+    OUTPUT "${HW_PROTO_SRCS}" "${HW_PROTO_HDRS}" "${HW_GRPC_SRCS}" "${HW_GRPC_HDRS}"
+    COMMAND ${_PROTOC}
+    ARGS --grpc_out "${CMAKE_CURRENT_BINARY_DIR}"
+         --cpp_out  "${CMAKE_CURRENT_BINARY_DIR}"
+         -I "${HW_PROTO_PATH}"
+         --plugin=protoc-gen-grpc="${_GRPC_CPP_PLUGIN}"
+         "${HW_PROTO}"
+    DEPENDS "${HW_PROTO}")
+
+add_library(hw_grpc_proto STATIC
+    ${HW_GRPC_SRCS} ${HW_GRPC_HDRS}
+    ${HW_PROTO_SRCS} ${HW_PROTO_HDRS})
+target_include_directories(hw_grpc_proto PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
+# Required on macOS: without these the Homebrew protobuf/grpc include dirs never
+# reach this target and google/protobuf/runtime_version.h is not found.
+target_link_libraries(hw_grpc_proto PUBLIC protobuf::libprotobuf gRPC::grpc++)
+
+# TWO PROTOBUF RUNTIMES IN ONE BINARY, AND THE ONE THAT WON WAS THE WRONG ONE.
+#
+# engine_runtime links sentencepiece, whose default SPM_PROTOBUF_PROVIDER
+# ("internal") builds the protobuf-lite 3.14.0 sources vendored under
+# external/sentencepiece/third_party/protobuf-lite. Our generated backend.pb.cc
+# is compiled against the toolchain's protobuf 3.21.12 headers and links
+# libprotobuf.so 3.21.12. Both used to end up in the executable: 476
+# google::protobuf:: symbols from that archive, 278 of them also defined by
+# libprotobuf.so.
+#
+# The binding is decided at STATIC LINK time. Once ld pulls a sentencepiece
+# member in for sentencepiece's own code, that member's protobuf definitions are
+# in the executable and references from libhw_grpc_proto.a bind to them. Do NOT
+# reach for -Wl,--exclude-libs: it flips those symbols to LOCAL in .dynsym and
+# the breakage is unchanged, because no visibility flag revisits a static
+# binding already made.
+#
+# What broke, measured rather than assumed:
+# google::protobuf::internal::ParseContext::ParseMessage(MessageLite*, const char*)
+# is what every generated _InternalParse calls for a SUBMESSAGE field and for
+# nothing else. Bound to the 3.14 definition it fails, so a flat message parsed
+# and every nested one did not: a TranscriptResult carrying segments serialized
+# to correct bytes that the same process could not read back, and
+# TranscriptLiveRequest, a oneof of submessages, could not have been parsed at
+# all. 3.21 generated code was also running 3.14 arena, ArenaStringPtr and
+# ExtensionSet code, which is an ABI mismatch rather than a missing feature, so
+# "not observed to bite yet" was never a reason to leave it.
+#
+# "package" makes sentencepiece use the protobuf found above, which is the one
+# the generated code was built against. It must be set before add_subdirectory,
+# since that is when sentencepiece's own cache entry is created.
+#
+# WHAT THAT BUYS IS ONE PROTOBUF RUNTIME, not an executable free of protobuf
+# symbols, and the difference matters to whoever checks this next. Measured with
+# nm -C --defined-only on the linked grpc-server, 2515 google::protobuf::
+# symbols are still DEFINED in it, and that is what should be there: they are
+# generated code, sentencepiece::ModelProto's own _InternalParse and
+# CheckTypeAndMergeFrom among them, which name protobuf types in their
+# signatures and are compiled into every user of a .proto. Expecting zero would
+# send a reader looking for a regression that is not one.
+#
+# The claim that decides whether the ABI mismatch above is gone is the RUNTIME
+# one, and it holds: google::protobuf::internal::ParseContext::ParseMessage is
+# UNDEFINED in the executable, so every generated _InternalParse resolves it to
+# libprotobuf.so at load instead of to a vendored 3.14 copy. No vendored
+# protobuf-lite archive is pulled in at all, and citrinet_asr, which parses a
+# SentencePiece ModelProto at load, tokenizes correctly as a result.
+set(SPM_PROTOBUF_PROVIDER "package" CACHE STRING
+    "Make sentencepiece use the found protobuf, not its vendored 3.14 copy" FORCE)
+
+# Upstream's global add_compile_options(-Wall -Wextra -Wpedantic -pedantic-errors)
+# is a directory property of the subdirectory and does not reach our targets.
+#
+# EXCLUDE_FROM_ALL is load-bearing, do not drop it: upstream's default target set
+# includes its CLI, server, converter and test binaries, none of which we ship.
+# Without it every build would compile all of them. The targets we do name in
+# target_link_libraries below are still built on demand, so nothing is lost.
+add_subdirectory("${AUDIO_CPP_DIR}" "${CMAKE_CURRENT_BINARY_DIR}/audio-cpp" EXCLUDE_FROM_ALL)
+
+add_executable(${TARGET}
+    grpc-server.cpp
+    model_options.cpp
+    capability_routing.cpp
+    family_gate.cpp
+    loaded_model.cpp
+    audio_io.cpp
+    audio_units.cpp
+    transcript_assembly.cpp
+    result_map.cpp
+    stem_selection.cpp
+    generation_request.cpp
+    stream_delta.cpp
+    wav_header.cpp
+    inference_lane.cpp
+    live_watchdog.cpp
+)
+
+# Two files carry a switch over an enum with no `default:` label, deliberately,
+# so that -Wswitch reports an enumerator nobody handled. -Wswitch is only a
+# warning by default, and a warning in a 600-file build log is a warning nobody
+# reads, so it is promoted to an error on exactly these two translation units.
+# Not project-wide: upstream's own sources are not held to this, and they are
+# where the churn is.
+#
+# loaded_model.cpp mirrors engine::runtime::VoiceTaskKind onto its own Task enum.
+# Its static_asserts catch an insertion or a reorder, but an enumerator APPENDED
+# after the last one shifts no value, so no assertion can see it. What does see
+# it is from_engine_task's switch over the engine enum. This is the difference
+# between a build failure and a backend that silently runs the wrong task.
+#
+# capability_routing.cpp's unsupported_surface() switches UnsupportedRpc onto the
+# row of unsupported_surfaces() that explains it. Left as a warning, a sixth
+# enumerator added without a row BUILDS AND SHIPS, and its trailing
+# `return surfaces[0];` then answers the new RPC with AudioEncode's codec reason:
+# a confident, specific and false statement about audio.cpp, on the wire, on the
+# one code path whose entire job is to be truthful about what this backend
+# cannot do. Verified rather than assumed: adding a sixth enumerator and building
+# the shipping target produced exit 0, a binary, and one warning. A compile-time
+# check is the better trade than the runtime fallback it replaced only if it is
+# fatal, so here it is fatal.
+if(NOT MSVC)
+    set_source_files_properties(loaded_model.cpp capability_routing.cpp
+        PROPERTIES COMPILE_OPTIONS "-Werror=switch")
+endif()
+
+target_include_directories(${TARGET} PRIVATE
+    "${AUDIO_CPP_DIR}/include"
+    "${CMAKE_CURRENT_SOURCE_DIR}")
+
+# The shipping binary is held to the same bar as the tests below. Upstream's own
+# add_compile_options is a property of its directory and never reached this
+# target, so until now "the build was clean" meant only that nothing was being
+# checked.
+if(NOT MSVC)
+    target_compile_options(${TARGET} PRIVATE -Wall -Wextra -Wpedantic)
+endif()
+
+target_link_libraries(${TARGET} PRIVATE
+    hw_grpc_proto
+    engine_runtime
+    ggml
+    gRPC::grpc++
+    gRPC::grpc++_reflection
+    protobuf::libprotobuf
+    Threads::Threads)
+
+# ENGINE_ENABLE_CPU_ALL_VARIANTS builds ggml backends as shared objects that sit
+# next to the binary in the package, so the binary must search its own directory.
+# BUILD_WITH_INSTALL_RPATH keeps the build-tree binary at exactly "$ORIGIN".
+# Upstream sets CMAKE_BUILD_WITH_INSTALL_RPATH in its own directory scope, which
+# does not reach ours, so without this CMake also appends its build-tree library
+# directory. That absolute build-host path would survive into the copied binary
+# and let a package.sh that forgot to bundle libggml*.so still pass on the build
+# machine while failing everywhere else.
+set_target_properties(${TARGET} PROPERTIES
+    BUILD_RPATH "$ORIGIN"
+    INSTALL_RPATH "$ORIGIN"
+    BUILD_WITH_INSTALL_RPATH TRUE)
+
+if(AUDIO_CPP_GRPC_BUILD_TESTS)
+    enable_testing()
+
+    # These are the units whose tests CANNOT run under
+    # backend/cpp/run-unit-tests.sh, because that script compiles each
+    # *_test.cpp standalone with no protobuf and no audio.cpp include path.
+    # They are named *_ctest.cpp so the script's glob does not pick them up and
+    # fail every backend's suite; everything that can be stdlib-only still is,
+    # and still lives in a *_test.cpp beside its unit.
+    add_executable(result_map_ctest
+        result_map_ctest.cpp
+        result_map.cpp
+        transcript_assembly.cpp
+        audio_units.cpp)
+    target_include_directories(result_map_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h. Every other target
+        # here inherits that directory from the ggml target it links; this one
+        # links no ggml, so it has to name it.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    # No engine_runtime: result_map touches only the plain structs in
+    # engine/framework/runtime/session.h, so the header is all it needs.
+    target_link_libraries(result_map_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(result_map_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME result_map COMMAND result_map_ctest)
+
+    # Same shape as result_map_ctest: generation_request touches only the plain
+    # structs in engine/framework/runtime/session.h plus the generated protobuf
+    # messages, so the headers are all it needs and no engine_runtime is linked.
+    add_executable(generation_request_ctest
+        generation_request_ctest.cpp
+        generation_request.cpp)
+    target_include_directories(generation_request_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h, and this target links
+        # no ggml, so it has to name the include directory itself.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    target_link_libraries(generation_request_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(generation_request_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME generation_request COMMAND generation_request_ctest)
+
+    add_executable(audio_io_ctest
+        audio_io_ctest.cpp
+        audio_io.cpp)
+    target_include_directories(audio_io_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(audio_io_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(audio_io_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    set_target_properties(audio_io_ctest PROPERTIES
+        BUILD_RPATH "$ORIGIN"
+        INSTALL_RPATH "$ORIGIN"
+        BUILD_WITH_INSTALL_RPATH TRUE)
+    add_test(NAME audio_io COMMAND audio_io_ctest)
+
+    # The streaming drivers live in loaded_model.cpp, which links the engine, so
+    # this cannot be a standalone *_test.cpp. It builds no model and reads no
+    # file: LoadedModel::Session is a plain struct holding a pointer to an
+    # engine interface, so the drivers are exercised against fake sessions.
+    add_executable(streaming_driver_ctest
+        streaming_driver_ctest.cpp
+        loaded_model.cpp
+        capability_routing.cpp
+        family_gate.cpp
+        model_options.cpp
+        inference_lane.cpp)
+    target_include_directories(streaming_driver_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(streaming_driver_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(streaming_driver_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    # No "$ORIGIN" rpath override here, unlike the shipping target and unlike
+    # audio_io_ctest. loaded_model.cpp reaches make_default_registry, so this
+    # binary genuinely links libggml, and CMake's own build-tree rpath is what
+    # finds it: the ggml shared objects land in ${CMAKE_CURRENT_BINARY_DIR}/bin
+    # while the test binary sits one directory up. A build-host absolute path in
+    # a test binary is harmless, since package.sh ships only grpc-server, and
+    # forcing "$ORIGIN" here means ctest cannot start the binary at all.
+    add_test(NAME streaming_driver COMMAND streaming_driver_ctest)
+
+    # Asserts that the upstream ABSENCES capability_routing.cpp's refusal
+    # messages rest on are still absences, by querying make_default_registry()
+    # rather than by re-reading upstream. This is what makes an AUDIO_CPP_VERSION
+    # bump that adds a codec task kind, an spk family or a streaming converter
+    # fail the build instead of leaving a false statement on the wire.
+    #
+    # It links engine_runtime purely to run that query, which is why it lives
+    # here rather than with the standalone *_test.cpp files, and it needs no
+    # "$ORIGIN" rpath override for the same reason streaming_driver_ctest does
+    # not: see the note above.
+    add_executable(upstream_absence_ctest upstream_absence_ctest.cpp)
+    target_include_directories(upstream_absence_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(upstream_absence_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(upstream_absence_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME upstream_absence COMMAND upstream_absence_ctest)
+endif()

backend/cpp/audio-cpp/Makefile

@@ -0,0 +1,172 @@
+# audio.cpp backend Makefile.
+#
+# Upstream pin lives below in the AUDIO_CPP_VERSION variable, so
+# .github/bump_deps.sh can find and update it, matching the llama-cpp / ds4
+# convention. That script seds every line matching the variable name followed by
+# an assignment, so this comment deliberately spells the name on its own: a
+# comment repeating the full assignment token gets rewritten and mangled by the
+# first auto-bump (backend/cpp/ds4/Makefile shows the damage). The clone
+# recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
+# rebuild and so the bump bot can see the pin.
+
+AUDIO_CPP_VERSION?=545e29a6f2fde24298cb3b0f07baab4352987ac9
+AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
+
+CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+BUILD_DIR := build
+
+BUILD_TYPE ?=
+NATIVE ?= false
+JOBS ?= $(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
+
+UNAME_S := $(shell uname -s)
+
+# AUDIOCPP_DEPLOYMENT_BUILD compiles the model_specs/*.json catalog into
+# engine_runtime, so the shipped package needs no model_specs directory and a
+# safetensors model tree still resolves its family spec.
+CMAKE_ARGS ?= -DCMAKE_BUILD_TYPE=Release -DAUDIOCPP_DEPLOYMENT_BUILD=ON
+
+# CMAKE_CUDA_ARCHITECTURES must be set explicitly for a cublas build, and this
+# is not a tuning knob: upstream's CMakeLists sets CUDA_ARCHITECTURES to
+# `native` on the engine_runtime target whenever the root-scope variable is
+# unset (audio.cpp/CMakeLists.txt, the `if (CMAKE_CUDA_ARCHITECTURES)` branch
+# next to the istft/torch_random .cu sources), and docs/build/linux.md says so
+# outright: "Leave CMAKE_CUDA_ARCHITECTURES unset to build for the GPUs present
+# at build time (native)". No CI runner has a GPU, so `native` has nothing to
+# enumerate. ggml's own default (external/ggml/src/ggml-cuda/CMakeLists.txt)
+# does not rescue this: it list(APPEND)s in the ggml subdirectory scope, which
+# never reaches the root scope where the engine_runtime property is decided.
+#
+# The values below are ggml's list for the matching toolkit, copied rather than
+# invented, so the two targets compile for exactly the same set:
+#   - CUDA 13 drops the Maxwell/Pascal/Volta virtual archs (50/61/70).
+#   - 121a-real needs CUDA >= 12.9, so the CUDA 12 list (built against 12.8)
+#     stops at 120a-real.
+#   - `a`-suffixed archs are used rather than ggml's rejected 120f-virtual: the
+#     `f` suffix needs CMake >= 3.31.8, and Ubuntu Noble ships 3.28.3. The
+#     3.28 validator (Modules/Internal/CMakeCUDAArchitecturesValidate.cmake)
+#     accepts `[0-9]+a?(-real|-virtual)?`.
+#
+# Setting it here also pins ggml's copy, since its default is guarded by
+# `if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES)`. CUDA_MAJOR_VERSION is the CI
+# build-arg, forwarded by Dockerfile.audio-cpp.
+#
+# An EMPTY major maps to `native`, NOT to the CUDA 12 list. Only CI declares a
+# major; a developer running `BUILD_TYPE=cublas make` locally declares none, and
+# the CUDA 12 list contains 120a-real, which needs nvcc >= 12.8. Falling through
+# to it turned every local build on a CUDA 12.0-12.7 host into a compile error,
+# where upstream's documented behaviour ("Leave CMAKE_CUDA_ARCHITECTURES unset
+# to build for the GPUs present at build time") worked. `native` restores that.
+# It does require a GPU to enumerate, so the escape hatch for a GPU-less local
+# cross-build is to set CUDA_ARCHITECTURES on the command line, which the ?=
+# assignments below leave untouched.
+CUDA_MAJOR_VERSION ?=
+ifeq ($(CUDA_MAJOR_VERSION),13)
+    CUDA_ARCHITECTURES ?= 75-virtual;80-virtual;86-real;89-real;120a-real;121a-real
+else ifeq ($(CUDA_MAJOR_VERSION),12)
+    CUDA_ARCHITECTURES ?= 50-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-real;89-real;120a-real
+else ifeq ($(CUDA_MAJOR_VERSION),)
+    CUDA_ARCHITECTURES ?= native
+else ifeq ($(BUILD_TYPE),cublas)
+    # Gated on cublas because the variable means nothing to any other build, so a
+    # stray CUDA_MAJOR_VERSION in the environment must not break `make clean` or
+    # a CPU build. It does still error for `BUILD_TYPE=cublas make clean`, which
+    # is the right trade: that invocation is asking about a CUDA build tree.
+    $(error CUDA_MAJOR_VERSION=$(CUDA_MAJOR_VERSION) has no architecture list here (12 and 13 do). Leave it empty for a native build, or pass CUDA_ARCHITECTURES explicitly.)
+endif
+
+ifeq ($(BUILD_TYPE),cublas)
+    CMAKE_ARGS += -DENGINE_ENABLE_CUDA=ON "-DCMAKE_CUDA_ARCHITECTURES=$(CUDA_ARCHITECTURES)"
+else ifeq ($(BUILD_TYPE),vulkan)
+    CMAKE_ARGS += -DENGINE_ENABLE_VULKAN=ON
+else ifeq ($(UNAME_S),Darwin)
+    # Metal. ggml embeds the shader library by default (GGML_METAL_EMBED_LIBRARY
+    # defaults to GGML_METAL), so the package needs no .metallib beside the
+    # binary. Darwin builds go through scripts/build/audio-cpp-darwin.sh.
+    CMAKE_ARGS += -DENGINE_ENABLE_METAL=ON
+    # AppleClang ships no OpenMP runtime and Homebrew's libomp is keg-only, so
+    # neither libomp.dylib nor omp.h is symlinked into /opt/homebrew and CMake's
+    # FindOpenMP cannot find them on its own (the workflow's `brew link libomp`
+    # is a no-op for a keg-only formula, and its failure is swallowed).
+    # audio.cpp calls find_package(OpenMP REQUIRED COMPONENTS CXX) whenever
+    # ENGINE_ENABLE_OPENMP is ON, so with no hint the macOS build dies at
+    # configure time before compiling anything. OpenMP_ROOT is honoured by the
+    # find_library/find_path calls inside FindOpenMP under CMP0074, which is NEW
+    # here because audio.cpp requires CMake 3.20.
+    #
+    # If the keg is absent, turn OpenMP off rather than fail: the tree's only
+    # <omp.h> include is guarded by #ifdef _OPENMP and a #pragma omp without
+    # -fopenmp is simply ignored, so an OpenMP-less build is CORRECT. It is not
+    # cheap, though: 108 `#pragma omp` directives across ~30 files (roformer,
+    # demucs, chatterbox, moss, supertonic, seed_vc, framework/audio/dsp) are
+    # compiled out, and clang says nothing about an ignored omp pragma unless
+    # -Wsource-uses-openmp is on. A green package that is quietly single-threaded
+    # in every host DSP loop gets blamed on Metal, not on packaging, so the
+    # fallback announces itself.
+    ifeq ($(origin LIBOMP_PREFIX),undefined)
+        LIBOMP_PREFIX := $(shell brew --prefix libomp 2>/dev/null)
+    endif
+    # Nested ifneq rather than $(and ...): $(and) needs GNU make 3.81, and while
+    # that is what Apple ships, an older make expands it to empty and would take
+    # the OpenMP-OFF branch with no way to tell that from a genuinely missing
+    # keg. Two plain conditionals cannot fail that way.
+    LIBOMP_USABLE :=
+    ifneq ($(wildcard $(LIBOMP_PREFIX)/lib/libomp.dylib),)
+        ifneq ($(wildcard $(LIBOMP_PREFIX)/include/omp.h),)
+            LIBOMP_USABLE := yes
+        endif
+    endif
+    ifeq ($(LIBOMP_USABLE),yes)
+        CMAKE_ARGS += "-DOpenMP_ROOT=$(LIBOMP_PREFIX)"
+    else
+        $(warning audio-cpp: libomp not found at '$(LIBOMP_PREFIX)'; building without OpenMP (single-threaded host DSP). Install it with `brew install libomp`, or set LIBOMP_PREFIX.)
+        CMAKE_ARGS += -DENGINE_ENABLE_OPENMP=OFF
+    endif
+else
+    # Portable Linux CPU. Upstream wires this to GGML_BACKEND_DL +
+    # GGML_CPU_ALL_VARIANTS + $ORIGIN rpath, so one build serves every CPU
+    # tier instead of an AVX-tier image fan-out.
+    CMAKE_ARGS += -DENGINE_ENABLE_CPU_ALL_VARIANTS=ON
+endif
+
+ifneq ($(NATIVE),true)
+    CMAKE_ARGS += -DENGINE_ENABLE_NATIVE_CPU=OFF
+endif
+
+.PHONY: all grpc-server package test test-engine clean purge
+all: grpc-server
+
+# Clone the upstream source at the pinned commit. The directory is the target
+# so make only re-clones when it is missing. After bumping AUDIO_CPP_VERSION,
+# run 'make purge && make' to refetch.
+audio.cpp:
+	mkdir -p audio.cpp
+	cd audio.cpp && \
+	git init -q && \
+	git remote add origin $(AUDIO_CPP_REPO) && \
+	git fetch --depth 1 origin $(AUDIO_CPP_VERSION) && \
+	git checkout FETCH_HEAD
+
+grpc-server: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS)
+	cp $(BUILD_DIR)/grpc-server grpc-server
+
+package: grpc-server
+	bash package.sh
+
+test:
+	@echo "audio-cpp: standalone unit tests run from the repo root via 'make test-backend-cpp'"
+
+# Engine-linked tests. Needs the upstream checkout and a full engine build.
+test-engine: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) -DAUDIO_CPP_GRPC_BUILD_TESTS=ON $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS) && ctest --output-on-failure --no-tests=error
+
+clean:
+	rm -rf $(BUILD_DIR) grpc-server package
+
+purge: clean
+	rm -rf audio.cpp

backend/cpp/audio-cpp/audio_io.cpp

@@ -0,0 +1,124 @@
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include "engine/framework/audio/conversion.h"
+#include "engine/framework/audio/wav_reader.h"
+#include "engine/framework/audio/wav_writer.h"
+
+#include <filesystem>
+#include <string>
+#include <utility>
+
+namespace audiocpp_backend {
+
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no input audio path was supplied");
+    }
+    std::error_code ec;
+    const bool present = std::filesystem::exists(std::filesystem::path(path), ec);
+    if (ec) {
+        // exists() returning false with ec set does NOT mean the file is
+        // absent, it means the question could not be answered: most often a
+        // parent directory is not searchable. Reporting that as "does not
+        // exist" sends the operator after the file when the fault is the
+        // permissions on the directory above it.
+        throw ConfigError("audio-cpp: cannot stat input audio " + path + ": " +
+                          ec.message());
+    }
+    if (!present) {
+        throw ConfigError("audio-cpp: input audio does not exist: " + path);
+    }
+    engine::audio::WavData wav;
+    try {
+        wav = engine::audio::read_wav_f32(std::filesystem::path(path));
+    } catch (const std::exception &err) {
+        throw ConfigError("audio-cpp: cannot read " + path +
+                          " as WAV: " + err.what());
+    }
+    if (wav.sample_rate <= 0) {
+        throw ConfigError("audio-cpp: " + path +
+                          " declares a non-positive sample rate; every "
+                          "timestamp derived from it would be zero");
+    }
+    // AudioBuffer's own default is 1, and a reader that reports 0 channels
+    // still gave us an interleaving of one. Normalised before the conversion
+    // below rather than after, because mixdown_interleaved_to_mono_average
+    // throws on a non-positive channel count.
+    if (wav.channels <= 0) {
+        wav.channels = 1;
+    }
+
+    engine::runtime::AudioBuffer buffer;
+    if (target_sample_rate <= 0) {
+        buffer.sample_rate = wav.sample_rate;
+        buffer.channels = wav.channels;
+        buffer.samples = std::move(wav.samples);
+        return buffer;
+    }
+
+    buffer.sample_rate = target_sample_rate;
+    buffer.channels = 1;
+    try {
+        // A no-op copy when the rates already match, so the common 16 kHz
+        // upload pays only the mono mixdown it would have paid inside the
+        // family anyway.
+        buffer.samples =
+            engine::audio::convert_wav_to_mono_linear_resampled(wav, target_sample_rate);
+    } catch (const std::exception &err) {
+        // ConfigError, so this is INVALID_ARGUMENT rather than INTERNAL. What
+        // reaches here is a malformed input: a sample count that is not a whole
+        // number of frames is the realistic one, and it is the uploader's file
+        // that is truncated, not this backend that is broken.
+        throw ConfigError("audio-cpp: cannot resample " + path + " from " +
+                          std::to_string(wav.sample_rate) + " Hz to " +
+                          std::to_string(target_sample_rate) +
+                          " Hz: " + err.what());
+    }
+    return buffer;
+}
+
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no output path was supplied");
+    }
+    const std::filesystem::path destination(path);
+    if (destination.has_parent_path()) {
+        // Best effort: a failure here shows up as a write failure below, with a
+        // message naming the file the caller actually asked for.
+        std::error_code ec;
+        std::filesystem::create_directories(destination.parent_path(), ec);
+    }
+    try {
+        engine::audio::write_pcm16_wav(destination, audio.sample_rate,
+                                       audio.channels > 0 ? audio.channels : 1,
+                                       audio.samples);
+    } catch (const std::exception &err) {
+        // NOT a ConfigError, and the distinction is not cosmetic. The
+        // destination is chosen by LocalAI rather than by the caller: it is a
+        // unique name inside GeneratedContentDir. A failure to write it is a
+        // full disk, a permission fault on the server's own directory, or a bad
+        // mount, none of which the caller can fix or is to blame for. As a
+        // ConfigError this surfaced as INVALID_ARGUMENT, which tells a client
+        // its request was wrong and not to retry; a plain runtime_error maps to
+        // INTERNAL, which is both true and retryable. The empty path above
+        // stays INVALID_ARGUMENT, because that one really is a malformed
+        // request.
+        throw std::runtime_error("audio-cpp: cannot write " + path + ": " +
+                                 err.what());
+    }
+}
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = 1;
+    buffer.samples = std::move(samples);
+    return buffer;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io.h

@@ -0,0 +1,61 @@
+#pragma once
+
+// Thin wrappers over the framework's public audio IO. Engine-linked, so this
+// unit is built and tested through the CMake target rather than by
+// backend/cpp/run-unit-tests.sh. The pure part of the arithmetic these
+// wrappers feed lives in audio_units, which is stdlib-only and does have a
+// standalone test.
+
+#include "engine/framework/runtime/session.h"
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Reads a WAV file. Throws ConfigError when the file is missing, is not
+// readable as WAV, or declares a non-positive sample rate: all three are
+// user-fixable input problems rather than backend faults.
+//
+// A declared sample rate of zero is refused rather than passed on, because
+// every downstream conversion in audio_units answers 0 for a non-positive rate.
+// Accepting it would turn a corrupt header into a response full of zero
+// timestamps, which reads as a real answer.
+//
+// `target_sample_rate` is the rate the CALLER needs, in Hz:
+//
+//   0 (or negative)  keep the file's own rate and channel count.
+//   positive         downmix to mono and resample to that rate. Resampling is
+//                    skipped when the file already declares it, so passing the
+//                    rate a route needs costs nothing on the common input.
+//
+// It is a parameter, and not a constant inside this function, because the
+// routes that read audio do not agree on an answer. Speech routes want 16 kHz
+// mono; source separation does not, and folding a 44.1 kHz stereo input to
+// 16 kHz mono for demucs or roformer would destroy the very thing they separate
+// (both refuse a rate other than their own outright). Making the caller name
+// the rate keeps that decision where the route is known.
+//
+// Downmixing along with the resample is not an extra liberty: every family a
+// positive rate is used for (silero_vad, sortformer_diar and every ASR family)
+// begins by calling the same mixdown_interleaved_to_mono_average on whatever it
+// is given. Doing it once here produces the identical samples and halves the
+// buffer that is then moved through the request.
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate);
+
+// Writes 16-bit PCM WAV, creating parent directories.
+//
+// Throws ConfigError, i.e. INVALID_ARGUMENT, ONLY for an empty path, which is a
+// malformed request. Every other failure throws a plain runtime_error, i.e.
+// INTERNAL: the destination is LocalAI's own generated-content directory and
+// not anything the caller named, so a full disk or a permission fault there is
+// a server fault and is worth retrying, which is the opposite of what
+// INVALID_ARGUMENT tells a client.
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio);
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io_ctest.cpp

@@ -0,0 +1,217 @@
+// Tests for audio_io's reading contract, and in particular for the resampling
+// that keeps a 44.1 or 48 kHz upload from reaching a family that only accepts
+// 16 kHz.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE: see the note at the top of
+// result_map_ctest.cpp. This file links the audio.cpp engine, so it is built
+// and run by ctest, not by backend/cpp/run-unit-tests.sh.
+//
+//   make -C backend/cpp/audio-cpp test-engine
+
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include <algorithm>
+#include <cmath>
+#include <cstdio>
+#include <filesystem>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// A one-second tone, interleaved across `channels`. Real audio rather than
+// silence so a resample that dropped its input would be visible as a flat
+// buffer, not just as a different length.
+static engine::runtime::AudioBuffer tone(int sample_rate, int channels,
+                                         float seconds) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = channels;
+    const auto frames =
+        static_cast<size_t>(static_cast<double>(sample_rate) * seconds);
+    buffer.samples.reserve(frames * static_cast<size_t>(channels));
+    for (size_t frame = 0; frame < frames; ++frame) {
+        const float value = 0.5f * std::sin(2.0f * 3.14159265f * 220.0f *
+                                            static_cast<float>(frame) /
+                                            static_cast<float>(sample_rate));
+        for (int channel = 0; channel < channels; ++channel) {
+            buffer.samples.push_back(value);
+        }
+    }
+    return buffer;
+}
+
+static float peak(const std::vector<float> &samples) {
+    float highest = 0.0f;
+    for (const float sample : samples) {
+        highest = std::max(highest, std::abs(sample));
+    }
+    return highest;
+}
+
+static std::filesystem::path scratch_dir() {
+    const auto dir = std::filesystem::temp_directory_path() / "audiocpp-io-ctest";
+    std::filesystem::create_directories(dir);
+    return dir;
+}
+
+// The I2 fixture. Before the resample this returned a 44.1 kHz buffer, which
+// silero_vad and sortformer_diar both reject with a plain runtime_error, which
+// the server maps to INTERNAL. A 44.1 kHz WAV is an ordinary upload.
+static void test_441k_stereo_is_read_as_16k_mono() {
+    const auto path = scratch_dir() / "input-44100-stereo.wav";
+    write_audio_file(path.string(), tone(44100, 2, 1.0f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "44.1 kHz input is resampled to 16 kHz");
+    check(audio.channels == 1, "stereo input is downmixed to mono");
+    // Linear resampling lands within a sample or two of the exact ratio.
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 15990 && frames < 16010,
+          "one second in stays one second out");
+    check(peak(audio.samples) > 0.2f,
+          "the resampled buffer still carries the signal");
+}
+
+static void test_48k_is_read_as_16k() {
+    const auto path = scratch_dir() / "input-48000-mono.wav";
+    write_audio_file(path.string(), tone(48000, 1, 0.5f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "48 kHz input is resampled to 16 kHz");
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 7990 && frames < 8010, "half a second in, half a second out");
+}
+
+// The common case: the upload is already 16 kHz mono, and nothing is resampled.
+static void test_16k_mono_passes_through_unchanged() {
+    const auto path = scratch_dir() / "input-16000-mono.wav";
+    const auto source = tone(16000, 1, 1.0f);
+    write_audio_file(path.string(), source);
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "16 kHz stays 16 kHz");
+    check(audio.channels == 1, "mono stays mono");
+    check(audio.samples.size() == source.samples.size(),
+          "a matching rate resamples nothing");
+}
+
+// Rate 0 means "give me the file as it is", which is what a source separation
+// route needs: demucs and roformer refuse anything but their own 44.1 kHz and
+// work on stereo, so the reader must not force them to 16 kHz mono.
+static void test_zero_target_keeps_the_native_format() {
+    const auto path = scratch_dir() / "input-native.wav";
+    write_audio_file(path.string(), tone(44100, 2, 0.25f));
+
+    const auto audio = read_audio_file(path.string(), 0);
+    check(audio.sample_rate == 44100, "a zero target keeps the file's rate");
+    check(audio.channels == 2, "a zero target keeps the file's channels");
+}
+
+static void test_missing_file_is_a_config_error() {
+    bool threw_config_error = false;
+    try {
+        read_audio_file((scratch_dir() / "does-not-exist.wav").string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+        // Any other type maps to INTERNAL, which is what this asserts against.
+    }
+    check(threw_config_error, "a missing input file is INVALID_ARGUMENT, not INTERNAL");
+}
+
+static void test_unreadable_file_is_a_config_error() {
+    const auto path = scratch_dir() / "not-a-wav.wav";
+    {
+        FILE *file = fopen(path.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("this is not a RIFF header", file);
+            fclose(file);
+        }
+    }
+    bool threw_config_error = false;
+    try {
+        read_audio_file(path.string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "a non-WAV input is INVALID_ARGUMENT, not INTERNAL");
+}
+
+// The write side of the same distinction. The destination is LocalAI's own
+// generated-content directory, not a caller-supplied path, so a failure to
+// write it is a server fault: INTERNAL, which a client may retry, and not
+// INVALID_ARGUMENT, which tells it the request itself was wrong.
+static void test_write_failure_is_not_a_config_error() {
+    // A regular file where a directory has to be. ENOTDIR defeats root as well
+    // as an ordinary user, unlike a chmod, which CI running as root would walk
+    // straight through.
+    const auto blocker = scratch_dir() / "blocking-file";
+    {
+        FILE *file = fopen(blocker.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("not a directory", file);
+            fclose(file);
+        }
+    }
+    const auto path = blocker / "nested" / "out.wav";
+
+    bool threw_config_error = false;
+    bool threw_something = false;
+    try {
+        write_audio_file(path.string(), tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+        threw_something = true;
+    } catch (const std::exception &) {
+        threw_something = true;
+    }
+    check(threw_something, "an unwritable destination is reported at all");
+    check(!threw_config_error,
+          "a failed write is INTERNAL, not INVALID_ARGUMENT: the caller did not "
+          "choose the destination and cannot fix it");
+    check(!std::filesystem::exists(path), "and nothing was written");
+}
+
+static void test_empty_output_path_is_a_config_error() {
+    // The one write failure that IS the caller's: no path at all.
+    bool threw_config_error = false;
+    try {
+        write_audio_file("", tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "an empty output path stays INVALID_ARGUMENT");
+}
+
+int main() {
+    test_441k_stereo_is_read_as_16k_mono();
+    test_48k_is_read_as_16k();
+    test_16k_mono_passes_through_unchanged();
+    test_zero_target_keeps_the_native_format();
+    test_missing_file_is_a_config_error();
+    test_unreadable_file_is_a_config_error();
+    test_write_failure_is_not_a_config_error();
+    test_empty_output_path_is_a_config_error();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_io checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/audio_units.cpp

@@ -0,0 +1,139 @@
+#include "audio_units.h"
+
+#include <algorithm>
+#include <cmath>
+#include <limits>
+
+namespace audiocpp_backend {
+
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels) {
+    const std::size_t lanes = channels > 0 ? static_cast<std::size_t>(channels)
+                                           : static_cast<std::size_t>(1);
+    // Truncating division is deliberate: a trailing partial frame is not a
+    // position every channel reached, so counting it would overstate the length.
+    return static_cast<std::int64_t>(sample_count / lanes);
+}
+
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0;
+    }
+    // Split into whole seconds plus a remainder so the intermediate product
+    // cannot overflow on long recordings, and so rates like 44100 stay exact.
+    // The remainder division truncates deliberately: that matches Go's
+    // time.Duration conventions and keeps successive sample indices monotonic.
+    const std::int64_t rate = static_cast<std::int64_t>(sample_rate);
+    const std::int64_t whole_seconds = samples / rate;
+    const std::int64_t remainder = samples % rate;
+    return whole_seconds * 1000000000LL + (remainder * 1000000000LL) / rate;
+}
+
+float samples_to_seconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0.0f;
+    }
+    return static_cast<float>(static_cast<double>(samples) /
+                              static_cast<double>(sample_rate));
+}
+
+std::int64_t seconds_to_samples(double seconds, int sample_rate) {
+    // !(seconds > 0.0) rather than seconds <= 0.0: every comparison against NaN
+    // is false, so the <= form lets NaN reach the cast below, which is undefined
+    // behaviour and lands on INT64_MIN in practice. This is the one entry point
+    // fed by untrusted-shaped input (a float-seconds timestamp off the wire, or
+    // a boundary from a model that diverged), and a hugely negative sample index
+    // used later as an offset or a length is a wild pointer rather than merely a
+    // wrong timestamp.
+    if (sample_rate <= 0 || !(seconds > 0.0)) {
+        return 0;
+    }
+    const double scaled = seconds * static_cast<double>(sample_rate);
+    // Bound before the cast for the same reason: converting a double at or above
+    // 2^63 (infinity included) is undefined behaviour, so saturate instead.
+    const double limit =
+        static_cast<double>(std::numeric_limits<std::int64_t>::max());
+    if (scaled >= limit) {
+        return std::numeric_limits<std::int64_t>::max();
+    }
+    // Round rather than truncate: these functions exist to cross the float
+    // seconds boundary the VAD and diarize messages use, so a value that came
+    // from samples_to_seconds converts back to the sample it started as.
+    // Truncation lost one sample about half the time, starting at n=1.
+    //
+    // That round trip is exact only below roughly 2^23 samples. Past that the
+    // float samples_to_seconds returns can no longer resolve adjacent indices
+    // and the trip fails whatever the rounding. Both the first failing INDEX
+    // and the duration it stands for depend on the rate, so they are listed per
+    // rate rather than folded into one range; measured:
+    //
+    //   16 kHz    16384001 samples   17.1 min
+    //   44.1 kHz  11289602 samples    4.3 min
+    //   48 kHz    12288002 samples    4.3 min
+    //   96 kHz    12288002 samples    2.1 min
+    //
+    // The shortest recording this bites is therefore a couple of minutes of
+    // 96 kHz audio. It is a property of the float seconds API itself, not of
+    // the rounding here, and it is why nothing should use these to carry a
+    // sample-accurate position in a long recording.
+    return static_cast<std::int64_t>(std::llround(scaled));
+}
+
+std::vector<float> s16le_to_f32(const std::string &bytes) {
+    std::vector<float> samples;
+    const size_t count = bytes.size() / 2;
+    samples.reserve(count);
+    for (size_t i = 0; i < count; ++i) {
+        const auto low = static_cast<unsigned char>(bytes[i * 2]);
+        const auto high = static_cast<unsigned char>(bytes[i * 2 + 1]);
+        const auto raw = static_cast<std::int16_t>(
+            static_cast<std::uint16_t>(low) |
+            (static_cast<std::uint16_t>(high) << 8));
+        // 32768 on decode against 32767 on encode is deliberate, not a typo.
+        // 32768 is what keeps INT16_MIN at exactly -1.0 and every other code
+        // inside the [-1, 1] range this header promises; dividing by 32767
+        // would decode INT16_MIN to -1.00003. See f32_to_s16le for the other
+        // half of the pair. The cost is that a round trip shrinks a sample by
+        // 32767/32768, well under one LSB.
+        samples.push_back(static_cast<float>(raw) / 32768.0f);
+    }
+    return samples;
+}
+
+std::string f32_to_s16le(const std::vector<float> &samples) {
+    std::string bytes;
+    bytes.reserve(samples.size() * 2);
+    for (const float sample : samples) {
+        // NaN maps to silence. A NaN sample rendered as a full-scale click is
+        // worse audio than a dropped one, and this unit converts audio that may
+        // have originated off the wire.
+        //
+        // This guard also removes what used to be a spelling hazard in the
+        // clamp below. std::min and std::max return their first argument when
+        // the comparison is false, and every comparison against NaN is false,
+        // so before this branch existed the choice of spelling silently decided
+        // whether a NaN reached std::lround, whose result is unspecified for
+        // NaN. These three leaked it, the last being the idiomatic C++17 way to
+        // write a clamp and so the likeliest future edit:
+        //   std::min(std::max(sample, -1.0f), 1.0f)
+        //   std::max(std::min(sample, 1.0f), -1.0f)
+        //   std::clamp(sample, -1.0f, 1.0f)
+        // The order is no longer load-bearing now that the guard runs first,
+        // but the history is why the guard is here, so do not drop it.
+        if (std::isnan(sample)) {
+            bytes.push_back(0);
+            bytes.push_back(0);
+            continue;
+        }
+        const float clamped = std::max(-1.0f, std::min(1.0f, sample));
+        // 32767 rather than 32768 so +1.0 saturates at INT16_MAX instead of
+        // overflowing to INT16_MIN. See s16le_to_f32 for why decode differs.
+        const auto value =
+            static_cast<std::int16_t>(std::lround(clamped * 32767.0f));
+        const auto raw = static_cast<std::uint16_t>(value);
+        bytes.push_back(static_cast<char>(raw & 0xFF));
+        bytes.push_back(static_cast<char>((raw >> 8) & 0xFF));
+    }
+    return bytes;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units.h

@@ -0,0 +1,50 @@
+#pragma once
+
+// Time and sample-format conversion between audio.cpp's runtime types (sample
+// indices, float PCM) and LocalAI's proto types. Standard library only.
+//
+// LocalAI uses three different time units:
+//   TranscriptSegment / TranscriptWord start,end : int64 nanoseconds
+//   VADSegment start,end                         : float seconds
+//   DiarizeSegment start,end                     : float seconds
+
+#include <cstddef>
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Frames in an interleaved buffer of `sample_count` floats laid out across
+// `channels` channels. A frame is one per-channel position, which is the unit
+// every duration and every span boundary in this backend is expressed in, so a
+// stereo buffer must not report twice its real length: feeding sample_count
+// straight to samples_to_seconds makes a 3 second stereo clip come back as 6.
+//
+// A non-positive channel count is treated as mono, matching
+// engine::runtime::AudioBuffer's own default of 1 and keeping a reader that
+// reports 0 channels from dividing by zero.
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels);
+
+// Returns 0 when sample_rate is not positive rather than dividing by zero.
+// Uses integer arithmetic so 44.1 kHz does not lose precision.
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate);
+
+float samples_to_seconds(std::int64_t samples, int sample_rate);
+
+// Rounds to nearest. Negative seconds and NaN both yield 0, and a value too
+// large to convert saturates at INT64_MAX rather than overflowing. Round trips
+// with samples_to_seconds only below roughly 2^23 samples, past which the float
+// seconds can no longer resolve adjacent sample indices.
+std::int64_t seconds_to_samples(double seconds, int sample_rate);
+
+// Decodes little-endian signed 16-bit PCM. A trailing odd byte is dropped.
+std::vector<float> s16le_to_f32(const std::string &bytes);
+
+// Encodes to little-endian signed 16-bit PCM, clamping to [-1, 1] first so an
+// overshooting sample saturates instead of wrapping to the opposite sign.
+// A NaN sample encodes to 0, on the grounds that silence beats a full-scale
+// click.
+std::string f32_to_s16le(const std::vector<float> &samples);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units_test.cpp

@@ -0,0 +1,226 @@
+// Unit tests for audio_units. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+
+#include "audio_units.cpp"
+
+#include <cfenv>
+#include <cmath>
+#include <cstdio>
+#include <limits>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static bool close_to(float a, float b, float tol) { return std::fabs(a - b) <= tol; }
+
+using namespace audiocpp_backend;
+
+static void test_nanoseconds() {
+    // LocalAI TranscriptSegment/TranscriptWord times are nanoseconds
+    // (Go reads them as time.Duration).
+    check(samples_to_nanoseconds(16000, 16000) == 1000000000LL, "1s at 16k is 1e9 ns");
+    check(samples_to_nanoseconds(8000, 16000) == 500000000LL, "0.5s at 16k");
+    check(samples_to_nanoseconds(0, 16000) == 0, "zero samples is zero ns");
+    check(samples_to_nanoseconds(1000, 0) == 0, "zero sample rate yields zero, not UB");
+    // 44.1 kHz must not lose precision to float arithmetic.
+    check(samples_to_nanoseconds(44100, 44100) == 1000000000LL, "1s at 44.1k");
+    check(samples_to_nanoseconds(22050, 44100) == 500000000LL, "0.5s at 44.1k");
+    // The cases above all land on values a float happens to hold exactly, so
+    // they do not actually rule float arithmetic out. These do:
+    // a fraction that does not divide evenly, and a duration whose magnitude
+    // exceeds a float's 24-bit mantissa at nanosecond resolution.
+    check(samples_to_nanoseconds(44099, 44100) == 999977324LL,
+          "44.1k fraction is exact, not rounded through a float");
+    check(samples_to_nanoseconds(44100LL * 3600, 44100) == 3600000000000LL,
+          "one hour at 44.1k is exact to the nanosecond");
+    // A naive samples * 1e9 would overflow int64 here; the split into whole
+    // seconds plus a remainder is what keeps this correct.
+    check(samples_to_nanoseconds(44100LL * 360000, 44100) == 360000000000000LL,
+          "100 hours at 44.1k does not overflow");
+    // Double arithmetic is close enough to pass everything above, but still
+    // truncates this one a nanosecond short. Integer division does not.
+    check(samples_to_nanoseconds(4004, 8000) == 500500000LL,
+          "0.5005s at 8k is exact to the nanosecond");
+    // Truncation, not rounding: this matches Go's time.Duration conventions and
+    // keeps successive sample indices monotonic. The exact value here is
+    // 22675.7...; rounding to nearest would give 22676.
+    check(samples_to_nanoseconds(1, 44100) == 22675LL,
+          "a sub-nanosecond fraction truncates rather than rounding up");
+}
+
+static void test_seconds() {
+    check(close_to(samples_to_seconds(24000, 24000), 1.0f, 1e-6f), "1s at 24k");
+    check(close_to(samples_to_seconds(12000, 24000), 0.5f, 1e-6f), "0.5s at 24k");
+    check(close_to(samples_to_seconds(100, 0), 0.0f, 1e-6f), "zero sample rate is 0s");
+    check(seconds_to_samples(1.0, 16000) == 16000, "1s to samples at 16k");
+    check(seconds_to_samples(0.5, 16000) == 8000, "0.5s to samples at 16k");
+    check(seconds_to_samples(1.0, 0) == 0, "zero sample rate yields zero samples");
+    check(seconds_to_samples(-1.0, 16000) == 0, "negative seconds clamps to zero");
+
+    // seconds_to_samples is the one entry point fed by untrusted-shaped input:
+    // a float-seconds timestamp off the wire, or a VAD boundary from a model
+    // that diverged. A hugely negative sample index used later as an offset or
+    // a length is a wild pointer, not merely a wrong timestamp.
+    const double nan_seconds = std::numeric_limits<double>::quiet_NaN();
+    const double inf_seconds = std::numeric_limits<double>::infinity();
+    const std::int64_t max_samples = std::numeric_limits<std::int64_t>::max();
+    check(seconds_to_samples(nan_seconds, 16000) == 0, "NaN seconds yields zero");
+    check(seconds_to_samples(inf_seconds, 16000) == max_samples,
+          "infinite seconds saturates instead of overflowing");
+    check(seconds_to_samples(1e30, 16000) == max_samples,
+          "out of range seconds saturates instead of overflowing");
+    check(seconds_to_samples(-inf_seconds, 16000) == 0,
+          "negative infinity clamps to zero");
+
+    // Crossing the float-seconds boundary and back is the expected round trip
+    // for the VAD and diarize messages, so it must not lose a sample.
+    // Truncation loses one about half the time, starting at n=1.
+    check(seconds_to_samples(samples_to_seconds(1, 44100), 44100) == 1,
+          "one sample survives the seconds round trip at 44.1k");
+    check(seconds_to_samples(samples_to_seconds(1, 16000), 16000) == 1,
+          "one sample survives the seconds round trip at 16k");
+    check(seconds_to_samples(samples_to_seconds(4001, 8000), 8000) == 4001,
+          "4001 samples survive the seconds round trip at 8k");
+}
+
+static void test_s16le_round_trip() {
+    const std::vector<float> original = {0.0f, 0.5f, -0.5f, 1.0f, -1.0f};
+    const std::string encoded = f32_to_s16le(original);
+    check(encoded.size() == original.size() * 2, "two bytes per sample");
+
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == original.size(), "round trip keeps the sample count");
+    for (size_t i = 0; i < original.size(); ++i) {
+        // 16-bit quantisation: one LSB is ~3.05e-5. Guard the index so a short
+        // result reports a named failure instead of aborting the whole suite.
+        check(i < decoded.size() && close_to(decoded[i], original[i], 1e-4f),
+              "round trip preserves sample " + std::to_string(i));
+    }
+}
+
+static void test_s16le_endianness() {
+    // 0.5 encodes to 16384 = 0x4000, little endian is 0x00 0x40.
+    const std::string encoded = f32_to_s16le({0.5f});
+    check(encoded.size() == 2, "one sample is two bytes");
+    check(static_cast<unsigned char>(encoded[0]) == 0x00, "low byte first");
+    check(static_cast<unsigned char>(encoded[1]) == 0x40, "high byte second");
+}
+
+static void test_s16le_clamping() {
+    // Values outside [-1, 1] must clamp, not wrap around to the opposite sign.
+    const std::string encoded = f32_to_s16le({2.0f, -2.0f});
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == 2, "two samples survive clamping");
+    check(decoded.size() > 0 && decoded[0] > 0.99f,
+          "positive overshoot clamps to full scale");
+    check(decoded.size() > 1 && decoded[1] < -0.99f,
+          "negative overshoot clamps to full scale");
+}
+
+static void test_s16le_decode_range() {
+    // INT16_MIN is the one value that pins the decode scale. Dividing by 32767
+    // instead of 32768 would decode it to -1.00003, outside the [-1, 1] range
+    // the header promises, and every other test would still pass.
+    const std::vector<float> decoded = s16le_to_f32(std::string("\x00\x80", 2));
+    check(decoded.size() == 1, "INT16_MIN decodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == -1.0f,
+          "INT16_MIN decodes to exactly -1.0, not past full scale");
+}
+
+static void test_s16le_nan_input() {
+    // A NaN sample must not reach std::lround, whose result is unspecified for
+    // NaN. Asserting a range is not enough to pin this: the three outcomes the
+    // plausible clamp spellings produce (full scale, negative full scale, zero)
+    // are all finite and all inside [-1, 1], so a range check passes for every
+    // one of them. Only an exact value distinguishes them.
+    // NaN maps to silence, not to full scale: a NaN sample rendered as a
+    // full-scale click is worse audio than a dropped one, and this unit
+    // converts audio that may have originated off the wire.
+    //
+    // volatile so the NaN cannot be constant-folded, which would let the
+    // compiler evaluate the conversion at compile time and raise no
+    // floating-point exception at run time for the check below to observe.
+    volatile float nan_source = std::numeric_limits<float>::quiet_NaN();
+    const std::vector<float> input = {nan_source};
+
+    std::feclearexcept(FE_ALL_EXCEPT);
+    const std::string encoded = f32_to_s16le(input);
+    const bool raised_invalid = std::fetestexcept(FE_INVALID) != 0;
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+
+    check(decoded.size() == 1, "a NaN sample still encodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == 0.0f,
+          "a NaN sample encodes to exactly zero, not to a full-scale click");
+    // Independent of the value: a quiet NaN raises invalid-operation as soon as
+    // it reaches any ordered comparison, which is what std::min and std::max
+    // use, so this fails unless the NaN is diverted before the clamp runs at
+    // all. That is what stops the explicit guard from being dropped in favour
+    // of a clamp spelling that happens to yield zero.
+    check(!raised_invalid,
+          "encoding a NaN sample raises no invalid-operation exception");
+}
+
+static void test_s16le_odd_length() {
+    // A truncated frame must drop the dangling byte rather than read past it.
+    const std::string odd(5, '\0');
+    check(s16le_to_f32(odd).size() == 2, "odd byte count drops the trailing byte");
+    check(s16le_to_f32(std::string()).empty(), "empty input yields no samples");
+}
+
+static void test_interleaved_frame_count() {
+    // Mono is a pass-through, which is the only case the VAD path exercises.
+    check(interleaved_frame_count(16000, 1) == 16000, "mono frames equal samples");
+    // The case that matters: a stereo buffer holds two floats per position, so a
+    // one second 16 kHz stereo clip is 32000 floats and still one second. Handing
+    // the raw float count to samples_to_seconds reports two seconds instead.
+    check(interleaved_frame_count(32000, 2) == 16000,
+          "stereo frames are half the samples");
+    check(samples_to_seconds(interleaved_frame_count(32000, 2), 16000) == 1.0f,
+          "a one second stereo clip measures one second, not two");
+    check(interleaved_frame_count(48000, 3) == 16000,
+          "three channels divide by three");
+    // engine::runtime::AudioBuffer defaults channels to 1, but a reader is free
+    // to report 0, and dividing by that is undefined rather than merely wrong.
+    check(interleaved_frame_count(1000, 0) == 1000,
+          "zero channels is treated as mono");
+    check(interleaved_frame_count(1000, -2) == 1000,
+          "a negative channel count is treated as mono");
+    // A dangling partial frame is not a position every channel reached.
+    check(interleaved_frame_count(3, 2) == 1,
+          "a trailing partial frame is not counted");
+    check(interleaved_frame_count(0, 2) == 0, "an empty buffer has no frames");
+    // Past 2^32 floats, so a size_t narrowed to 32 bits on the way in, or a
+    // signed 32-bit intermediate, shows up here rather than in a multi-hour
+    // recording nobody tests with.
+    check(interleaved_frame_count(static_cast<std::size_t>(9000000000ULL), 2) ==
+              4500000000LL,
+          "a buffer beyond 2^32 floats counts frames without truncating");
+}
+
+int main() {
+    test_interleaved_frame_count();
+    test_nanoseconds();
+    test_seconds();
+    test_s16le_round_trip();
+    test_s16le_endianness();
+    test_s16le_clamping();
+    test_s16le_decode_range();
+    test_s16le_nan_input();
+    test_s16le_odd_length();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_units checks passed\n");
+    return 0;
+}