ci: route site deploy and lint to the self-hosted runner

The GitHub-hosted runner pool is shared per ACCOUNT, not per repo, so a
burst in one repo starves every other. On 2026-07-31 it reached zero
scheduled jobs for 35 consecutive minutes with 39 jobs queued, while
arc-runner-set completed 12 jobs without interruption across the same
window. Actions was healthy globally (other public repos were scheduling
normally), so this is an account-level throttle we cannot fix from inside
the workflows, only route around.

Site publishing and lint are small, run on nearly every commit, and gain
nothing from waiting behind a saturated hosted queue, so both move to
arc-runner-set, the label already proven in generate_intel_image.yaml.

lint.yml is routed for PUSH ONLY, and this is the important part: that
workflow also triggers on pull_request, and a fork PR executes untrusted
contributor code. Running that on a persistent self-hosted runner would be
a real compromise vector, so anything that is not a push to mudler/LocalAI
stays on the ephemeral hosted pool. gh-pages.yml needs no such clause: it
triggers only on push-to-master and workflow_dispatch, so it never runs
pull-request code. Both carry a repository guard so forks, which have no
such runner label, fall back to hosted instead of queueing forever.

Neither workflow uses sudo or apt, and both fetch their own toolchains via
setup-go / actions-hugo. A self-hosted image can still be leaner than the
hosted one, so each lint job opens with a preflight that names the missing
tool (curl/unzip/make for protoc and lint; gcc/ldd/python3 for the
packaging-script tests) rather than failing opaquely mid-build. Reverting
is one runs-on expression per job.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]

.agents/building-and-testing.md

@@ -28,7 +28,6 @@ The core Go suites (`./pkg`, `./core`, plus the in-process integration suite `./
   - **Build tags (`COVERAGE_TAGS`, passed via `GINKGO_TAGS`):** defaults to `debug auth`. The `auth` tag is required to compile the real (sqlite-backed) auth implementation and its ~150 `//go:build auth` tests — without it those files aren't built, the tests don't run, and the gate scores auth against a stub (~3.7% instead of ~38%). If you add new tag-gated tests, extend `COVERAGE_TAGS` or they won't count (and likely won't run in CI at all).
 - `make test-coverage-check` — runs `test-coverage`, then `scripts/coverage-check.sh` fails the build if total coverage is **below** the committed baseline in `coverage-baseline.txt`. The Linux job in `.github/workflows/test.yml` runs this instead of `make test`.
 - `make test-coverage-baseline` — regenerates and overwrites `coverage-baseline.txt` from the current run.
-- `make install-hooks` — sets `core.hooksPath` to the versioned `.githooks/`, whose `pre-commit` runs checks scoped to what's staged: Go changes → `make lint` + `make test-coverage-check`; `core/http/react-ui/` changes → `make test-ui-coverage-check` (Playwright e2e + UI coverage gate). A commit touching neither is skipped; bypass with `git commit --no-verify`. The hook resolves golangci-lint's new-from base to `upstream/master` → `origin/master` → `master`, so it works from a fork clone where `origin/master` is stale (passed to `make lint` via `LINT_NEW_FROM`).
 
 ### React UI coverage
 
@@ -38,12 +37,11 @@ The React UI (`core/http/react-ui/`) has **no component/unit tests** — its onl
 - **Browser:** the flake dev shell ships `chromium` and exports `PLAYWRIGHT_CHROMIUM_PATH`; `playwright.config.js` uses it via `launchOptions.executablePath`, and the Makefile skips `playwright install` when it's set. This avoids Playwright's downloaded browser, which can't resolve system libs (`libglib-2.0`, …) on NixOS. In CI (no `PLAYWRIGHT_CHROMIUM_PATH`) the Makefile falls back to `playwright install --with-deps chromium`.
 - The app is a React SPA, so coverage accumulates across in-app navigation within a test; a full `page.goto`/reload resets it.
 - `.nycrc.json` uses `all: true`, so **every `src/**` file is in the report**, including 0%-coverage ones — that's how you spot features with no test at all (sort the HTML report or `coverage-summary.json` by line% ascending). 
-- **UI coverage gate:** `make test-ui-coverage-check` runs the suite then `scripts/ui-coverage-check.sh`, failing if total line coverage drops more than `UI_COVERAGE_TOLERANCE` below `core/http/react-ui/coverage-baseline.txt`. `make test-ui-coverage-baseline` regenerates the baseline. Runs in CI (`tests-ui-e2e.yml`) and pre-commit on `core/http/react-ui/` changes.
+- **UI coverage gate:** `make test-ui-coverage-check` runs the suite then `scripts/ui-coverage-check.sh`, failing if total line coverage drops more than `UI_COVERAGE_TOLERANCE` below `core/http/react-ui/coverage-baseline.txt`. `make test-ui-coverage-baseline` regenerates the baseline. Runs in CI (`tests-ui-e2e.yml`).
 - **Why it has a tolerance (unlike the strict Go gate):** UI e2e coverage is *non-deterministic*. Specs that assert on state and end while async/lazy render work is still in flight collect those lines only when the render beats the coverage teardown — so the total drifts with machine speed/load (a fast local box reads higher than a slow CI runner), diffusely across many specs. The tolerance absorbs that drift, so set the baseline *below* the slow-CI floor, never to a fast-local `make test-ui-coverage-baseline` number, or CI flaps.
 - **Raising coverage is cheap:** a *render-smoke* spec (navigate to a route, assert its header renders) mounts a lazy page and runs its full render + initial effects, capturing most of its lines in a few lines of test — see `e2e/page-render-smoke.spec.js`. Auth is disabled in the test server (`isAdmin=true`), so `RequireAdmin`/`RequireFeature` routes render without a mock. The most *deterministic* win is removing a race: make a spec `await` a rendered element before ending (see `e2e/agents.spec.js` → AgentCreate) so its lines count every run.
 
 Rules (both gates):
-- **Install the hooks:** `make install-hooks` once per clone so lint + coverage run pre-commit. Don't lean on CI for what the hook catches.
-- **Don't work around the gate:** never `git commit --no-verify`, and never hand-lower a baseline or widen a tolerance to turn a red gate green. The ratchet only moves up.
+- **Don't weaken the gate:** never hand-lower a baseline or widen a tolerance to turn a red gate green. The ratchet only moves up.
 - If a change drops coverage, **add tests** (sort `coverage-summary.json` by line% ascending to find untested code) rather than editing the baseline. When coverage legitimately rises, commit the regenerated baseline (`make test-coverage-baseline` / `test-ui-coverage-baseline`).
 - The Go gate is **strict — no tolerance**; `covermode=atomic` keeps it deterministic. The UI gate keeps a small tolerance only because its e2e coverage isn't.

.agents/ci-caching.md

@@ -122,7 +122,7 @@ The per-backend prefix match only sees files under a backend's own directory, so
 
 | Changed path | Rebuilds |
 |---|---|
-| `backend/backend.proto` | everything (all languages compile or copy it) |
+| `backend/backend.proto` | nothing if the edit is additive-only, otherwise everything (see below) |
 | `backend/Dockerfile.<x>` | the Linux entries whose `dockerfile:` names it |
 | `backend/python/common/` | Python, Linux + Darwin |
 | `scripts/build/package-gpu-libs.sh` | Python, Linux only |
@@ -132,8 +132,79 @@ The per-backend prefix match only sees files under a backend's own directory, so
 
 Deliberately excluded: `backend/index.yaml` (gallery metadata, never enters an image), `.github/backend-matrix.yml` (adding a backend would rebuild all of them), `backend/Dockerfile.base-grpc-builder` (owned by `base-images.yml`), and the root `Makefile` (touched in ~11% of commits, and its backend-relevant edits arrive alongside the backend directory anyway). `make test-ci-scripts` pins all of this.
 
+#### `backend/backend.proto` is content-filtered, not path-filtered
+
+Every language consumes the proto, so a path rule for it can only ever say "rebuild all 473 images". It changes in ~1.3% of commits, and that was enough to make it the single largest CI cost driver in the repo: on 2026-07-29 four runs totalling 935 queued jobs traced to nothing but a proto edit, one of which (#11158) was a six-line diff adding `bool cache_prompt = 8;`.
+
+An additive proto edit cannot change how a backend that never references the new symbol behaves, so `filterMatrix()` suppresses the rule for one. `changed-backends.js` fetches `backend/backend.proto` at the base revision (same contents-API pattern as `.github/backend-matrix.yml`) and hands both texts to `protoChangeIsAdditive()`, which compares them structurally rather than textually:
+
+- **Additive, rebuilds nothing**: a new field with an unused number, a new message, a new enum value, a new RPC. Comment, whitespace and ordering changes also land here.
+- **Breaking, rebuilds everything**: a removed, renumbered, retyped or renamed field, a dropped RPC, a changed `option` or `package`. So does an unresolvable base revision, matching the run-all posture used for a truncated diff.
+
+Checked against every proto commit in the preceding six months, all nine resolvable ones classify as additive. Note the tradeoff this accepts: generated stubs do change for an additive edit, so image bytes would differ on a rebuild even though behavior does not. That is the same standard already applied when the filter declines to rebuild on unrelated `pkg/` changes, and the weekly cron remains the backstop.
+
 The Sunday 06:00 UTC cron on `backend.yml` exists specifically because path filtering can leave Python backends frozen on stale wheels. `DEPS_REFRESH` (below) only fires when the build actually runs, so an untouched Python backend would never re-resolve its unpinned deps. The weekly cron is the safety net.
 
+## Content-blind PRs skip the workflows that cannot see them
+
+`backend_pr.yml` and `test-extra.yml` filter themselves (matrix generation and a `detect-changes` job), so a gallery-only or docs-only PR costs them about one job each. The Go and image workflows had no filter of any kind, so a one-line `gallery/index.yaml` edit queued 20 jobs, and a docs-only PR queued the same.
+
+This is worth more than it looks. Measured over the week to 2026-07-30, **97% of CI wall-clock is queueing, 3% is execution** (median queue ~5h against a 4-20min median job). Cutting job count is therefore the only lever that shortens feedback time; making individual jobs faster moves 3%.
+
+The volume is real: 13 gallery-only PRs merged that week with 10 open at once, and 78 of the 137 PRs opened were bot-generated.
+
+`paths-ignore` on the PR trigger of `image-pr.yml` (7 jobs), `build-test.yaml` (3), `lint.yml` (2) and `tests-e2e.yml` (1) drops 13 of those 20. The excluded set:
+
+| Path | Why no image or Go build can see it |
+|---|---|
+| `gallery/**` | Model-gallery metadata, parsed at runtime, never copied into an image |
+| `docs/**`, `examples/**`, `**/*.md` | Never enter an image or a binary. `lint.yml` already excluded these before gallery was added |
+
+### `backend/{cpp,go,python}/**` on `image-pr.yml` and `build-test.yaml` only
+
+Version-pin bumps dominate PR volume: 48 `update/*` PRs in the week to 2026-07-30, from 16 pins, each a two-line diff. Most edit nothing but one `backend/*/<name>/Makefile`.
+
+Neither of those two workflows can observe such a change. `make build` is `go build ./cmd/local-ai`, GoReleaser builds the same plus `./cmd/launcher`, and the core image's final stage ships only `entrypoint.sh`, `healthcheck.sh` and that binary. The per-backend trees are copied into the builder but nothing in them reaches the output.
+
+What still triggers a full run, because none of it lives under those prefixes:
+
+- `backend/backend.proto` — feeds `protogen-go`, so it does change the binary.
+- `go.mod` / `go.sum` — the `go mod tidy` before-hook.
+- `backend/Dockerfile.*` and anything else directly under `backend/`.
+
+Deliberately **not** applied to:
+
+| Workflow | Why it must keep seeing `backend/**` |
+|---|---|
+| `test.yml` | `TEST_PATHS` explicitly includes `./backend/go/cloud-proxy/...`, `./backend/go/local-store/...` and `./backend/go/valkey-store/...` |
+| `lint.yml` | `.golangci.yml` carries `backend/`-scoped rules, so golangci-lint covers that tree |
+| `tests-e2e.yml` | The e2e suite drives real backends over gRPC |
+| `backend_pr.yml` | This is the workflow whose entire job is to rebuild the changed backend |
+
+What still runs, and why it has to:
+
+| Workflow | Why it keeps running |
+|---|---|
+| `test.yml` (`tests`) | `core/gallery/variants_lint_test.go` reads the real `gallery/index.yaml` and asserts the index invariants (no duplicate entry names, no build claimed by two parents). This is the only schema-level check the gallery has. |
+| `yaml-check.yml` (`Yamllint`) | Lints `gallery/` for syntax. |
+| `backend_pr.yml`, `test-extra.yml` | Already self-filtering; they stop after the detect step. |
+
+Two properties this relies on:
+
+- `paths-ignore` skips a run only when **every** changed file matches, so a PR touching the gallery *and* Go code still runs everything. That is what makes the exclusion safe rather than a hole.
+- `master` carries no branch protection and no rulesets, so a skipped workflow reports no status and nothing waits on it. If required status checks are ever introduced, these four entries must be excluded from the required set or PRs will hang on "Expected — Waiting for status to be reported".
+
+### `image.yml` on master push is gated too, by a job rather than a path filter
+
+The same reasoning applies to master pushes, and the volume is larger there: on 2026-07-30, **12 of the 23 queued `image.yml` runs** were commits like "add 1 new model to gallery" or a docs fix, each rebuilding all 18 container images.
+
+`image.yml` now has a `changes` job that decides once whether the push can affect any image; the other 11 jobs carry `needs: changes` plus an `if:` on its output. Verified against the shipped `Dockerfile`: the final stage copies only `entrypoint.sh`, `healthcheck.sh` and the `local-ai` binary, there is no `go:embed` of `gallery/` or `docs/`, and the gallery is fetched at runtime from `github:mudler/LocalAI/gallery/index.yaml@master`. A gallery-only commit therefore produces byte-identical images, and the gallery change reaches users through GitHub immediately whether or not an image is rebuilt.
+
+Two properties to preserve if you touch it:
+
+- **It is a job gate, not `paths-ignore`.** `paths-ignore` on `push` also applies to tag pushes, and a tag created on an existing commit carries an empty commits list, which would silently skip the release image build. The gate short-circuits to "build" for `refs/tags/*`, and for any push whose base commit is missing, zero, or unresolvable.
+- **The merge jobs must name the gate explicitly.** They use `if: ${{ !cancelled() && ... }}`, and `!cancelled()` is true when a dependency is *skipped*, so without the extra condition they would run and try to merge manifest lists for images that were never built.
+
 ## The `DEPS_REFRESH` cache-buster (Python backends)
 
 Every Python backend goes through the shared `backend/Dockerfile.python`, which ends with:
@@ -169,15 +240,38 @@ RUN --mount=type=cache,target=/root/.ccache,id=<backend>-ccache-${TARGETARCH}-${
     bash /usr/local/sbin/compile.sh
 ```
 
-The compile script exports `CMAKE_C/CXX/CUDA_COMPILER_LAUNCHER=ccache` so CMake threads ccache through gcc/g++/nvcc. `cache-to: type=registry,mode=max` exports the cache mount data into the registry cache, so subsequent builds restore it.
+The compile script exports `CMAKE_C/CXX/CUDA_COMPILER_LAUNCHER=ccache` so CMake threads ccache through gcc/g++/nvcc. Cache scope is per `(TARGETARCH, BUILD_TYPE)` so e.g. cublas-12 doesn't share with cublas-13 (their CUDA headers differ; cross-pollination would just be cache misses anyway).
 
-On a `LLAMA_VERSION` bump, most translation units are byte-identical to the previous version's preprocessed source — ccache returns the previous `.o` and skips the real compile. Same for LocalAI source changes that don't actually touch llama.cpp's CMake inputs. Cache scope is per `(TARGETARCH, BUILD_TYPE)` so e.g. cublas-12 doesn't share with cublas-13 (their CUDA headers differ; cross-pollination would just be cache misses anyway).
+### ⚠️ This ccache does nothing in CI today
+
+This section previously claimed that `cache-to: type=registry,mode=max` "exports the cache mount data into the registry cache, so subsequent builds restore it". **That is not true.** BuildKit does not export the contents of a `--mount=type=cache` to a registry cache export. A cache mount lives in the builder's local state, and every CI job gets a fresh runner with a fresh builder, so `/root/.ccache` starts empty on every single build.
+
+Measured on 2026-07-30 from the `ccache -s` output the compile script already prints (it runs `ccache -z` first, so the numbers are per-build):
+
+| Job | Commit touched | Build time | ccache |
+|---|---|---|---|
+| 89766266951 (llama-cpp, cublas 13) | `backend/go/magpie-tts-cpp/Makefile` only | 6369s | **0 / 889 hits**, and 0 / 1778 |
+| 89766267281 (llama-cpp, hipblas) | same commit | 8160s | **0 / 537 hits** |
+| 90210828110 (llama-cpp, cublas 12.8) | `LLAMA_VERSION` bump | 5673s | **0 / 813 hits** |
+
+The first two are the decisive control: commit `90355cd44` changed exactly one file, `backend/go/magpie-tts-cpp/Makefile`, nowhere near llama.cpp. The engine source was byte-identical to the previous build, which is precisely the case this section says ccache should serve, and the hit rate was still **0.00%**. A cache that was being restored but merely matching poorly would show partial hits; 0-of-N is the signature of an empty cache.
+
+So the paragraph above about `LLAMA_VERSION` bumps reusing previous `.o` files describes an intended design that is not in effect. `Dockerfile.{llama-cpp,ik-llama-cpp,turboquant,bonsai,ds4,privacy-filter}` pay the ccache wrapper overhead and get nothing back. Multi-hour C++ rebuilds are recompiling identical translation units from scratch.
+
+**Do not "fix" this by adding cache mounts to more Dockerfiles.** Wiring the same mount into `Dockerfile.golang` (215 of the 434 matrix entries) was measured locally at 18% faster on a rebuild after a source edit, with a 71.5% ccache hit rate — but only because the local test reused one builder across both builds. In CI it would be a no-op for exactly the reason above.
+
+Making this actually work needs the cache to live outside the builder. The options, none of them free:
+
+- **ccache `remote_storage`** (ccache ≥ 4.4, HTTP or Redis backend) or **sccache** with an S3/GCS/Redis backend. Genuinely works across runners; needs a cache service to point at. quay.io is a registry, not a blob store, so the existing infra does not cover it.
+- **Round-trip the cache dir through `actions/cache` on the runner**: restore it, pass it in, and export it back out via a build stage output. No external infra, but clunky, and the repo already sits at GitHub's 10 GB cache ceiling while the llama-cpp ccache alone is capped at 5 GB.
+
+Until one of those lands, treat C++ backend builds as always-cold and spend the effort on not running them instead (path filtering, see above).
 
 ## Composite actions
 
 Two composite actions handle runner-side prep:
 
-- **`.github/actions/free-disk-space/action.yml`** — wraps `jlumbroso/free-disk-space@main` plus an explicit apt purge of dotnet/android/ghc/mono/etc. Reclaims ~6–10 GB on `ubuntu-latest`. No-op on self-hosted runners. Used by `backend_build.yml`, `image_build.yml`, `test.yml`, `tests-aio.yml`, etc.
+- **`.github/actions/free-disk-space/action.yml`** — wraps `jlumbroso/free-disk-space@main` plus an explicit apt purge of dotnet/android/ghc/mono/etc. Reclaims ~6–10 GB on `ubuntu-latest`. No-op on self-hosted runners. Used by `backend_build.yml`, `image_build.yml` and `base-images.yml` — the jobs that actually build images. Deliberately **not** used by `test.yml`, which runs no buildx step.
 - **`.github/actions/setup-build-disk/action.yml`** — relocates Docker's data-root to `/mnt` on hosted X64 runners. GHA hosted `ubuntu-latest` ships ~75 GB of unused space at `/mnt`; combined with the free-disk-space cleanup this gives ~100 GB working space — enough for ROCm dev image + vLLM torch install + flash-attn intermediate layers. No-op on self-hosted and on non-X64 hosted runners. Used by `backend_build.yml`, `image_build.yml`, `base-images.yml`.
 
 Both actions run before any docker buildx step.
@@ -218,10 +312,20 @@ Eviction is rarely needed in normal operation — `DEPS_REFRESH` handles weekly
 
 ## What the cache does **not** cover
 
-- The `free-disk-space` and `setup-build-disk` composite actions run on every job — these reclaim runner-state, not Docker layers, so BuildKit caches don't apply.
+- The `free-disk-space` and `setup-build-disk` composite actions run on every job — these reclaim runner-state, not Docker layers, so BuildKit caches don't apply. `test.yml` deliberately does **not** use `free-disk-space`: it runs no buildx step, and the multi-GB fixture downloads that once justified it left `make test` in the test-suite reorg.
 - Intermediate artifacts of `Build (PR)` are not pushed anywhere — PRs only build for verification.
 - Darwin builds (see below) — macOS runners have no Docker daemon, so the registry-backed BuildKit cache cannot apply.
 
+### The Linux Go workflows set `cache: false` on purpose
+
+`test.yml`, `lint.yml`, `tests-e2e.yml` and friends pass `cache: false` to `actions/setup-go@v5`, unlike the darwin jobs. This looks like an oversight and is not.
+
+Measured over the week to 2026-07-30, the `Set up Go` step has a **median of 11 seconds** on these runners. There is essentially nothing to win: the module download is not where the time goes. The expensive steps are compilation and test execution (`Test (with coverage gate)` at ~18.6min, `Test Backend E2E` at ~14.5min), and Go's build cache would have to survive across runners to touch those.
+
+Enabling it also has a real cost. GitHub caps Actions cache at **10 GB per repo and the repo already sits at that ceiling** (31 entries), so every `setup-go` entry written by a branch with a distinct `go.sum` (222-375 MB on Linux, up to 1.4 GB on macOS) evicts something else. See the darwin cache budget below.
+
+Before re-enabling this, measure `Set up Go` again and confirm it has actually become slow. If room is needed in the 10 GB budget, the cheapest evictions are the `docker.io--tonistiigi--binfmt` entries (~30 MB each, trivially re-fetched).
+
 ## Darwin native caches
 
 `backend_build_darwin.yml` runs natively on `macOS-14` GitHub-hosted runners — there is no Docker, no BuildKit, no cross-job registry cache. Instead, the reusable workflow uses `actions/cache@v4` for four native caches that mirror the spirit of the Linux cache (warm by default, weekly refresh for unpinned Python deps, PRs read-only).
@@ -255,6 +359,21 @@ GitHub Actions caches are limited to 10 GB per repo. Steady-state worst case: ~8
 
 One residual self-hosted reference remains in `test-extra.yml` (`tests-vibevoice-cpp-grpc-transcription` uses `bigger-runner` for the 30s JFK-decode timeout headroom). That's a separate concern.
 
+### Small always-on jobs routed to `arc-runner-set`
+
+The hosted pool is shared across the whole *account*, not per repo, so a burst in one repo starves the others. On 2026-07-31 it went to **zero scheduled jobs for 35 consecutive minutes** with 39 jobs queued, while `arc-runner-set` completed 12 jobs without interruption over the same window. Actions was healthy globally at the time (other public repos were scheduling normally), so this is an account-level throttle, not an outage.
+
+Two small, high-frequency workflows are therefore routed to `arc-runner-set`:
+
+| workflow | jobs | routing expression selects self-hosted when |
+|---|---|---|
+| `gh-pages.yml` | `build`, `deploy` | `github.repository == 'mudler/LocalAI'` |
+| `lint.yml` | `golangci-lint`, `build-scripts` | `github.event_name == 'push'` **and** the repo guard |
+
+The `lint.yml` routing is push-only **on purpose**. That workflow also triggers on `pull_request`, and a fork PR runs untrusted contributor code; that must stay on the ephemeral hosted pool and never touch a self-hosted runner. `gh-pages.yml` needs no such clause because it only triggers on push-to-master and `workflow_dispatch`. The repository guard in both keeps forks (which have no such runner label) from queueing forever.
+
+Both workflows fetch their own toolchains (`setup-go`, `actions-hugo`) and use no `sudo`/`apt`. Because a self-hosted image may be leaner than the hosted one, each `lint.yml` job opens with a preflight step that names any missing tool (`curl`/`unzip`/`make` for protoc and lint, `gcc`/`ldd`/`python3` for the packaging-script tests) instead of failing opaquely mid-build. If the runner image turns out to lack them, either extend the image or revert the single `runs-on:` expression per job.
+
 ## Touching the cache pipeline
 
 When changing `image_build.yml`, `backend_build.yml`, any of the `backend/Dockerfile.*` files, `Dockerfile.base-grpc-builder`, `.docker/install-base-deps.sh`, `.docker/<backend>-compile.sh`, or `scripts/changed-backends.js`:

.agents/coding-style.md

@@ -70,3 +70,37 @@ The project documentation is located in `docs/content`. When adding new features
 - **Configuration**: If you modify configuration options, update the relevant sections in `docs/content/`.
 - **Examples**: providing concrete examples (like YAML configuration blocks) is highly encouraged to help users get started quickly.
 - **Shortcodes**: Use `{{% notice note %}}`, `{{% notice tip %}}`, or `{{% notice warning %}}` for callout boxes. Do **not** use `{{% alert %}}` — that shortcode does not exist in this project's Hugo theme and will break the docs build.
+
+## React UI styling
+
+The React UI ships a design system in `core/http/react-ui/src/App.css`: design
+tokens, form grids, data tables, stat cards, callouts, plus a small semantic
+primitive layer (`.stack`, `.hstack`, `.text-note`, `.text-meta`, `.tone-*`,
+`.icon-chip`). **Use it instead of `style={{ ... }}`.** Inline styles are a
+spacing or colour decision made in one file, so no two pages end up sharing a
+rhythm, which is the main reason the app reads as unfinished.
+
+Inline styles are still correct for values that are genuinely computed at
+runtime: `width: ${pct}%`, a data-driven `background`, a tooltip's coordinates.
+Everything else belongs in a class.
+
+A ratchet enforces this:
+
+```sh
+cd core/http/react-ui
+npm run lint:inline-styles          # fails if the count went UP
+npm run lint:inline-styles:report   # per-file counts, worst first
+npm run lint:inline-styles:write    # refresh the baseline after converting
+```
+
+The gate also fails on **duplicate `className` attributes on one element**. JSX
+keeps the last and silently drops the first, so `<i className={icon}
+className="text-xs" />` loses its icon while passing lint, the build and the e2e
+suite. Converting a style to a class on an element that already has a
+`className` is the usual way to introduce one; merge them into a single
+attribute instead.
+
+When converting a page, prefer naming the shapes it actually has
+(`.p2p-diagram`, `.usage-tile`) over adding more utilities, and check whether an
+existing block already covers it: the Nodes page reuses the P2P setup shapes,
+and Model Editor reuses the Settings section rail.

.agents/preparing-a-release.md

@@ -0,0 +1,26 @@
+# Preparing a Release
+
+A release is not finished when the tag is pushed. The GitHub release, the blog post and the demo clips ship together, because the changelog says what moved and the post and the clips are what make anyone care.
+
+## What a release must include
+
+1. **Labels on the merged PRs.** GitHub generates the raw notes from PR labels, so label first, generate second. Wrong labels mean a miscategorised changelog that has to be edited by hand.
+2. **`RELEASE_NOTES_vX.Y.Z.md`** at the repository root, in the house style: what changed, why it matters, PR numbers so people can read the diffs.
+3. **A blog post under `website/content/blog/`.** One post per release, front matter with `title`, `date`, `author`, `category: "Release"`, `tags`, `summary` and `extracss: ["blog.css"]`. Cover the two or three changes that alter what a user does day to day, not the whole changelog, and link the PR numbers. See `website/content/blog/what-landed-in-localai-4-8.md` for the shape.
+4. **Demo clips for the notable features.** Anything visible (a new backend, a UI change, a new endpoint, a measured speedup) gets a short screen recording. Put the file in `website/static/media/`, reference it from the blog post, and reuse it on the marketing pages where it fits.
+
+A release without a post and without clips is incomplete, in the same way a user-facing code change without a docs update is incomplete.
+
+## Clip conventions
+
+- MP4, H.264, no audio track unless the feature is about audio. Keep them short (10 to 30 seconds) and loopable.
+- Record the real thing. A clip from the engine's own benchmark suite or a real session, never a mockup.
+- Where the change is a speedup, record both sides on the same machine on the same input, so the comparison is honest.
+- Name the file after the feature, not the release (`vllm-race.mp4`, not `v4-8-demo.mp4`), so it stays reusable once the release is old.
+- The marketing site plays clips with `muted loop playsinline preload="none"` and a `data-lazy` attribute, which the site's IntersectionObserver uses to play and pause them on scroll. Follow that pattern for anything you add.
+
+## Order of work
+
+Label the PRs, generate and edit the release notes, cut the draft release, record the clips while the branch is still fresh in your head, then write the post against the notes and the clips. Publishing the release and merging the post should happen on the same day.
+
+The `creating-localai-releases` skill drives steps 1 to 3 and captures the React UI screenshots that go into the notes.

.agents/vllm-backend.md

@@ -21,6 +21,17 @@ options:
   - reasoning_parser:qwen3
 ```
 
+## `Options[]` doubles as CLI-style engine flags
+
+Beyond the parser names above, `Options[]` carries `--` prefixed engine flags (`--enable-prefix-caching`, `--kv-cache-dtype:fp8_e5m2`). `apply_options_to_engine_args` in `backend/python/common/vllm_utils.py` maps them onto `AsyncEngineArgs` fields, and it must run **before** `AsyncLLMEngine.from_engine_args()` - applying them afterwards is a silent no-op, which is exactly what issue #11130 was.
+
+Things to keep straight when touching this:
+
+- Precedence is typed proto fields → `options:` → `engine_args:`. `applyEngineArgDefaults` in `core/config/hooks_vllm.go` therefore skips seeding a production default whose key the user already set as an option, otherwise the later `engine_args:` pass would silently override them.
+- Only `--` prefixed entries are engine flags; `tool_parser:`/`reasoning_parser:` and friends keep their meaning. Parser lookups accept both spellings via `normalize_option_key`.
+- Unknown or uncoercible flags warn and are skipped, unlike `engine_args:` which is strict - `Options[]` is a shared bag and knows entries this mapping doesn't.
+- Field types come from the annotation's *base* (`Literal["auto","float16"]` is not a float). The helper's tests are stdlib-only: `make test-python-helpers`.
+
 Auto-defaults for known model families live in `core/config/parser_defaults.json` and are applied:
 - at gallery import time by `core/gallery/importers/vllm.go`
 - at model load time by the `vllm` / `vllm-omni` backend hook in `core/config/hooks_vllm.go`

.docker/llama-cpp-compile.sh

@@ -28,6 +28,10 @@ if [ -z "${BUILD_TYPE:-}" ]; then
   # variants with it (the host never *selects* SME unless it has it, but every variant must
   # still compile).
   if [ "${TARGETARCH}" = "arm64" ]; then
+    # The prebuilt base inherits default ports.ubuntu.com sources; honor the
+    # APT_*_MIRROR build args here like the from-source path does, so this
+    # apt step survives a mirror outage.
+    sh /LocalAI/.docker/apt-mirror.sh || true
     apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
     export CC=gcc-14 CXX=g++-14
   fi

.dockerignore

@@ -40,6 +40,16 @@ backend/cpp/privacy-filter/build
 backend/cpp/privacy-filter/grpc-server
 backend/cpp/privacy-filter/package
 
+# audio-cpp: same in-place pattern. The Makefile clones audio.cpp at the pinned
+# AUDIO_CPP_VERSION and the `audio.cpp:` target is the directory itself, so a
+# stale host checkout COPY'd in makes the build compile against whatever commit
+# the host had. build/ is worse than stale: its CMakeCache.txt records the host
+# source, prefix and compiler paths, and cmake refuses to reconfigure from it.
+backend/cpp/audio-cpp/audio.cpp
+backend/cpp/audio-cpp/build
+backend/cpp/audio-cpp/grpc-server
+backend/cpp/audio-cpp/package
+
 # Rust backend build output (sources are tracked; target/ is generated)
 backend/rust/*/target
 

.githooks/pre-commit

@@ -1,72 +0,0 @@
-#!/usr/bin/env sh
-#
-# LocalAI pre-commit hook. Install it (once per clone) with:
-#
-#     make install-hooks
-#
-# Runs only the checks relevant to what's staged:
-#   - Go files          -> make lint + make test-coverage-check
-#   - core/http/react-ui -> make test-ui-coverage-check (Playwright e2e + gate)
-#   - realtime state machines / specs -> make test-realtime-conformance
-#       (respcoord/**, turncoord/**, or formal-verification/** -- a pure .fizz
-#        spec edit must still re-verify the design, detected separately from Go)
-# A commit touching none of these is skipped entirely (other docs/YAML can't
-# change lint findings, Go coverage, the UI, or the realtime conformance gate).
-#
-# To bypass for a single commit (e.g. a WIP checkpoint): git commit --no-verify
-set -eu
-
-repo_root="$(git rev-parse --show-toplevel)"
-cd "$repo_root"
-
-staged="$(git diff --cached --name-only --diff-filter=ACMRD)"
-
-go_changed=0
-ui_changed=0
-rt_changed=0
-if echo "$staged" | grep -qE '\.go$'; then go_changed=1; fi
-if echo "$staged" | grep -qE '^core/http/react-ui/'; then ui_changed=1; fi
-if echo "$staged" | grep -qE '^(core/http/endpoints/openai/(coordinator|respcoord|turncoord|conncoord|compactcoord|ttscoord)/|formal-verification/)'; then rt_changed=1; fi
-
-if [ "$go_changed" -eq 0 ] && [ "$ui_changed" -eq 0 ] && [ "$rt_changed" -eq 0 ]; then
-	echo "pre-commit: no Go, React UI, or realtime-spec changes staged — skipping."
-	exit 0
-fi
-
-if [ "$go_changed" -eq 1 ]; then
-	# Resolve the ref golangci-lint's new-from-merge-base should compare
-	# against. .golangci.yml pins origin/master, which is correct in CI
-	# (origin == the canonical repo) but wrong from a fork clone, where
-	# origin/master lags behind and lint would report the whole upstream
-	# backlog. Prefer upstream/master, then origin/master, then master.
-	lint_base=""
-	for ref in upstream/master origin/master master; do
-		if git rev-parse --verify --quiet "${ref}^{commit}" >/dev/null 2>&1; then
-			lint_base="$ref"
-			break
-		fi
-	done
-
-	echo "pre-commit ▶ golangci-lint (make lint${lint_base:+, new-from $lint_base})"
-	make lint LINT_NEW_FROM="$lint_base"
-
-	echo "pre-commit ▶ coverage gate (make test-coverage-check) — builds and runs the"
-	echo "             pkg/core suites plus tests/e2e; can take a few minutes."
-	make test-coverage-check
-fi
-
-if [ "$ui_changed" -eq 1 ]; then
-	echo "pre-commit ▶ React UI e2e + coverage gate (make test-ui-coverage-check) —"
-	echo "             rebuilds the UI + ui-test-server, runs the Playwright specs, and"
-	echo "             fails if line coverage regressed; can take a couple of minutes."
-	make test-ui-coverage-check
-fi
-
-if [ "$rt_changed" -eq 1 ]; then
-	echo "pre-commit ▶ realtime state-machine conformance (make test-realtime-conformance) —"
-	echo "             Go transition/rapid tests under -race + FizzBee model check of the"
-	echo "             authoritative specs. Fail-closed: needs FizzBee (make install-fizzbee)."
-	make test-realtime-conformance
-fi
-
-echo "pre-commit ✓ all relevant checks passed"

.github/backend-matrix.yml

@@ -66,6 +66,34 @@ include:
     dockerfile: "./backend/Dockerfile.python"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-kokoro'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "kokoro"
+    dockerfile: "./backend/Dockerfile.python"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-kokoro'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "kokoro"
+    dockerfile: "./backend/Dockerfile.python"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -728,6 +756,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "8"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-12-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "8"
@@ -871,6 +912,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "8"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-12-magpie-tts-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "8"
@@ -1675,6 +1729,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-13-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "13"
     cuda-minor-version: "0"
@@ -1688,6 +1755,19 @@ include:
     backend: "stablediffusion-ggml"
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/arm64'
+    skip-drivers: 'false'
+    tag-latest: 'auto'
+    tag-suffix: '-nvidia-l4t-cuda-13-arm64-trellis2cpp'
+    base-image: "ubuntu:24.04"
+    ubuntu-version: '2404'
+    runs-on: 'ubuntu-24.04-arm'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
   - build-type: 'cublas'
     cuda-major-version: "13"
     cuda-minor-version: "0"
@@ -1935,6 +2015,32 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-13-magpie-tts-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-13-vllm-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "vllm-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "13"
     cuda-minor-version: "0"
@@ -2000,6 +2106,32 @@ include:
     backend: "moss-tts-cpp"
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-nvidia-l4t-cuda-13-arm64-vllm-cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "vllm-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/arm64'
+    skip-drivers: 'false'
+    tag-latest: 'auto'
+    tag-suffix: '-nvidia-l4t-cuda-13-arm64-magpie-tts-cpp'
+    base-image: "ubuntu:24.04"
+    ubuntu-version: '2404'
+    runs-on: 'ubuntu-24.04-arm'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
   - build-type: 'cublas'
     cuda-major-version: "13"
     cuda-minor-version: "0"
@@ -2963,6 +3095,19 @@ include:
     dockerfile: "./backend/Dockerfile.privacy-filter"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'vulkan'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-vulkan-vllm-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "vllm-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   # Vulkan: base-grpc-vulkan-amd64 carries the SDK. arm64 vulkan is a one-line
   # add once amd64 is proven in CI.
   - build-type: 'vulkan'
@@ -2998,6 +3143,97 @@ include:
     dockerfile: "./backend/Dockerfile.privacy-filter"
     context: "./"
     ubuntu-version: '2404'
+  # audio-cpp: 0xShug0/audio.cpp, a multi-family ggml audio engine (TTS, ASR,
+  # VAD, diarization, source separation, music generation).
+  #
+  # These entries deliberately carry NO builder-base-image, unlike the
+  # privacy-filter and llama-cpp blocks above. The prebuilt
+  # quay.io/go-skynet/ci-cache:base-grpc-* images ship a from-source gRPC whose
+  # protobuf is v26, and protobuf has depended on abseil since v22. audio.cpp
+  # links sentencepiece with SPM_PROTOBUF_PROVIDER=package (needed to stop
+  # sentencepiece's vendored protobuf 3.14 from colliding with the 3.21 the
+  # generated backend.pb.cc is built against, which broke every nested-message
+  # parse), so sentencepiece then sees real abseil's
+  # `absl::lts_20240116::internal` alongside its own vendored plain
+  # `absl::internal` and every `absl::internal::` reference becomes ambiguous.
+  # Verified, not theorised: building against base-grpc-amd64 fails at
+  # sentencepiece-static.dir/error.cc.o with "reference to 'internal' is
+  # ambiguous". Dockerfile.audio-cpp therefore installs Ubuntu Noble's apt
+  # gRPC/protobuf 3.21.12 itself and has a single `builder` stage, so the
+  # BUILDER_BASE_IMAGE / BUILDER_TARGET / SKIP_DRIVERS build-args are never
+  # consumed. Same reason CUDA needs its toolkit in base-image rather than in a
+  # builder image: this is the ds4 shape, not the llama-cpp one.
+  #
+  # No ROCm entry: upstream has no HIP configuration. No CUDA arm64 or L4T
+  # entry: upstream documents and validates CUDA on x86 only. Darwin/Metal is in
+  # the includeDarwin matrix below, built by scripts/build/audio-cpp-darwin.sh.
+  #
+  # No vulkan entry either, though Dockerfile.audio-cpp and the backend Makefile
+  # both handle BUILD_TYPE=vulkan for local builds. Every other vulkan backend
+  # gets its Mesa ICD drivers from .docker/install-base-deps.sh, which installs
+  # mesa-vulkan-drivers so package-gpu-libs.sh can bundle them; this Dockerfile
+  # calls neither, so the image would ship a Vulkan loader that finds no GPU. No
+  # CI job runs a vulkan image against real hardware, so it would pass green and
+  # fail in users' hands. The entry comes back once the ICD question is settled.
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-audio-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-audio-cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
+  # cuda-major-version is forwarded into the build (Dockerfile.audio-cpp -> the
+  # backend Makefile) and picks the CMAKE_CUDA_ARCHITECTURES list, which upstream
+  # otherwise sets to `native` and no CI runner can enumerate. cuda-minor-version
+  # and the base-image tag encode the same toolkit and must move together;
+  # nothing checks that for you.
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "8"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-12-audio-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "nvidia/cuda:12.8.1-devel-ubuntu24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-13-audio-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "nvidia/cuda:13.0.0-devel-ubuntu24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -3161,6 +3397,35 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  # trellis2cpp
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-trellis2cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   # sam3-cpp
   - build-type: ''
     cuda-major-version: ""
@@ -3486,6 +3751,34 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'vulkan'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-vulkan-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: 'vulkan'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-vulkan-trellis2cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "0"
@@ -3499,6 +3792,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2204'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "0"
+    platforms: 'linux/arm64'
+    skip-drivers: 'false'
+    tag-latest: 'auto'
+    tag-suffix: '-nvidia-l4t-arm64-trellis2cpp'
+    base-image: "nvcr.io/nvidia/l4t-jetpack:r36.4.0"
+    runs-on: 'ubuntu-24.04-arm'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2204'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "0"
@@ -4583,6 +4889,20 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-magpie-tts-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -4597,7 +4917,50 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  # vllm-cpp
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-vllm-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "vllm-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-vllm-cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "vllm-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   # omnivoice-cpp
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-magpie-tts-cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -4652,6 +5015,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'sycl_f32'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-intel-sycl-f32-magpie-tts-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "intel/oneapi-basekit:2025.3.0-0-devel-ubuntu24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'sycl_f32'
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -4691,6 +5067,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'sycl_f16'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-intel-sycl-f16-magpie-tts-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "intel/oneapi-basekit:2025.3.0-0-devel-ubuntu24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'sycl_f16'
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -4732,6 +5121,20 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'vulkan'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-vulkan-magpie-tts-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'vulkan'
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -4774,6 +5177,20 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'vulkan'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-vulkan-magpie-tts-cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'vulkan'
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -4814,6 +5231,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2204'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "0"
+    platforms: 'linux/arm64'
+    skip-drivers: 'false'
+    tag-latest: 'auto'
+    tag-suffix: '-nvidia-l4t-arm64-magpie-tts-cpp'
+    base-image: "nvcr.io/nvidia/l4t-jetpack:r36.4.0"
+    runs-on: 'ubuntu-24.04-arm'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2204'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "0"
@@ -4853,6 +5283,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'hipblas'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-rocm-hipblas-magpie-tts-cpp'
+    base-image: "rocm/dev-ubuntu-24.04:6.4.4"
+    runs-on: 'ubuntu-latest'
+    skip-drivers: 'false'
+    backend: "magpie-tts-cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'hipblas'
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -5193,6 +5636,35 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  # valkey-store
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-valkey-store'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "valkey-store"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-valkey-store'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "valkey-store"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   # rfdetr
   - build-type: ''
     cuda-major-version: ""
@@ -5734,6 +6206,10 @@ includeDarwin:
     tag-suffix: "-metal-darwin-arm64-stablediffusion-ggml"
     build-type: "metal"
     lang: "go"
+  - backend: "trellis2cpp"
+    tag-suffix: "-metal-darwin-arm64-trellis2cpp"
+    build-type: "metal"
+    lang: "go"
   - backend: "whisper"
     tag-suffix: "-metal-darwin-arm64-whisper"
     build-type: "metal"
@@ -5774,6 +6250,14 @@ includeDarwin:
     tag-suffix: "-metal-darwin-arm64-moss-tts-cpp"
     build-type: "metal"
     lang: "go"
+  - backend: "magpie-tts-cpp"
+    tag-suffix: "-metal-darwin-arm64-magpie-tts-cpp"
+    build-type: "metal"
+    lang: "go"
+  - backend: "vllm-cpp"
+    tag-suffix: "-metal-darwin-arm64-vllm-cpp"
+    build-type: "metal"
+    lang: "go"
   - backend: "omnivoice-cpp"
     tag-suffix: "-metal-darwin-arm64-omnivoice-cpp"
     build-type: "metal"
@@ -5807,6 +6291,18 @@ includeDarwin:
   - backend: "privacy-filter"
     tag-suffix: "-metal-darwin-arm64-privacy-filter"
     lang: "go"
+  # audio-cpp is the same shape: a C++/ggml backend built by a bespoke darwin
+  # script (make backends/audio-cpp-darwin), which reuses the backend's own
+  # package.sh so the Darwin package keeps the root-level layout the Linux image
+  # has (grpc-server, run.sh and assets/ in one directory, dylibs in lib/).
+  # No build-type: the backend Makefile turns ENGINE_ENABLE_METAL on from
+  # uname -s. lang=go drives runner/toolchain selection only - there is no
+  # backend/go/audio-cpp, which is why backend_build_darwin.yml and
+  # DARWIN_BESPOKE_BUILDERS in scripts/lib/backend-filter.mjs both route this
+  # backend away from the generic Go path.
+  - backend: "audio-cpp"
+    tag-suffix: "-metal-darwin-arm64-audio-cpp"
+    lang: "go"
   # LocalVQE has no Metal path; on Apple Silicon it builds CPU-only (GGML_METAL
   # OFF) but is still a native arm64 image. Uses the darwin/metal build profile.
   - backend: "localvqe"
@@ -5903,6 +6399,10 @@ includeDarwin:
     tag-suffix: "-metal-darwin-arm64-cloud-proxy"
     build-type: "metal"
     lang: "go"
+  - backend: "valkey-store"
+    tag-suffix: "-metal-darwin-arm64-valkey-store"
+    build-type: "metal"
+    lang: "go"
   - backend: "llama-cpp-quantization"
     tag-suffix: "-metal-darwin-arm64-llama-cpp-quantization"
     build-type: "mps"

.github/bump_deps.sh

@@ -1,5 +1,8 @@
 #!/bin/bash
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1
 BRANCH=$2
 VAR=$3
@@ -9,10 +12,11 @@ if [ -z "$FILE" ]; then
     FILE="Makefile"
 fi
 
-# -L so a renamed/transferred upstream repo (GitHub answers 301) still
-# resolves instead of handing us the redirect body, and -f so an HTTP error
-# aborts the run rather than letting an error page reach sed below.
-LAST_COMMIT=$(curl -sfL -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH")
+# gh_curl follows redirects so a renamed/transferred upstream repo (GitHub
+# answers 301) still resolves, and fails on HTTP errors rather than letting an
+# error page reach sed below. `|| true` keeps a failed lookup from aborting the
+# script at exit 22 with no context — the SHA guard below reports it instead.
+LAST_COMMIT=$(gh_curl -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH" || true)
 
 # Guard the sed input: anything that is not a bare 40-hex SHA (an API error
 # body, an empty response) would otherwise be spliced into the Makefile pin —

.github/bump_docs.sh

@@ -1,7 +1,18 @@
 #!/bin/bash
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1
 
-LATEST_TAG=$(curl -s "https://api.github.com/repos/$REPO/releases/latest" | jq -r '.tag_name')
+LATEST_TAG=$(gh_curl -H "Accept: application/vnd.github+json" \
+    "https://api.github.com/repos/$REPO/releases/latest" | jq -r '.tag_name')
+
+# jq prints the string "null" for a missing key, so a throttled or otherwise
+# unexpected API response would otherwise be published as the docs version.
+if [ -z "$LATEST_TAG" ] || [ "$LATEST_TAG" = "null" ]; then
+    echo "Refusing to bump docs version: could not resolve the latest release tag for $REPO." >&2
+    exit 1
+fi
 
 cat <<< $(jq ".version = \"$LATEST_TAG\"" docs/data/version.json) > docs/data/version.json

.github/bump_vllm_metal.sh

@@ -11,6 +11,9 @@
 # darwin build can only use the exact vLLM version vllm-metal supports, so it may
 # lag the Linux pin (requirements-cublas13-after.txt) until vllm-metal catches up.
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1   # vllm-project/vllm-metal
 FILE=$2   # backend/python/vllm/install.sh
 VAR=$3    # VLLM_METAL_VERSION (used for the workflow's output file names)
@@ -22,12 +25,12 @@ fi
 
 # vllm-metal ships frequent dev releases, all flagged as non-prerelease, so
 # /releases/latest returns the newest one (with its cp312 wheel asset).
-LATEST_TAG=$(curl -sS -H "Accept: application/vnd.github+json" \
+LATEST_TAG=$(gh_curl -H "Accept: application/vnd.github+json" \
     "https://api.github.com/repos/$REPO/releases/latest" \
     | python3 -c "import json,sys; print(json.load(sys.stdin)['tag_name'])")
 
 # The coupled vLLM source version lives in vllm-metal's installer at that tag.
-NEW_VLLM_VERSION=$(curl -fsSL \
+NEW_VLLM_VERSION=$(gh_curl \
     "https://raw.githubusercontent.com/$REPO/$LATEST_TAG/install.sh" \
     | grep -oE 'vllm_v="[0-9]+\.[0-9]+\.[0-9]+"' | head -1 | cut -d'"' -f2)
 

.github/bump_vllm_wheel.sh

@@ -9,6 +9,9 @@
 # vars in Makefiles; this script handles the two-value rewrite specific to the
 # vLLM requirements file.
 set -xe
+
+source "$(dirname "${BASH_SOURCE[0]}")/gh_curl.sh"
+
 REPO=$1   # vllm-project/vllm
 FILE=$2   # backend/python/vllm/requirements-cublas13-after.txt
 VAR=$3    # VLLM_VERSION (used for output file names so the workflow can read them)
@@ -19,7 +22,7 @@ if [ -z "$FILE" ] || [ -z "$REPO" ] || [ -z "$VAR" ]; then
 fi
 
 # /releases/latest returns the most recent non-prerelease tag.
-LATEST_TAG=$(curl -sS -H "Accept: application/vnd.github+json" \
+LATEST_TAG=$(gh_curl -H "Accept: application/vnd.github+json" \
     "https://api.github.com/repos/$REPO/releases/latest" \
     | python3 -c "import json,sys; print(json.load(sys.stdin)['tag_name'])")
 

.github/ci/apexentries/README.md

@@ -0,0 +1,194 @@
+# apexentries
+
+Generates gallery entries for the `mudler/*-APEX-GGUF` HuggingFace repositories.
+
+Each APEX repo becomes one **family**: one entry per quality rung the repo
+publishes and one per quantization rung its unsloth counterpart publishes, all
+gathered under the **base model's** entry. LocalAI's variant selector then picks
+the build that fits the hardware in front of it.
+
+## The hub is the base model entry, never a generated `*-apex` parent
+
+Somebody looking for `qwen3.6-35b-a3b` must find every build of those weights
+under that one name: the APEX imatrix rungs, the unsloth quant rungs and any
+speculative build. A separate `qwen3.6-35b-a3b-apex` hub competing with the base
+entry would split the family in two and leave whichever half the user did not
+search for effectively invisible.
+
+So the generator resolves the hub by stripping the `-APEX`, `-MTP` and `-TQ`
+markers and looking the result up in the index, trying both the repo-derived and
+the stem-derived candidate the same way `CounterpartCandidates` does. Then:
+
+- **The hub exists** (14 of the 45 repos, resolving to 10 distinct entries).
+  Nothing new is emitted for the family root. A `variants:` block is spliced into
+  the entry that is already there, textually, leaving its description, icon,
+  tags, overrides and files untouched. The line editing is shared with the
+  `variantproposals` job via `.github/ci/galleryedit`.
+- **The hub is absent** (the other 31). A new hub is emitted, named for the base
+  model and never for the APEX repo. It carries one of the discovered builds as
+  its own payload so it is a complete installable entry rather than a bare index,
+  and that payload is what gives it an `overrides.backend`. Without a declared
+  backend the verifier would skip it, so a hub carrying feature tags would escape
+  the tagging check in silence.
+
+Several APEX repos routinely resolve to one base model, so both paths accumulate
+by hub name rather than assuming one family per hub.
+
+Two references are always filtered out of a hub's list: anything the entry
+already declares, and the hub's own name. The self reference is not merely
+redundant. An unsloth rung whose weights the gallery already ships under the base
+name resolves, through the merge, straight back to the hub, and the verifier
+reads a self reference as a variant that declares variants of its own.
+
+The four hand-written `*-apex` entries (`qwen3.6-35b-a3b-apex`,
+`gemma-4-26b-a4b-it-apex`, `qwen3.5-35b-a3b-apex`,
+`nemotron-3-nano-omni-30b-a3b-reasoning-apex`) are **ordinary builds**, not hubs.
+They are referenced from their hub's variants list like any other rung, and are
+never deleted or renamed.
+
+## Flags
+
+| Flag | Default | Meaning |
+|------|---------|---------|
+| `-index <path>` | `gallery/index.yaml` | Gallery index to dedup against. Read only, unless `-apply` is passed. |
+| `-only <a,b,c>` | (all) | Comma-separated full repo names (`mudler/Foo-APEX-GGUF`) to restrict generation to. A name that matches nothing is reported as a warning, since it is a typo rather than an empty result. |
+| `-out <path>` | (none) | Write the entries to add to this file. Nothing is written to the gallery. |
+| `-apply` | `false` | Splice the variants into `-index` and append the new entries to it. |
+| `-verify <path>` | (none) | Verify a gallery index and exit. Ignores every other flag. |
+
+Either `-out` or `-apply` is required, otherwise the run has nothing to do.
+
+`-apply` splices variant lines into existing entries and **appends** new ones. It
+never re-serialises the index: it is roughly 40,000 lines, and a YAML round trip
+would reflow the whole file, drop the anchors and merge keys the gallery relies
+on, and produce a diff nobody can review. On the three-family sample the splice
+is 24 added lines across 3 hunks with zero deletions.
+
+## Discovery is by filename suffix, never by repo name
+
+Builds come from the files a repo actually publishes. A filename is never
+constructed from a repo name, because the two disagree:
+`mudler/gemma-4-26B-A4B-it-APEX-GGUF` ships `gemma-4-26B-A4B-APEX-*.gguf`, and
+five other repos likewise drop a suffix (`-it`, `-2603`) or a vendor prefix
+(`NVIDIA-`) that the repo name carries. Composing a URL from the repo name would
+produce a 404 for every one of them, and the 404 would only surface after the
+entry shipped.
+
+The quality ladder is matched on the trailing tier marker, `-(I-)?(Quality|
+Balanced|Compact|Mini|Nano).gguf`. The `I-` prefix marks the imatrix ladder. The
+imatrix ladder is emitted when it is non-empty and the plain ladder is used only
+as a fallback, because two of the 45 repos publish no imatrix tiers at all and
+must still contribute. Eleven repos carry a fifth `I-Nano` rung, so nothing
+assumes a fixed number of rungs.
+
+Every run prints, per repo, the counts that discovery accounted for. If the
+number of classified files is short of the number of `.gguf` files the repo
+publishes, the shortfall is printed as `UNCLASSIFIED`. That check is a set
+difference on counts rather than a second pass over filenames: a second matcher
+would duplicate the tier regex and the two copies would drift. The failure it
+catches is quiet. A publishing-script typo that breaks every imatrix filename in
+a repo does not produce a short ladder; it makes the imatrix ladder empty, and
+the fallback then downgrades the whole family to the plain ladder with nothing
+said. A downstream HTTP check cannot catch it either, because it validates the
+URLs that were emitted, and an undiscovered tier emits none.
+
+The same reasoning applies to `UNACCOUNTED QUANT`, printed when the unsloth
+counterpart demonstrably publishes a wanted quant that produced no build. It is
+reported at discovery time because a dropped quant leaves no trace at all in the
+finished gallery file.
+
+## sha256 always comes from the API
+
+Every file stanza takes its `sha256` from the HuggingFace models API
+(`lfs.sha256`). A GGUF the API describes without one is a fatal error for that
+family: the repo is reported by name and the run ends non-zero. It is never
+substituted from another field, because that is exactly how a Xet hash ends up
+masquerading as a content hash.
+
+## The dflash / mtp tagging rule
+
+An entry is tagged `dflash` or `mtp` **if and only if** it configures the
+matching `spec_type:draft-<feature>`. Variant ranking reads tags and nothing
+else, so a tag that does not match the configuration either promotes a build
+that is no faster or hides one that genuinely is.
+
+A repo name is not configuration. `mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF` ships
+weights that carry MTP heads; an entry that does not enable them is not an MTP
+entry and is not tagged as one.
+
+A generated hub inherits the tags of the build it carries as its payload, rather
+than rebuilding them from the base set, so a hub whose payload configures a
+`spec_type` stays tagged consistently with the overrides copied alongside it.
+
+## Reuse reporting: two categories, not one
+
+Generated entries are deduped against the gallery and against the batch itself.
+The run prints the result under two separate headings, because the two cases are
+not equivalent:
+
+- **URI MATCHES** mean the gallery, or an earlier entry in this batch, already
+  ships exactly these weights. Pointing the hub at the existing entry is correct
+  and needs no thought.
+- **NAME COLLISIONS** mean an entry already owns the name but holds different
+  weights. Referencing it would point the hub at a build other than the one
+  generated. Every one of these must be inspected by hand.
+
+The run then prints `HUBS SPLICED`, listing every reference that will be added to
+an entry the gallery already ships along with the line it will be added at, and
+`HUBS CREATED` for the families that get a new hub. The splices are the part a
+review has to read closely, because they modify entries somebody else wrote.
+
+Hubs are deliberately kept out of the merge. A new hub carries the family's top
+rung as its own payload, so URI dedup would fold the hub into that rung and the
+family would lose the very entry point this command exists to create.
+
+## Workflow: sample first, then the full set
+
+Never run the full generation straight into the gallery. Generate a small,
+deliberately awkward sample, have it reviewed, then run the rest.
+
+```bash
+# 1. Sample three families that between them cover the awkward shapes:
+#    a standard four-rung repo, one with the extra I-Nano rung AND a file stem
+#    that differs from its repo name, and one whose unsloth counterpart shards
+#    its quants across subdirectories.
+go run ./.github/ci/apexentries \
+  -index gallery/index.yaml \
+  -only mudler/Qwen3.6-35B-A3B-APEX-GGUF,mudler/gemma-4-26B-A4B-it-APEX-GGUF,mudler/Step-3.7-Flash-APEX-GGUF \
+  -out /tmp/sample.yaml
+
+# 2. Verify the sample against the gallery it would join, splices included. Apply
+#    to a COPY, never to the real index, and check that the diff is only the
+#    intended variant lines. Compare the verifier output to the gallery's own
+#    baseline: what matters is that the sample adds no new problem, not that the
+#    total is zero.
+cp gallery/index.yaml /tmp/index-copy.yaml
+go run ./.github/ci/apexentries -index /tmp/index-copy.yaml -only <same list> -apply
+diff -u gallery/index.yaml /tmp/index-copy.yaml   # expect zero deletions
+
+go run ./.github/ci/apexentries -verify gallery/index.yaml > /tmp/baseline.log 2>&1
+go run ./.github/ci/apexentries -verify /tmp/index-copy.yaml > /tmp/spliced.log 2>&1
+diff /tmp/baseline.log /tmp/spliced.log
+
+# 3. Have a human review /tmp/sample.yaml and every reported name collision.
+
+# 4. Only then, the full set.
+go run ./.github/ci/apexentries -index gallery/index.yaml -apply
+```
+
+## Tests
+
+```bash
+go test ./.github/ci/apexentries/
+```
+
+The shared line editor has its own package:
+
+```bash
+go test ./.github/ci/galleryedit/
+```
+
+`.github/ci/` is invisible to `go list ./...`, so these specs are not covered by
+`make lint` or the repository test run. `.github/workflows/ci-tools-tests.yaml`
+names the package explicitly; keep that workflow in step with any package added
+under `.github/ci/`.

.github/ci/apexentries/discover.go

@@ -0,0 +1,70 @@
+package main
+
+import (
+	"regexp"
+	"strings"
+)
+
+// tierRE matches the tier marker APEX repos put at the end of a weight
+// filename. Discovery is by suffix because the stem is not predictable from
+// the repo name: six of the 45 repos drop a suffix ("-it", "-2603") or a
+// vendor prefix ("NVIDIA-") that the repo name carries.
+var tierRE = regexp.MustCompile(`-(I-)?(Quality|Balanced|Compact|Mini|Nano)\.gguf$`)
+
+// fullPrecisionRE matches the unquantized source weights an APEX repo publishes
+// alongside its ladder, flat (-F16.gguf) or sharded across a numbered set
+// (-F16-00001-of-00010.gguf). bf16 is accepted because some repos publish that
+// instead, and the match is case-insensitive because the casing varies between
+// publishing scripts.
+//
+// These are deliberately not tiers: they are the weights the ladder is quantized
+// FROM, and generation is scoped to the ladder itself.
+var fullPrecisionRE = regexp.MustCompile(`(?i)-b?f16(-\d{5}-of-\d{5})?\.gguf$`)
+
+// IsFullPrecision reports whether a weight filename is an unquantized source.
+func IsFullPrecision(name string) bool {
+	return fullPrecisionRE.MatchString(name)
+}
+
+// Tier is one discovered build of an APEX repo.
+type Tier struct {
+	Label string
+	File  GGUFFile
+}
+
+// DiscoverAPEXTiers splits a repo's weight files into the imatrix ladder and
+// the plain ladder. mmproj files are never tiers.
+func DiscoverAPEXTiers(files []GGUFFile) (imatrix, plain []Tier) {
+	for _, f := range files {
+		if strings.HasPrefix(f.Name, "mmproj") {
+			continue
+		}
+		m := tierRE.FindStringSubmatch(f.Name)
+		if m == nil {
+			continue
+		}
+		if m[1] != "" {
+			imatrix = append(imatrix, Tier{Label: "I-" + m[2], File: f})
+			continue
+		}
+		plain = append(plain, Tier{Label: m[2], File: f})
+	}
+	return imatrix, plain
+}
+
+// DiscoverMMProj returns the repo's projector file, if it publishes one. The
+// name varies across repos (mmproj.gguf, mmproj-F16.gguf,
+// mmproj-step3.7-flash-f16.gguf), so match the prefix rather than a fixed name.
+func DiscoverMMProj(files []GGUFFile) (GGUFFile, bool) {
+	for _, f := range files {
+		if strings.HasPrefix(f.Name, "mmproj") {
+			return f, true
+		}
+	}
+	return GGUFFile{}, false
+}
+
+// FileStem returns a tier's filename with its tier suffix removed.
+func FileStem(t Tier) string {
+	return tierRE.ReplaceAllString(t.File.Name, "")
+}

.github/ci/apexentries/discover_test.go

@@ -0,0 +1,68 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("DiscoverAPEXTiers", func() {
+	It("finds tiers regardless of how the stem relates to the repo name", func() {
+		// This repo is mudler/gemma-4-26B-A4B-it-APEX-GGUF but its files drop "-it".
+		files := []GGUFFile{
+			{Name: "gemma-4-26B-A4B-APEX-I-Quality.gguf", SHA256: "a"},
+			{Name: "gemma-4-26B-A4B-APEX-I-Nano.gguf", SHA256: "b"},
+			{Name: "gemma-4-26B-A4B-APEX-Quality.gguf", SHA256: "c"},
+			{Name: "mmproj-F16.gguf", SHA256: "d"},
+		}
+
+		imatrix, plain := DiscoverAPEXTiers(files)
+
+		Expect(labels(imatrix)).To(ConsistOf("I-Quality", "I-Nano"))
+		Expect(labels(plain)).To(ConsistOf("Quality"))
+	})
+
+	It("excludes mmproj from the tier list", func() {
+		files := []GGUFFile{{Name: "mmproj.gguf", SHA256: "d"}}
+
+		imatrix, plain := DiscoverAPEXTiers(files)
+
+		Expect(imatrix).To(BeEmpty())
+		Expect(plain).To(BeEmpty())
+	})
+})
+
+var _ = Describe("DiscoverMMProj", func() {
+	It("finds an mmproj whatever its suffix", func() {
+		files := []GGUFFile{
+			{Name: "Model-APEX-I-Mini.gguf", SHA256: "a"},
+			{Name: "mmproj-step3.7-flash-f16.gguf", SHA256: "b"},
+		}
+
+		got, ok := DiscoverMMProj(files)
+
+		Expect(ok).To(BeTrue())
+		Expect(got.Name).To(Equal("mmproj-step3.7-flash-f16.gguf"))
+	})
+
+	It("reports absence when the repo ships none", func() {
+		_, ok := DiscoverMMProj([]GGUFFile{{Name: "Model-APEX-Quality.gguf", SHA256: "a"}})
+
+		Expect(ok).To(BeFalse())
+	})
+})
+
+var _ = Describe("FileStem", func() {
+	It("strips the tier suffix", func() {
+		t := Tier{Label: "I-Quality", File: GGUFFile{Name: "gemma-4-26B-A4B-APEX-I-Quality.gguf"}}
+
+		Expect(FileStem(t)).To(Equal("gemma-4-26B-A4B-APEX"))
+	})
+})
+
+func labels(ts []Tier) []string {
+	out := make([]string, 0, len(ts))
+	for _, t := range ts {
+		out = append(out, t.Label)
+	}
+	return out
+}

.github/ci/apexentries/hf.go

@@ -0,0 +1,130 @@
+package main
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"net/http"
+	"strings"
+	"time"
+
+	"github.com/mudler/LocalAI/pkg/httpclient"
+)
+
+// ErrNoSHA256 marks a GGUF the HuggingFace API describes without an
+// lfs.sha256. Emitting an entry without a hash would ship an unverifiable
+// download, and guessing one from another field is how a Xet hash ends up
+// masquerading as a content hash, so this is fatal rather than skippable.
+var ErrNoSHA256 = errors.New("gguf file has no lfs.sha256")
+
+// GGUFFile is one .gguf sibling of a HuggingFace repo.
+type GGUFFile struct {
+	Name   string
+	Size   int64
+	SHA256 string
+}
+
+type apiSibling struct {
+	RFilename string `json:"rfilename"`
+	Size      int64  `json:"size"`
+	LFS       *struct {
+		SHA256 string `json:"sha256"`
+	} `json:"lfs"`
+}
+
+type apiModel struct {
+	Siblings []apiSibling `json:"siblings"`
+}
+
+// ParseRepoFiles returns every .gguf sibling described by a models API body.
+func ParseRepoFiles(body []byte) ([]GGUFFile, error) {
+	var m apiModel
+	if err := json.Unmarshal(body, &m); err != nil {
+		return nil, fmt.Errorf("decoding model response: %w", err)
+	}
+
+	var out []GGUFFile
+	for _, s := range m.Siblings {
+		if !strings.HasSuffix(s.RFilename, ".gguf") {
+			continue
+		}
+		if s.LFS == nil || s.LFS.SHA256 == "" {
+			return nil, fmt.Errorf("%s: %w", s.RFilename, ErrNoSHA256)
+		}
+		out = append(out, GGUFFile{Name: s.RFilename, Size: s.Size, SHA256: s.LFS.SHA256})
+	}
+	return out, nil
+}
+
+// FetchOptionalRepoFiles asks the models API for a repo the caller can do
+// without, and reports separately whether the repo was merely unreadable.
+//
+// HuggingFace answers 401 Unauthorized, not 404, for a repo that does not exist
+// when the request carries no credentials. Without a token there is therefore no
+// way to tell "this repo was never published" from "this repo is private", so an
+// optional probe has to treat 401 and 403 exactly like 404: whatever the reason,
+// there is nothing here for us to read, so there is no counterpart.
+//
+// The second return value exists because that collapse is lossy in one
+// direction: 401/403 can also mean a real, gated repo whose quants we would
+// genuinely want. The caller reports those repos so a silently dropped
+// counterpart is visible to a human rather than invisible.
+func FetchOptionalRepoFiles(client *http.Client, repo string) ([]GGUFFile, bool, error) {
+	files, status, err := fetchRepoFiles(client, repo)
+	if err != nil && (status == http.StatusUnauthorized || status == http.StatusForbidden) {
+		return nil, true, nil
+	}
+	return files, false, err
+}
+
+// FetchRepoFiles asks the models API for one repo. A 404 yields (nil, nil) so
+// that probing for an optional counterpart repo is not an error. Every other
+// non-200, 401 and 403 included, is an error: for a repo the run REQUIRES there
+// is no benign reading of "we cannot see it".
+func FetchRepoFiles(client *http.Client, repo string) ([]GGUFFile, error) {
+	files, _, err := fetchRepoFiles(client, repo)
+	return files, err
+}
+
+// fetchRepoFiles does the request and returns the HTTP status alongside the
+// result, so the optional and required callers can apply different policies to
+// the same response without duplicating the request.
+func fetchRepoFiles(client *http.Client, repo string) ([]GGUFFile, int, error) {
+	url := fmt.Sprintf("https://huggingface.co/api/models/%s?blobs=true", repo)
+	req, err := http.NewRequest(http.MethodGet, url, nil)
+	if err != nil {
+		return nil, 0, err
+	}
+	req.Header.Set("User-Agent", "localai-apexentries/1.0")
+
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, 0, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode == http.StatusNotFound {
+		return nil, resp.StatusCode, nil
+	}
+	if resp.StatusCode != http.StatusOK {
+		return nil, resp.StatusCode, fmt.Errorf("%s: unexpected status %d", repo, resp.StatusCode)
+	}
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return nil, resp.StatusCode, err
+	}
+	files, err := ParseRepoFiles(body)
+	return files, resp.StatusCode, err
+}
+
+// newHTTPClient builds the client used against the HuggingFace API. It goes
+// through pkg/httpclient rather than a bare &http.Client{} because the std
+// client follows redirects and forwards custom credential headers to the
+// redirect target on a cross-host hop (GHSA-3mj3-57v2-4636). This caller sends
+// only a User-Agent today, but it talks to an external API that could start
+// redirecting, and an HF_TOKEN header here later would then leak.
+func newHTTPClient() *http.Client {
+	return httpclient.NewWithTimeout(60 * time.Second)
+}

.github/ci/apexentries/hf_test.go

@@ -0,0 +1,142 @@
+package main
+
+import (
+	"bytes"
+	"io"
+	"net/http"
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestApexEntries(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "apexentries")
+}
+
+// stubTransport answers every request with one canned status and body, so the
+// status handling of the fetchers can be exercised without reaching the real
+// HuggingFace API.
+type stubTransport struct {
+	status int
+	body   string
+}
+
+func (t stubTransport) RoundTrip(req *http.Request) (*http.Response, error) {
+	return &http.Response{
+		StatusCode: t.status,
+		Body:       io.NopCloser(bytes.NewBufferString(t.body)),
+		Header:     make(http.Header),
+		Request:    req,
+	}, nil
+}
+
+func stubClient(status int, body string) *http.Client {
+	return &http.Client{Transport: stubTransport{status: status, body: body}}
+}
+
+const oneGGUFBody = `{"siblings":[{"rfilename":"Model-APEX-I-Quality.gguf","size":10,"lfs":{"sha256":"aa","size":10}}]}`
+
+var _ = Describe("FetchOptionalRepoFiles", func() {
+	// HuggingFace answers 401 rather than 404 for a repo that does not exist
+	// when the client carries no credentials, so an optional probe cannot tell
+	// "absent" from "unauthorized" and must treat both as "no counterpart".
+	It("treats a 401 as an absent repo and flags it as unavailable", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusUnauthorized, ""), "unsloth/Nope-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+		Expect(unavailable).To(BeTrue())
+	})
+
+	It("treats a 403 as an absent repo and flags it as unavailable", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusForbidden, ""), "unsloth/Gated-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+		Expect(unavailable).To(BeTrue())
+	})
+
+	// A clean 404 is an unambiguous absence, so it must NOT be reported as
+	// unavailable: the whole point of the flag is to separate the ambiguous
+	// case a human may need to look at from the settled one.
+	It("treats a 404 as an absent repo without flagging it as unavailable", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusNotFound, ""), "unsloth/Nope-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+		Expect(unavailable).To(BeFalse())
+	})
+
+	It("parses a 200 body as usual", func() {
+		files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusOK, oneGGUFBody), "unsloth/Real-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(unavailable).To(BeFalse())
+		Expect(files).To(HaveLen(1))
+		Expect(files[0].Name).To(Equal("Model-APEX-I-Quality.gguf"))
+		Expect(files[0].SHA256).To(Equal("aa"))
+	})
+
+	// Tolerating 401/403 must not widen into tolerating everything: a 500 is a
+	// broken API, not evidence about whether the repo exists.
+	It("still errors on a 500", func() {
+		_, _, err := FetchOptionalRepoFiles(stubClient(http.StatusInternalServerError, ""), "unsloth/Real-GGUF")
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unexpected status 500"))
+	})
+})
+
+var _ = Describe("FetchRepoFiles", func() {
+	// The APEX repo itself is not optional. A 401 there means the repo the run
+	// was asked to publish cannot be read, which is a real failure and must not
+	// be quietly downgraded to "no files".
+	It("errors on a 401 for a required repo", func() {
+		_, err := FetchRepoFiles(stubClient(http.StatusUnauthorized, ""), "mudler/Model-APEX-GGUF")
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unexpected status 401"))
+	})
+
+	It("errors on a 403 for a required repo", func() {
+		_, err := FetchRepoFiles(stubClient(http.StatusForbidden, ""), "mudler/Model-APEX-GGUF")
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unexpected status 403"))
+	})
+
+	It("still treats a 404 as an absent repo", func() {
+		files, err := FetchRepoFiles(stubClient(http.StatusNotFound, ""), "mudler/Model-APEX-GGUF")
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(BeEmpty())
+	})
+})
+
+var _ = Describe("ParseRepoFiles", func() {
+	It("returns gguf siblings with their lfs sha256", func() {
+		body := []byte(`{"siblings":[
+			{"rfilename":"Model-APEX-I-Quality.gguf","size":10,"lfs":{"sha256":"aa","size":10}},
+			{"rfilename":"README.md"},
+			{"rfilename":"mmproj.gguf","size":5,"lfs":{"sha256":"bb","size":5}}
+		]}`)
+
+		files, err := ParseRepoFiles(body)
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(files).To(HaveLen(2))
+		Expect(files[0].Name).To(Equal("Model-APEX-I-Quality.gguf"))
+		Expect(files[0].SHA256).To(Equal("aa"))
+		Expect(files[1].Name).To(Equal("mmproj.gguf"))
+	})
+
+	It("reports a gguf that carries no lfs sha256", func() {
+		body := []byte(`{"siblings":[{"rfilename":"mmproj.gguf","size":5}]}`)
+
+		_, err := ParseRepoFiles(body)
+
+		Expect(err).To(MatchError(ErrNoSHA256))
+	})
+})

.github/ci/apexentries/hub.go

@@ -0,0 +1,143 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+// IndexText is the gallery index seen as text: the entries it declares plus the
+// exact lines each one occupies, which is what splicing a variants block into an
+// entry the gallery already ships requires.
+//
+// It is a second, narrower read of the same file LoadExisting parses. The two
+// answer different questions: LoadExisting answers "do these weights already
+// exist anywhere", this one answers "where in the file does this entry live".
+type IndexText struct {
+	Lines   []string
+	Entries []*indexEntry
+
+	byName map[string]*indexEntry
+}
+
+// indexEntry is one entry of the index: its name, the variants it already
+// declares, and its coordinates in the file.
+type indexEntry struct {
+	Name     string       `yaml:"name"`
+	Variants []VariantRef `yaml:"variants"`
+
+	Pos galleryedit.Entry `yaml:"-"`
+}
+
+// LoadIndexText reads the gallery index for editing.
+func LoadIndexText(path string) (*IndexText, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return nil, err
+	}
+	return ParseIndexText(string(raw))
+}
+
+// ParseIndexText pairs the decoded entries with the top level list items the
+// text actually contains.
+//
+// If the two views disagree on how many entries there are then every line number
+// a splice would compute is suspect, and the failure mode is writing a variants
+// block into the wrong model. The parse refuses instead.
+func ParseIndexText(text string) (*IndexText, error) {
+	var entries []*indexEntry
+	if err := yaml.Unmarshal([]byte(text), &entries); err != nil {
+		return nil, fmt.Errorf("decoding gallery index: %w", err)
+	}
+
+	lines, starts := galleryedit.Scan(text)
+	if len(starts) != len(entries) {
+		return nil, fmt.Errorf("gallery index has %d decoded entries but %d top level list items; refusing to edit by line number",
+			len(entries), len(starts))
+	}
+
+	ix := &IndexText{Lines: lines, Entries: entries, byName: map[string]*indexEntry{}}
+	for i, e := range entries {
+		if e == nil {
+			return nil, fmt.Errorf("gallery index list item %d is empty; refusing to edit by line number", i)
+		}
+		end := len(lines)
+		if i+1 < len(starts) {
+			end = starts[i+1]
+		}
+		e.Pos = galleryedit.Entry{Name: e.Name, StartLine: starts[i], EndLine: end}
+
+		// First occurrence wins, matching the gallery's own resolution.
+		key := strings.ToLower(e.Name)
+		if _, seen := ix.byName[key]; !seen {
+			ix.byName[key] = e
+		}
+	}
+	return ix, nil
+}
+
+// Find looks an entry up by name, case insensitively.
+func (ix *IndexText) Find(name string) *indexEntry {
+	return ix.byName[strings.ToLower(name)]
+}
+
+// ResolveHub returns the gallery name of a family's hub and whether the gallery
+// already ships an entry under it.
+//
+// The hub is the BASE model entry, never a generated *-apex parent. Somebody
+// looking for qwen3.6-35b-a3b has to find every build of those weights under
+// that one name: the APEX imatrix rungs, the unsloth quant rungs and any
+// speculative build. A separate qwen3.6-35b-a3b-apex hub competing with the base
+// entry would split the family in two and leave whichever half the user did not
+// search for invisible.
+//
+// Both candidates are tried for the same reason CounterpartCandidates tries
+// both. The repo name and the published file stem disagree for several of these
+// repos, and either one may be what the base entry was named after.
+func ResolveHub(ix *IndexText, repoBase, stem string) (name string, exists bool) {
+	candidates := CounterpartCandidates(repoBase, stem)
+	for _, c := range candidates {
+		if n := slug(c); ix.Find(n) != nil {
+			return n, true
+		}
+	}
+	// Nothing matched, so the family needs a hub of its own under the repo
+	// derived name, which is the more reliable of the two.
+	return slug(candidates[0]), false
+}
+
+// HubLabel is the human-cased base model name, for prose rather than lookup.
+func HubLabel(repoBase, stem string) string {
+	return CounterpartCandidates(repoBase, stem)[0]
+}
+
+// filterVariants drops the references a hub must not carry: itself, and anything
+// it already lists.
+//
+// The self reference is not merely redundant. A hub that names itself makes the
+// verifier resolve the reference back to the hub, see that the hub declares
+// variants, and report a variant that declares variants of its own. It arises
+// for real rather than in theory: an unsloth rung whose weights the gallery
+// already ships under the base model name resolves, through Merge, straight back
+// to the hub that is about to reference it.
+func filterVariants(hub string, already []VariantRef, want []string) []string {
+	seen := map[string]bool{strings.ToLower(hub): true}
+	for _, v := range already {
+		seen[strings.ToLower(v.Model)] = true
+	}
+
+	var out []string
+	for _, w := range want {
+		key := strings.ToLower(w)
+		if seen[key] {
+			continue
+		}
+		seen[key] = true
+		out = append(out, w)
+	}
+	return out
+}

.github/ci/apexentries/main.go

@@ -0,0 +1,795 @@
+// Command apexentries generates gallery entries for the mudler APEX GGUF
+// repositories: one entry per imatrix tier and per unsloth quant rung, all
+// gathered under the BASE model's entry. Builds off a *-APEX-MTP-GGUF repo turn
+// speculative decoding on, because those weights retain the model's MTP heads
+// and are only worth their extra size with the heads in use.
+//
+// The base model entry is the hub. Somebody looking for qwen3.6-35b-a3b must
+// find every build of those weights under that one name, so when the gallery
+// already ships the base entry this command splices a variants block into it
+// rather than emitting a competing *-apex parent beside it. Only a family whose
+// base model the gallery does not ship at all gets a new hub entry, and that one
+// is still named for the base model.
+//
+// Builds are discovered by inspecting the filenames a repo actually publishes.
+// Repo names do not reliably predict them: mudler/gemma-4-26B-A4B-it-APEX-GGUF
+// ships gemma-4-26B-A4B-APEX-*.gguf, and six of the 45 repos drop a suffix or a
+// vendor prefix in the same way.
+package main
+
+import (
+	"encoding/json"
+	"flag"
+	"fmt"
+	"io"
+	"net/http"
+	"os"
+	"path"
+	"sort"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+const (
+	// entryTemplate carries no backend and no parameters of its own, which is
+	// why RenderChild states everything inline.
+	entryTemplate = "virtual.yaml"
+	unslothOwner  = "unsloth"
+	authorListURL = "https://huggingface.co/api/models?author=mudler&limit=300"
+)
+
+// rungRank orders the quality ladder from best to smallest. The HuggingFace API
+// returns siblings alphabetically and DiscoverAPEXTiers preserves that order, so
+// an unsorted variants list reads I-Balanced, I-Compact, I-Mini, I-Nano,
+// I-Quality. Selection ignores authored order, so this is purely so the file a
+// human reviews scans in a meaningful sequence.
+var rungRank = map[string]int{
+	"I-Quality": 0, "I-Balanced": 1, "I-Compact": 2, "I-Mini": 3, "I-Nano": 4,
+	"Quality": 5, "Balanced": 6, "Compact": 7, "Mini": 8, "Nano": 9,
+}
+
+// baseTags are the tags every generated entry carries. dflash and mtp are never
+// among them: RenderChild adds those if and only if the entry configures the
+// matching spec_type.
+var baseTags = []string{"llm", "gguf", "cpu", "gpu"}
+
+// childBuild pairs a rendered entry with its position on the quality ladder, so
+// the parent's variants list can be sorted without re-parsing entry names.
+type childBuild struct {
+	entry GalleryEntry
+	rank  int
+}
+
+// family is one APEX repo's full generated output.
+type family struct {
+	repo      string
+	repoBase  string
+	stem      string
+	hasMMProj bool
+	children  []childBuild
+
+	// skippedRepos are counterpart candidates HuggingFace would not describe.
+	// Carried on the family rather than printed and forgotten so the run can
+	// summarize them next to everything else a reviewer has to eyeball.
+	skippedRepos []string
+	census       fileCensus
+	unaccounted  int
+}
+
+// fileCensus splits the files discovery emitted nothing for into the ones a
+// reviewer must chase and the ones that are deliberately out of scope.
+//
+// Full-precision sources are the second kind: they are the unquantized weights
+// the ladder is derived FROM, not a rung of it. Folding them into the
+// unclassified total would leave a permanent benign baseline, and a permanent
+// baseline is exactly what hides the one file that ever genuinely matters.
+type fileCensus struct {
+	unclassified  int
+	fullPrecision int
+}
+
+// add accumulates one repo's census into a running total.
+func (c *fileCensus) add(o fileCensus) {
+	c.unclassified += o.unclassified
+	c.fullPrecision += o.fullPrecision
+}
+
+// sortedChildren returns the family's builds in ladder order, best first.
+func (f *family) sortedChildren() []childBuild {
+	sorted := append([]childBuild{}, f.children...)
+	sort.SliceStable(sorted, func(i, j int) bool { return sorted[i].rank < sorted[j].rank })
+	return sorted
+}
+
+func main() {
+	verify := flag.String("verify", "", "verify a gallery index and exit")
+	index := flag.String("index", "gallery/index.yaml", "gallery index to dedup against")
+	only := flag.String("only", "", "comma-separated repo names to restrict generation to")
+	out := flag.String("out", "", "write the entries to add to this file")
+	apply := flag.Bool("apply", false, "append the entries to add to -index")
+	flag.Parse()
+
+	if *verify != "" {
+		problems := Verify(*verify)
+		for _, p := range problems {
+			fmt.Fprintln(os.Stderr, p)
+		}
+		if len(problems) > 0 {
+			fmt.Fprintf(os.Stderr, "%d problem(s)\n", len(problems))
+			os.Exit(1)
+		}
+		fmt.Println("index is sound")
+		return
+	}
+
+	if err := generate(*index, *only, *out, *apply); err != nil {
+		fmt.Fprintln(os.Stderr, "error:", err)
+		os.Exit(1)
+	}
+}
+
+func generate(indexPath, only, outPath string, apply bool) error {
+	if outPath == "" && !apply {
+		return fmt.Errorf("nothing to do: pass -out <file> or -apply")
+	}
+
+	client := newHTTPClient()
+
+	repos, err := listAPEXRepos(client)
+	if err != nil {
+		return err
+	}
+	if only != "" {
+		repos = restrict(repos, only)
+	}
+	if len(repos) == 0 {
+		return fmt.Errorf("no APEX repos selected")
+	}
+	fmt.Printf("repos selected: %d\n", len(repos))
+
+	var families []family
+	var failed []string
+
+	for _, repo := range repos {
+		f, err := buildFamily(client, repo)
+		if err != nil {
+			// A missing sha256 is fatal for the family rather than skippable: an
+			// entry without one ships an unverifiable download. Report which repo
+			// and keep going, so one bad repo does not hide the state of the rest.
+			fmt.Fprintf(os.Stderr, "FAILED %s: %v\n", repo, err)
+			failed = append(failed, repo)
+			continue
+		}
+		families = append(families, *f)
+	}
+
+	existing, err := LoadExisting(indexPath)
+	if err != nil {
+		return err
+	}
+	ixText, err := LoadIndexText(indexPath)
+	if err != nil {
+		return err
+	}
+	fmt.Printf("existing index: %d names, %d weight URIs, %d lines\n",
+		len(existing.ByName), len(existing.ByURI), len(ixText.Lines))
+
+	// Only the builds go through Merge. A hub is deliberately kept out of it: a
+	// new hub carries the family's top rung as its own payload, so Merge's URI
+	// dedup would fold the hub into that rung and the family would lose the very
+	// entry point this command exists to create. Hub names are checked against
+	// the index directly, by ResolveHub.
+	var generated []GalleryEntry
+	for _, f := range families {
+		for _, c := range f.children {
+			generated = append(generated, c.entry)
+		}
+	}
+
+	add, reused := Merge(existing, generated)
+	reportReuse(existing, generated, reused)
+
+	// Variant references are resolved from `reused`, never used to decide what to
+	// emit: on a within-batch name collision Merge records reused[name] = name
+	// while the first entry of that name is still in `add`, so treating presence
+	// in `reused` as "dropped" would silently emit nothing for it.
+	added := map[string]bool{}
+	for _, e := range add {
+		added[e.Name] = true
+	}
+
+	inserts, newHubs, err := planHubs(families, ixText, reused, added)
+	if err != nil {
+		return err
+	}
+	reportHubs(ixText, inserts, newHubs)
+
+	skipped, census, fullPrecisionRepos, unaccounted := reportSkipped(families)
+
+	add = append(add, newHubs...)
+
+	fmt.Printf("\nentries generated: %d\nentries to add:    %d\nentries reused:    %d\nhubs spliced:      %d\nhubs created:      %d\nrepos skipped:     %d\nexcluded (full precision): %d files across %d repos\nunclassified:      %d\nunaccounted:       %d\n",
+		len(generated), len(add), len(reused), len(inserts), len(newHubs), len(skipped),
+		census.fullPrecision, fullPrecisionRepos, census.unclassified, unaccounted)
+
+	lines, err := galleryedit.Apply(ixText.Lines, inserts)
+	if err != nil {
+		return err
+	}
+
+	if err := writeEntries(add, lines, outPath, apply, indexPath); err != nil {
+		return err
+	}
+
+	if len(failed) > 0 {
+		return fmt.Errorf("%d repo(s) failed: %s", len(failed), strings.Join(failed, ", "))
+	}
+	return nil
+}
+
+// resolveVariant maps a generated child name onto whatever entry actually stands
+// for it after the merge. `added` is consulted first because a within-batch name
+// collision puts a name in BOTH add and reused, and the entry that was emitted
+// is the one the parent must reference.
+func resolveVariant(name string, reused map[string]string, added map[string]bool) string {
+	if added[name] {
+		return name
+	}
+	if target, ok := reused[name]; ok {
+		return target
+	}
+	return name
+}
+
+// SpecTypeForRepo reports the speculative decoding mechanism a repo's builds can
+// turn on with no extra download.
+//
+// The *-APEX-MTP-GGUF repos republish the base weights with the model's own MTP
+// heads retained, so those builds are only worth their extra size if the heads
+// are actually used. Every other APEX repo drops them, and switching MTP on
+// there would name a mechanism the weights cannot serve.
+//
+// The suffix is read off the repo the FILES come from, so nothing downstream has
+// to infer a capability from an entry name.
+func SpecTypeForRepo(repo string) string {
+	if strings.HasSuffix(path.Base(repo), "-APEX-MTP-GGUF") {
+		return "draft-mtp"
+	}
+	return ""
+}
+
+// buildFamily discovers everything one APEX repo and its unsloth counterpart
+// publish, and renders it.
+func buildFamily(client *http.Client, repo string) (*family, error) {
+	files, err := FetchRepoFiles(client, repo)
+	if err != nil {
+		return nil, err
+	}
+	if len(files) == 0 {
+		return nil, fmt.Errorf("no gguf files")
+	}
+
+	imatrix, plain := DiscoverAPEXTiers(files)
+	mmproj, hasMMProj := DiscoverMMProj(files)
+
+	census := reportUnclassified(repo, files, imatrix, plain)
+
+	// The imatrix ladder is preferred, but two of the 45 repos publish no
+	// imatrix tiers at all and must still contribute their plain ladder.
+	ladder := imatrix
+	ladderKind := "imatrix"
+	if len(ladder) == 0 {
+		ladder = plain
+		ladderKind = "plain"
+	}
+	if len(ladder) == 0 {
+		return nil, fmt.Errorf("no tiers discovered")
+	}
+
+	sortTiers(ladder)
+
+	var mm *GGUFFile
+	if hasMMProj {
+		mm = &mmproj
+	}
+
+	repoBase := strings.TrimSuffix(path.Base(repo), "-GGUF")
+	f := &family{repo: repo, repoBase: repoBase, hasMMProj: hasMMProj, census: census}
+
+	// Only the APEX ladder can carry MTP heads; the unsloth counterpart quantizes
+	// the plain weights and gets nothing from this.
+	specType := SpecTypeForRepo(repo)
+
+	for _, t := range ladder {
+		f.children = append(f.children, childBuild{
+			rank: rungRank[t.Label],
+			entry: RenderChild(ChildInput{
+				Name:     slug(repoBase) + "-" + slug(t.Label),
+				Repo:     repo,
+				Template: entryTemplate,
+				SpecType: specType,
+				Weights:  []GGUFFile{t.File},
+				MMProj:   mm,
+				BaseTags: baseTags,
+			}),
+		})
+	}
+
+	stem := FileStem(ladder[0])
+	f.stem = stem
+	fmt.Printf("%s: %d %s tier(s) [%s], stem %s, mmproj %v\n",
+		repo, len(ladder), ladderKind, tierLabels(ladder), stem, hasMMProj)
+
+	counterpart, cpFiles, skipped, err := resolveCounterpart(client, repoBase, stem)
+	f.skippedRepos = skipped
+	if err != nil {
+		return nil, err
+	}
+	if counterpart != "" {
+		builds := DiscoverUnslothQuants(cpFiles)
+
+		// Called here rather than inside Verify: a quant dropped at discovery
+		// leaves no trace at all in the finished gallery file, so the only place
+		// the shortfall is still visible is the moment of discovery.
+		unaccounted := UnaccountedQuants(cpFiles, builds)
+		f.unaccounted = len(unaccounted)
+		for _, p := range unaccounted {
+			fmt.Fprintf(os.Stderr, "UNACCOUNTED QUANT %s: %s\n", counterpart, p)
+		}
+
+		cpMMProj, hasCPMMProj := DiscoverMMProj(cpFiles)
+		var cpMM *GGUFFile
+		if hasCPMMProj {
+			cpMM = &cpMMProj
+		}
+		cpBase := strings.TrimSuffix(path.Base(counterpart), "-GGUF")
+
+		for i, b := range builds {
+			f.children = append(f.children, childBuild{
+				rank: 100 + i,
+				entry: RenderChild(ChildInput{
+					Name:     slug(cpBase) + "-" + slug(b.Quant),
+					Repo:     counterpart,
+					Template: entryTemplate,
+					Weights:  b.Files,
+					MMProj:   cpMM,
+					BaseTags: baseTags,
+				}),
+			})
+		}
+		fmt.Printf("%s: counterpart %s, %d quant build(s) %s\n", repo, counterpart, len(builds), quantLabels(builds))
+	} else {
+		fmt.Printf("%s: no unsloth counterpart\n", repo)
+	}
+
+	return f, nil
+}
+
+// planHubs decides, per family, whether the family's builds are spliced into a
+// base model entry the gallery already ships or gathered under a new hub.
+//
+// Splicing is strongly preferred and is the measured majority-adjacent case. The
+// existing entry keeps its description, icon, tags, overrides and files
+// untouched; only variant lines are added to it.
+func planHubs(families []family, ix *IndexText, reused map[string]string, added map[string]bool) ([]galleryedit.Insert, []GalleryEntry, error) {
+	// Several APEX repos can resolve to one base model, so both paths accumulate
+	// by hub name rather than assuming one family per hub.
+	wantByHub := map[string][]string{}
+	var spliceOrder []string
+
+	var newHubs []GalleryEntry
+	hubAt := map[string]int{}
+
+	for i := range families {
+		f := &families[i]
+
+		hubName, exists := ResolveHub(ix, f.repoBase, f.stem)
+		want := hubVariants(f, ix, reused, added)
+
+		if exists {
+			if _, seen := wantByHub[hubName]; !seen {
+				spliceOrder = append(spliceOrder, hubName)
+			}
+			wantByHub[hubName] = append(wantByHub[hubName], want...)
+			continue
+		}
+
+		if at, dup := hubAt[hubName]; dup {
+			for _, v := range filterVariants(hubName, newHubs[at].Variants, want) {
+				newHubs[at].Variants = append(newHubs[at].Variants, VariantRef{Model: v})
+			}
+			continue
+		}
+
+		builds := f.sortedChildren()
+		if len(builds) == 0 {
+			return nil, nil, fmt.Errorf("%s: no builds to hang a hub on", f.repo)
+		}
+		hubAt[hubName] = len(newHubs)
+		newHubs = append(newHubs, renderHub(hubName, f, builds[0], filterVariants(hubName, nil, want)))
+	}
+
+	var inserts []galleryedit.Insert
+	for _, name := range spliceOrder {
+		e := ix.Find(name)
+		items := filterVariants(name, e.Variants, wantByHub[name])
+		if len(items) == 0 {
+			continue
+		}
+		inserts = append(inserts, galleryedit.Insert{Entry: e.Pos, Variants: items})
+	}
+	return inserts, newHubs, nil
+}
+
+// hubVariants is a family's full build list, in ladder order, named as the hub
+// must reference them after the merge.
+func hubVariants(f *family, ix *IndexText, reused map[string]string, added map[string]bool) []string {
+	var out []string
+
+	// A hand-written *-apex entry is an ordinary build of these weights. It is
+	// never deleted, never renamed and never treated as a hub; it is simply
+	// referenced like any other rung.
+	if apex := slug(f.repoBase); ix.Find(apex) != nil {
+		out = append(out, apex)
+	}
+	for _, c := range f.sortedChildren() {
+		out = append(out, resolveVariant(c.entry.Name, reused, added))
+	}
+	return out
+}
+
+// renderHub builds the hub for a family whose base model the gallery does not
+// ship at all. It is named for the BASE model, never for the APEX repo.
+//
+// It carries one of the discovered builds as its own payload so it is a complete
+// installable entry rather than a bare index pointing at other entries. That
+// payload is what supplies overrides.backend, which matters beyond installation:
+// the verifier can only judge the tagging rule for a backend it can read, so a
+// hub carrying feature tags and no backend would escape the check in silence.
+//
+// The payload's own tags are kept rather than rebuilt from baseTags, so a hub
+// whose payload configures a spec_type stays tagged for it and consistent with
+// the overrides copied alongside.
+func renderHub(name string, f *family, payload childBuild, variants []string) GalleryEntry {
+	e := payload.entry
+	e.Name = name
+	e.Description = fmt.Sprintf(
+		"%s. Quality ladder and quantization rungs published by %s and its unsloth counterpart; LocalAI picks the build that fits the hardware.",
+		HubLabel(f.repoBase, f.stem), f.repo)
+
+	e.Tags = append([]string{}, payload.entry.Tags...)
+	if f.hasMMProj && !hasTag(e.Tags, "vision") {
+		e.Tags = append(e.Tags, "vision")
+	}
+
+	e.Variants = nil
+	for _, v := range variants {
+		e.Variants = append(e.Variants, VariantRef{Model: v})
+	}
+	return e
+}
+
+func hasTag(tags []string, want string) bool {
+	for _, t := range tags {
+		if t == want {
+			return true
+		}
+	}
+	return false
+}
+
+// resolveCounterpart probes the unsloth candidates in order and returns the
+// first that publishes files.
+//
+// CounterpartCandidates is handed a BARE repo name: its cleaner does not strip
+// an owner prefix, so passing "mudler/Foo-APEX-GGUF" would yield "mudler/Foo"
+// and compose into the nonsense probe "unsloth/mudler/Foo".
+//
+// It also returns the candidates HuggingFace refused to describe. Those are
+// indistinguishable from absent without credentials, so they are skipped, but
+// they are named rather than dropped: one of them could be a real gated repo
+// whose quants belong in the gallery.
+func resolveCounterpart(client *http.Client, repoBase, stem string) (string, []GGUFFile, []string, error) {
+	var unavailable []string
+	for _, cand := range CounterpartCandidates(repoBase, stem) {
+		repo := unslothOwner + "/" + cand + "-GGUF"
+		files, unreadable, err := FetchOptionalRepoFiles(client, repo)
+		if err != nil {
+			return "", nil, unavailable, fmt.Errorf("probing %s: %w", repo, err)
+		}
+		if unreadable {
+			unavailable = append(unavailable, repo)
+			continue
+		}
+		if len(files) > 0 {
+			return repo, files, unavailable, nil
+		}
+	}
+	return "", nil, unavailable, nil
+}
+
+// reportUnclassified prints the files discovery turned into nothing.
+//
+// It is a set difference on COUNTS, not a re-match of filenames: re-matching
+// would duplicate the tier regex from discover.go and the two copies would
+// drift. The likeliest trigger is a typo or case change from a publishing script
+// rather than a genuine sixth tier, and because generation falls back to the
+// plain ladder when the imatrix one is empty, a repo whose imatrix files all
+// fail to match silently downgrades the whole family instead of erroring. The
+// downstream HTTP check cannot catch that: it validates URLs that were emitted,
+// and an undiscovered tier emits none.
+// It returns the census so the run can total it.
+func reportUnclassified(repo string, files []GGUFFile, imatrix, plain []Tier) fileCensus {
+	mmprojCount, fullPrecision := 0, 0
+	for _, f := range files {
+		// The mmproj test comes first because projectors are themselves often
+		// published at f16 (mmproj-F16.gguf), and counting such a file in both
+		// buckets would understate the unclassified remainder.
+		if strings.HasPrefix(f.Name, "mmproj") {
+			mmprojCount++
+			continue
+		}
+		if IsFullPrecision(f.Name) {
+			fullPrecision++
+		}
+	}
+
+	classified := len(imatrix) + len(plain) + mmprojCount + fullPrecision
+	if classified >= len(files) {
+		return fileCensus{fullPrecision: fullPrecision}
+	}
+	fmt.Fprintf(os.Stderr, "UNCLASSIFIED %s: %d of %d .gguf files classified, %d unaccounted for\n",
+		repo, classified, len(files), len(files)-classified)
+	return fileCensus{unclassified: len(files) - classified, fullPrecision: fullPrecision}
+}
+
+// reportReuse splits Merge's single reused map into the two cases it conflates.
+//
+// A URI match means the gallery already ships exactly these weights, and
+// pointing the parent at the existing entry is correct. A NAME match with a
+// different URI means an unrelated entry happens to own the name, and
+// referencing it would point the parent at different weights than were
+// generated, substituting a build without saying so. Only the first is safe to
+// wave through.
+func reportReuse(existing *ExistingIndex, generated []GalleryEntry, reused map[string]string) {
+	byName := map[string]GalleryEntry{}
+	for _, e := range generated {
+		if _, seen := byName[e.Name]; !seen {
+			byName[e.Name] = e
+		}
+	}
+
+	var nameCollisions, uriMatches []string
+	for name, target := range reused {
+		gen := byName[name]
+		uri := ""
+		if len(gen.Files) > 0 {
+			uri = gen.Files[0].URI
+		}
+
+		switch {
+		case hasName(existing, name):
+			nameCollisions = append(nameCollisions,
+				fmt.Sprintf("  %s -> gallery entry of the same name (generated uri: %s)", name, orNone(uri)))
+		case target == name:
+			nameCollisions = append(nameCollisions,
+				fmt.Sprintf("  %s -> earlier entry of the same name in this batch (generated uri: %s)", name, orNone(uri)))
+		default:
+			uriMatches = append(uriMatches, fmt.Sprintf("  %s -> %s (same weights: %s)", name, target, orNone(uri)))
+		}
+	}
+	sort.Strings(nameCollisions)
+	sort.Strings(uriMatches)
+
+	fmt.Printf("\nNAME COLLISIONS (%d) - inspect each by hand, the target may hold different weights\n", len(nameCollisions))
+	for _, l := range nameCollisions {
+		fmt.Println(l)
+	}
+	fmt.Printf("\nURI MATCHES (%d) - the gallery or this batch already ships these exact weights\n", len(uriMatches))
+	for _, l := range uriMatches {
+		fmt.Println(l)
+	}
+}
+
+// reportHubs prints exactly what will be written where. The splices are the part
+// a human has to read: they modify entries the gallery already ships, so the
+// review needs the target, the line, and every added reference spelled out.
+func reportHubs(ix *IndexText, inserts []galleryedit.Insert, newHubs []GalleryEntry) {
+	fmt.Printf("\nHUBS SPLICED (%d) - variants added to the EXISTING base model entry, nothing else touched\n", len(inserts))
+	for _, in := range inserts {
+		e := ix.Find(in.Entry.Name)
+		fmt.Printf("  %s (line %d, %d variant(s) already declared):\n", in.Entry.Name, in.Entry.StartLine+1, len(e.Variants))
+		for _, v := range in.Variants {
+			fmt.Printf("    + - model: %s\n", galleryedit.QuoteName(v))
+		}
+	}
+
+	fmt.Printf("\nHUBS CREATED (%d) - the gallery ships no base model entry, so one is emitted for it\n", len(newHubs))
+	for _, h := range newHubs {
+		fmt.Printf("  %s:\n", h.Name)
+		for _, v := range h.Variants {
+			fmt.Printf("    - model: %s\n", v.Model)
+		}
+	}
+}
+
+// reportSkipped names the counterpart repos HuggingFace would not describe, and
+// totals the other two silent-shortfall counters alongside them.
+//
+// A skipped repo is not the same as a clean 404. HuggingFace answers 401 for a
+// nonexistent repo to an unauthenticated client, so the overwhelmingly likely
+// reading is "there is no such counterpart", which is the normal case for the
+// community merges. But a private or gated repo answers 401 too, and that one
+// WOULD have quants worth shipping. Printing the list is what keeps that
+// possibility auditable instead of silently discarded.
+func reportSkipped(families []family) ([]string, fileCensus, int, int) {
+	var skipped []string
+	var census fileCensus
+	fullPrecisionRepos, unaccounted := 0, 0
+	for _, f := range families {
+		skipped = append(skipped, f.skippedRepos...)
+		census.add(f.census)
+		if f.census.fullPrecision > 0 {
+			fullPrecisionRepos++
+		}
+		unaccounted += f.unaccounted
+	}
+	sort.Strings(skipped)
+
+	fmt.Printf("\nREPOS SKIPPED AS UNAVAILABLE (%d) - HuggingFace answered 401/403, which is indistinguishable from absent without a token; check none of these is a real gated repo\n", len(skipped))
+	for _, r := range skipped {
+		fmt.Printf("  %s\n", r)
+	}
+	return skipped, census, fullPrecisionRepos, unaccounted
+}
+
+func hasName(ix *ExistingIndex, name string) bool {
+	_, ok := ix.ByName[name]
+	return ok
+}
+
+func orNone(s string) string {
+	if s == "" {
+		return "(no files)"
+	}
+	return s
+}
+
+// writeEntries emits the additions.
+//
+// -apply does two things in one pass: it writes back the spliced lines, which
+// differ from the original only by the variant lines galleryedit inserted, and
+// then appends the new entries. New entries are APPENDED rather than merged into
+// the structure, for the same reason the splice is textual: a YAML round trip
+// over 40,000 lines would reflow the whole file into an unreviewable diff.
+func writeEntries(add []GalleryEntry, lines []string, outPath string, apply bool, indexPath string) error {
+	if apply {
+		if err := os.WriteFile(indexPath, []byte(strings.Join(lines, "\n")), 0o644); err != nil {
+			return err
+		}
+		fmt.Printf("spliced %s\n", indexPath)
+	}
+
+	if len(add) == 0 {
+		fmt.Println("nothing to append")
+		return nil
+	}
+
+	blob, err := yaml.Marshal(add)
+	if err != nil {
+		return err
+	}
+
+	if outPath != "" {
+		if err := os.WriteFile(outPath, blob, 0o644); err != nil {
+			return err
+		}
+		fmt.Printf("wrote %d entries to %s\n", len(add), outPath)
+	}
+
+	if apply {
+		f, err := os.OpenFile(indexPath, os.O_APPEND|os.O_WRONLY, 0o644)
+		if err != nil {
+			return err
+		}
+		defer f.Close()
+		if _, err := f.Write(blob); err != nil {
+			return err
+		}
+		fmt.Printf("appended %d entries to %s\n", len(add), indexPath)
+	}
+	return nil
+}
+
+// listAPEXRepos returns the mudler repos whose name marks them as APEX builds.
+func listAPEXRepos(client *http.Client) ([]string, error) {
+	req, err := http.NewRequest(http.MethodGet, authorListURL, nil)
+	if err != nil {
+		return nil, err
+	}
+	req.Header.Set("User-Agent", "localai-apexentries/1.0")
+
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != http.StatusOK {
+		return nil, fmt.Errorf("listing models: unexpected status %d", resp.StatusCode)
+	}
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return nil, err
+	}
+
+	var models []struct {
+		ID string `json:"id"`
+	}
+	if err := json.Unmarshal(body, &models); err != nil {
+		return nil, fmt.Errorf("decoding model list: %w", err)
+	}
+
+	var out []string
+	for _, m := range models {
+		if strings.Contains(m.ID, "APEX") {
+			out = append(out, m.ID)
+		}
+	}
+	sort.Strings(out)
+	return out, nil
+}
+
+func restrict(repos []string, only string) []string {
+	want := map[string]bool{}
+	for _, r := range strings.Split(only, ",") {
+		if r = strings.TrimSpace(r); r != "" {
+			want[r] = true
+		}
+	}
+
+	var out []string
+	for _, r := range repos {
+		if want[r] {
+			out = append(out, r)
+			delete(want, r)
+		}
+	}
+	// A name in -only that matched nothing is a typo, not an empty result.
+	for r := range want {
+		fmt.Fprintf(os.Stderr, "WARNING: -only names %s, which is not an APEX repo of this author\n", r)
+	}
+	return out
+}
+
+func sortTiers(tiers []Tier) {
+	sort.SliceStable(tiers, func(i, j int) bool { return rungRank[tiers[i].Label] < rungRank[tiers[j].Label] })
+}
+
+func tierLabels(tiers []Tier) string {
+	var out []string
+	for _, t := range tiers {
+		out = append(out, t.Label)
+	}
+	return strings.Join(out, ",")
+}
+
+func quantLabels(builds []QuantBuild) string {
+	var out []string
+	for _, b := range builds {
+		l := b.Quant
+		if b.Sharded {
+			l += fmt.Sprintf("(%d shards)", len(b.Files))
+		}
+		out = append(out, l)
+	}
+	return strings.Join(out, ",")
+}
+
+// slug turns a repo, tier or quant label into a gallery entry name component.
+func slug(s string) string {
+	return strings.ReplaceAll(strings.ToLower(s), "_", "-")
+}

.github/ci/apexentries/main_test.go

@@ -0,0 +1,344 @@
+package main
+
+import (
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
+)
+
+func mustIndex(text string) *IndexText {
+	ix, err := ParseIndexText(text)
+	ExpectWithOffset(1, err).ToNot(HaveOccurred())
+	return ix
+}
+
+// buildOf renders a realistic child so the specs exercise the payload a hub
+// actually inherits rather than a bare name.
+func buildOf(name, repo, file string, rank int) childBuild {
+	return childBuild{
+		rank: rank,
+		entry: RenderChild(ChildInput{
+			Name:     name,
+			Repo:     repo,
+			Template: entryTemplate,
+			Weights:  []GGUFFile{{Name: file, SHA256: "aa"}},
+			BaseTags: baseTags,
+		}),
+	}
+}
+
+var _ = Describe("ResolveHub", func() {
+	It("picks the base model name over the APEX name, even when both are in the gallery", func() {
+		// The hub is the entry a user searches for. If the *-apex entry were
+		// chosen the family would be gathered under a name nobody looks up, and
+		// the base entry would go on advertising only its own build.
+		ix := mustIndex("- name: qwen3.6-35b-a3b\n  url: u\n- name: qwen3.6-35b-a3b-apex\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "Qwen3.6-35B-A3B-APEX", "Qwen3.6-35B-A3B-APEX")
+
+		Expect(name).To(Equal("qwen3.6-35b-a3b"))
+		Expect(exists).To(BeTrue())
+	})
+
+	It("falls back to the stem-derived candidate when the repo-derived one is absent", func() {
+		// gemma's repo says "-it" and its published files do not, so only one of
+		// the two candidates can match whatever the base entry was named after.
+		ix := mustIndex("- name: gemma-4-26b-a4b\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "gemma-4-26B-A4B-it-APEX", "gemma-4-26B-A4B-APEX")
+
+		Expect(name).To(Equal("gemma-4-26b-a4b"))
+		Expect(exists).To(BeTrue())
+	})
+
+	It("reports the base name as absent rather than settling for the APEX entry", func() {
+		ix := mustIndex("- name: qwen3.5-35b-a3b-apex\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "Qwen3.5-35B-A3B-APEX", "Qwen3.5-35B-A3B-APEX")
+
+		Expect(name).To(Equal("qwen3.5-35b-a3b"))
+		Expect(exists).To(BeFalse())
+	})
+
+	It("strips the MTP and TQ markers as well as APEX", func() {
+		ix := mustIndex("- name: qwen3.6-35b-a3b\n  url: u\n")
+
+		name, exists := ResolveHub(ix, "Qwen3.6-35B-A3B-APEX-MTP", "Qwen3.6-35B-A3B-APEX-MTP")
+
+		Expect(name).To(Equal("qwen3.6-35b-a3b"))
+		Expect(exists).To(BeTrue())
+	})
+})
+
+var _ = Describe("planHubs", func() {
+	noReuse := map[string]string{}
+	allAdded := func(names ...string) map[string]bool {
+		out := map[string]bool{}
+		for _, n := range names {
+			out[n] = true
+		}
+		return out
+	}
+
+	It("splices into the existing base entry instead of emitting an *-apex parent", func() {
+		ix := mustIndex("- name: step-3.7-flash\n  url: u\n- name: other\n  url: u\n")
+		fams := []family{{
+			repo:     "mudler/Step-3.7-Flash-APEX-GGUF",
+			repoBase: "Step-3.7-Flash-APEX",
+			stem:     "Step-3.7-Flash-APEX",
+			children: []childBuild{buildOf("step-3.7-flash-apex-i-quality", "mudler/Step-3.7-Flash-APEX-GGUF", "a.gguf", 0)},
+		}}
+
+		inserts, newHubs, err := planHubs(fams, ix, noReuse, allAdded("step-3.7-flash-apex-i-quality"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(newHubs).To(BeEmpty())
+		Expect(inserts).To(HaveLen(1))
+		Expect(inserts[0].Entry.Name).To(Equal("step-3.7-flash"))
+		Expect(inserts[0].Variants).To(Equal([]string{"step-3.7-flash-apex-i-quality"}))
+	})
+
+	It("merges into an entry that already declares variants, without repeating one", func() {
+		// The gallery's qwen3.6-35b-a3b already lists its APEX build. Re-adding it
+		// would put a duplicate key's worth of noise in the diff and a duplicate
+		// reference in the entry.
+		ix := mustIndex("- name: qwen3.6-35b-a3b\n  variants:\n    - model: qwen3.6-35b-a3b-apex\n  url: u\n" +
+			"- name: qwen3.6-35b-a3b-apex\n  url: u\n")
+		fams := []family{{
+			repo:     "mudler/Qwen3.6-35B-A3B-APEX-GGUF",
+			repoBase: "Qwen3.6-35B-A3B-APEX",
+			stem:     "Qwen3.6-35B-A3B-APEX",
+			children: []childBuild{buildOf("qwen3.6-35b-a3b-apex-i-quality", "mudler/Qwen3.6-35B-A3B-APEX-GGUF", "a.gguf", 0)},
+		}}
+
+		inserts, newHubs, err := planHubs(fams, ix, noReuse, allAdded("qwen3.6-35b-a3b-apex-i-quality"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(newHubs).To(BeEmpty())
+		Expect(inserts[0].Variants).To(Equal([]string{"qwen3.6-35b-a3b-apex-i-quality"}))
+
+		out, err := galleryedit.Apply(ix.Lines, inserts)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(strings.Count(strings.Join(out, "\n"), "variants:")).To(Equal(1))
+		Expect(out).To(HaveLen(len(ix.Lines) + 1))
+	})
+
+	It("never lets the hub reference itself", func() {
+		// An unsloth rung whose weights the gallery already ships under the base
+		// name resolves, through Merge, straight back to the hub. The verifier
+		// reads a self reference as a variant that declares variants of its own.
+		ix := mustIndex("- name: step-3.7-flash\n  url: u\n")
+		fams := []family{{
+			repo:     "mudler/Step-3.7-Flash-APEX-GGUF",
+			repoBase: "Step-3.7-Flash-APEX",
+			stem:     "Step-3.7-Flash-APEX",
+			children: []childBuild{buildOf("step-3.7-flash-ud-q4-k-m", "unsloth/Step-3.7-Flash-GGUF", "a.gguf", 100)},
+		}}
+
+		inserts, _, err := planHubs(fams, ix, map[string]string{"step-3.7-flash-ud-q4-k-m": "step-3.7-flash"}, map[string]bool{})
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(inserts).To(BeEmpty())
+	})
+
+	It("emits a hub named for the base model when the gallery has none", func() {
+		ix := mustIndex("- name: qwen3.5-35b-a3b-apex\n  url: u\n")
+		fams := []family{{
+			repo:      "mudler/Qwen3.5-35B-A3B-APEX-GGUF",
+			repoBase:  "Qwen3.5-35B-A3B-APEX",
+			stem:      "Qwen3.5-35B-A3B-APEX",
+			hasMMProj: true,
+			children: []childBuild{
+				buildOf("qwen3.5-35b-a3b-apex-i-quality", "mudler/Qwen3.5-35B-A3B-APEX-GGUF", "a.gguf", 0),
+				buildOf("qwen3.5-35b-a3b-ud-q6-k", "unsloth/Qwen3.5-35B-A3B-GGUF", "b.gguf", 102),
+			},
+		}}
+
+		inserts, newHubs, err := planHubs(fams, ix, noReuse,
+			allAdded("qwen3.5-35b-a3b-apex-i-quality", "qwen3.5-35b-a3b-ud-q6-k"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(inserts).To(BeEmpty())
+		Expect(newHubs).To(HaveLen(1))
+
+		hub := newHubs[0]
+		Expect(hub.Name).To(Equal("qwen3.5-35b-a3b"))
+		Expect(hub.Name).ToNot(HaveSuffix("-apex"))
+
+		// A hand-written *-apex entry is an ordinary build, referenced like any
+		// other rung and never deleted or renamed.
+		Expect(hub.Variants).To(Equal([]VariantRef{
+			{Model: "qwen3.5-35b-a3b-apex"},
+			{Model: "qwen3.5-35b-a3b-apex-i-quality"},
+			{Model: "qwen3.5-35b-a3b-ud-q6-k"},
+		}))
+
+		// The verifier skips entries with no declared backend, so a hub without
+		// one would escape the tagging check in silence.
+		Expect(hub.Overrides).To(HaveKeyWithValue("backend", "llama-cpp"))
+		Expect(hub.Files).ToNot(BeEmpty())
+		Expect(hub.Tags).To(ContainElement("vision"))
+	})
+
+	It("gathers two APEX repos that share one base model under a single hub", func() {
+		ix := mustIndex("- name: unrelated\n  url: u\n")
+		fams := []family{
+			{
+				repo:     "mudler/Solo-APEX-GGUF",
+				repoBase: "Solo-APEX",
+				stem:     "Solo-APEX",
+				children: []childBuild{buildOf("solo-apex-i-quality", "mudler/Solo-APEX-GGUF", "a.gguf", 0)},
+			},
+			{
+				repo:     "mudler/Solo-APEX-MTP-GGUF",
+				repoBase: "Solo-APEX-MTP",
+				stem:     "Solo-APEX-MTP",
+				children: []childBuild{buildOf("solo-apex-mtp-i-quality", "mudler/Solo-APEX-MTP-GGUF", "b.gguf", 0)},
+			},
+		}
+
+		_, newHubs, err := planHubs(fams, ix, noReuse, allAdded("solo-apex-i-quality", "solo-apex-mtp-i-quality"))
+
+		Expect(err).ToNot(HaveOccurred())
+		Expect(newHubs).To(HaveLen(1))
+		Expect(newHubs[0].Name).To(Equal("solo"))
+		Expect(newHubs[0].Variants).To(Equal([]VariantRef{
+			{Model: "solo-apex-i-quality"},
+			{Model: "solo-apex-mtp-i-quality"},
+		}))
+	})
+})
+
+var _ = Describe("hubVariants", func() {
+	It("orders builds by quality rung rather than discovery order", func() {
+		// DiscoverAPEXTiers preserves input order and the HF API returns siblings
+		// alphabetically, so an unsorted list reads I-Balanced, I-Compact, I-Mini,
+		// I-Nano, I-Quality. Selection ignores authored order; this is for the
+		// human reading the file.
+		f := family{repoBase: "X-APEX", stem: "X-APEX", children: []childBuild{
+			{rank: rungRank["I-Nano"], entry: GalleryEntry{Name: "x-i-nano"}},
+			{rank: 100, entry: GalleryEntry{Name: "x-ud-q4-k-m"}},
+			{rank: rungRank["I-Quality"], entry: GalleryEntry{Name: "x-i-quality"}},
+			{rank: rungRank["I-Compact"], entry: GalleryEntry{Name: "x-i-compact"}},
+		}}
+
+		got := hubVariants(&f, mustIndex("- name: x\n  url: u\n"), map[string]string{}, map[string]bool{})
+
+		Expect(got).To(Equal([]string{"x-i-quality", "x-i-compact", "x-i-nano", "x-ud-q4-k-m"}))
+	})
+})
+
+var _ = Describe("ParseIndexText", func() {
+	It("refuses to edit by line number when the two views of the file disagree", func() {
+		_, err := ParseIndexText("- name: one\n  url: u\n-\n")
+		Expect(err).To(MatchError(ContainSubstring("empty")))
+	})
+
+	It("records the line range of each entry", func() {
+		ix := mustIndex("- name: first\n  url: u\n- name: second\n  url: u\n")
+
+		Expect(ix.Find("FIRST").Pos.StartLine).To(Equal(0))
+		Expect(ix.Find("first").Pos.EndLine).To(Equal(2))
+		Expect(ix.Find("second").Pos.StartLine).To(Equal(2))
+	})
+})
+
+var _ = Describe("resolveVariant", func() {
+	It("keeps an entry that was emitted even when it is also in reused", func() {
+		// A within-batch name collision records reused[name] = name while the
+		// FIRST entry of that name is still in add. Treating presence in reused as
+		// "dropped" would emit nothing for it.
+		added := map[string]bool{"dup": true}
+		reused := map[string]string{"dup": "dup"}
+
+		Expect(resolveVariant("dup", reused, added)).To(Equal("dup"))
+	})
+
+	It("redirects a reused name at the entry that stands in for it", func() {
+		added := map[string]bool{}
+		reused := map[string]string{"generated": "already-in-gallery"}
+
+		Expect(resolveVariant("generated", reused, added)).To(Equal("already-in-gallery"))
+	})
+})
+
+var _ = Describe("slug", func() {
+	It("lowercases and turns quant underscores into hyphens", func() {
+		Expect(slug("UD-Q4_K_M")).To(Equal("ud-q4-k-m"))
+		Expect(slug("gemma-4-26B-A4B-it-APEX")).To(Equal("gemma-4-26b-a4b-it-apex"))
+		Expect(slug("I-Nano")).To(Equal("i-nano"))
+	})
+})
+
+var _ = Describe("sortTiers", func() {
+	It("puts the imatrix ladder in descending quality order", func() {
+		tiers := []Tier{
+			{Label: "I-Balanced"}, {Label: "I-Compact"}, {Label: "I-Mini"},
+			{Label: "I-Nano"}, {Label: "I-Quality"},
+		}
+		sortTiers(tiers)
+		Expect(tierLabels(tiers)).To(Equal("I-Quality,I-Balanced,I-Compact,I-Mini,I-Nano"))
+	})
+})
+
+var _ = Describe("restrict", func() {
+	It("keeps only the named repos", func() {
+		got := restrict([]string{"mudler/A-APEX-GGUF", "mudler/B-APEX-GGUF"}, "mudler/B-APEX-GGUF")
+		Expect(got).To(Equal([]string{"mudler/B-APEX-GGUF"}))
+	})
+
+	It("returns nothing when the filter matches nothing", func() {
+		Expect(restrict([]string{"mudler/A-APEX-GGUF"}, "mudler/typo")).To(BeEmpty())
+	})
+})
+
+var _ = Describe("reportUnclassified", func() {
+	// One real imatrix rung is always present so the specs measure how the
+	// remaining files are bucketed, not an empty-repo edge case.
+	tier := Tier{Label: "I-Quality", File: GGUFFile{Name: "Model-APEX-I-Quality.gguf"}}
+
+	censusOf := func(names ...string) fileCensus {
+		files := []GGUFFile{tier.File}
+		for _, n := range names {
+			files = append(files, GGUFFile{Name: n})
+		}
+		return reportUnclassified("mudler/Model-APEX-GGUF", files, []Tier{tier}, nil)
+	}
+
+	It("counts a flat full-precision source as excluded, not unclassified", func() {
+		got := censusOf("Carnice-MoE-35B-A3B-F16.gguf")
+		Expect(got.fullPrecision).To(Equal(1))
+		Expect(got.unclassified).To(Equal(0))
+	})
+
+	It("counts every shard of a sharded full-precision source as excluded", func() {
+		got := censusOf(
+			"MiniMax-M2.7-APEX-F16-00001-of-00003.gguf",
+			"MiniMax-M2.7-APEX-F16-00002-of-00003.gguf",
+			"MiniMax-M2.7-APEX-F16-00003-of-00003.gguf",
+		)
+		Expect(got.fullPrecision).To(Equal(3))
+		Expect(got.unclassified).To(Equal(0))
+	})
+
+	It("treats bf16 the same as f16, in either case", func() {
+		got := censusOf("Model-APEX-BF16.gguf", "Model-APEX-bf16-00001-of-00002.gguf", "Model-APEX-f16.gguf")
+		Expect(got.fullPrecision).To(Equal(3))
+		Expect(got.unclassified).To(Equal(0))
+	})
+
+	It("still reports a genuinely unknown filename as unclassified", func() {
+		got := censusOf("Model-APEX-Turbo.gguf")
+		Expect(got.unclassified).To(Equal(1))
+		Expect(got.fullPrecision).To(Equal(0))
+	})
+
+	It("separates the two kinds when a repo publishes both", func() {
+		got := censusOf("Model-APEX-F16.gguf", "Model-APEX-Turbo.gguf")
+		Expect(got.fullPrecision).To(Equal(1))
+		Expect(got.unclassified).To(Equal(1))
+	})
+})

.github/ci/apexentries/merge.go

@@ -0,0 +1,143 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+const (
+	hfShorthandPrefix = "huggingface://"
+	hfResolvePrefix   = "https://huggingface.co/"
+	hfResolveInfix    = "/resolve/main/"
+)
+
+// canonicalURI reduces the two interchangeable spellings of a HuggingFace file
+// to one key, so a generated resolve/main URI dedups against the shorthand the
+// gallery uses for the majority of its entries.
+//
+// The repo is exactly the first two path segments; everything after is the file
+// path, which may itself contain slashes because sharded quants live in a
+// subdirectory. Anything that is not recognisably one of the two forms is
+// returned unchanged rather than guessed at, so mirrors and other hosts still
+// dedup on their literal string.
+func canonicalURI(uri string) string {
+	switch {
+	case strings.HasPrefix(uri, hfShorthandPrefix):
+		rest := strings.TrimPrefix(uri, hfShorthandPrefix)
+		owner, after, ok := strings.Cut(rest, "/")
+		if !ok {
+			return uri
+		}
+		name, file, ok := strings.Cut(after, "/")
+		if !ok || owner == "" || name == "" || file == "" {
+			return uri
+		}
+		return hfShorthandPrefix + owner + "/" + name + "/" + file
+
+	case strings.HasPrefix(uri, hfResolvePrefix):
+		rest := strings.TrimPrefix(uri, hfResolvePrefix)
+		repo, file, ok := strings.Cut(rest, hfResolveInfix)
+		if !ok || file == "" {
+			return uri
+		}
+		// A repo is owner/name and nothing more; a longer prefix means this is
+		// some other huggingface.co URL that must not be rewritten.
+		owner, name, ok := strings.Cut(repo, "/")
+		if !ok || owner == "" || name == "" || strings.Contains(name, "/") {
+			return uri
+		}
+		return hfShorthandPrefix + repo + "/" + file
+
+	default:
+		return uri
+	}
+}
+
+// ExistingIndex is the lookup built from the current gallery: entry names, and
+// which entry claims each weight URI.
+type ExistingIndex struct {
+	ByName map[string]int
+	ByURI  map[string]string
+}
+
+// LoadExisting reads the gallery index for dedup purposes only. It is
+// deliberately not used to rewrite the file: the index is 40,000 lines, and a
+// YAML round trip would reflow the whole thing into an unreviewable diff.
+func LoadExisting(path string) (*ExistingIndex, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return nil, err
+	}
+
+	var entries []struct {
+		Name  string `yaml:"name"`
+		Files []struct {
+			URI string `yaml:"uri"`
+		} `yaml:"files"`
+	}
+	if err := yaml.Unmarshal(raw, &entries); err != nil {
+		return nil, fmt.Errorf("parsing %s: %w", path, err)
+	}
+
+	ix := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+	for i, e := range entries {
+		ix.ByName[e.Name] = i
+		for _, f := range e.Files {
+			if f.URI != "" {
+				ix.ByURI[canonicalURI(f.URI)] = e.Name
+			}
+		}
+	}
+	return ix, nil
+}
+
+// Merge splits generated entries into those to add and those already covered.
+// reused maps a generated name to the existing entry that stands in for it, so
+// a parent can reference what is already there instead of duplicating weights.
+// Several APEX repos share one base model, so the same counterpart rungs are
+// generated more than once in a batch. The batch has to dedup against itself as
+// well as against the gallery, tracked locally because the caller may reuse the
+// ExistingIndex it passed in.
+func Merge(existing *ExistingIndex, generated []GalleryEntry) (add []GalleryEntry, reused map[string]string) {
+	reused = map[string]string{}
+	batchNames := map[string]string{}
+	batchURIs := map[string]string{}
+
+	// Canonicalized into a local copy rather than in place: an ExistingIndex may
+	// be hand-built or reused by the caller, so Merge must not rewrite it.
+	existingURIs := make(map[string]string, len(existing.ByURI))
+	for uri, owner := range existing.ByURI {
+		existingURIs[canonicalURI(uri)] = owner
+	}
+
+	for _, e := range generated {
+		// Name is checked before URI: a name collision must block the add
+		// whatever the weights say, since duplicate names corrupt the index.
+		if _, clash := existing.ByName[e.Name]; clash {
+			reused[e.Name] = e.Name
+			continue
+		}
+		if claimant, clash := batchNames[e.Name]; clash {
+			reused[e.Name] = claimant
+			continue
+		}
+		if len(e.Files) > 0 {
+			uri := canonicalURI(e.Files[0].URI)
+			if owner, ok := existingURIs[uri]; ok {
+				reused[e.Name] = owner
+				continue
+			}
+			if claimant, ok := batchURIs[uri]; ok {
+				reused[e.Name] = claimant
+				continue
+			}
+			batchURIs[uri] = e.Name
+		}
+		batchNames[e.Name] = e.Name
+		add = append(add, e)
+	}
+	return add, reused
+}

.github/ci/apexentries/merge_test.go

@@ -0,0 +1,183 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Merge", func() {
+	It("drops a generated entry whose weight URI already exists and reports the existing name", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"qwen3.6-35b-a3b-apex": 0},
+			ByURI: map[string]string{
+				"https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Quality.gguf": "qwen3.6-35b-a3b-apex",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "x-apex-i-quality",
+			Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Quality.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("x-apex-i-quality", "qwen3.6-35b-a3b-apex"))
+	})
+
+	It("keeps a generated entry whose weights are new", func() {
+		existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+		gen := []GalleryEntry{{
+			Name:  "x-apex-i-mini",
+			Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Mini.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(reused).To(BeEmpty())
+	})
+
+	It("refuses to add an entry whose name collides with an existing one", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"x-apex-i-mini": 0},
+			ByURI:  map[string]string{},
+		}
+		gen := []GalleryEntry{{
+			Name:  "x-apex-i-mini",
+			Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/other.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("x-apex-i-mini", "x-apex-i-mini"))
+	})
+
+	// The gallery records most of its URIs in huggingface:// shorthand while
+	// render.go only ever emits the resolve/main form, so without
+	// canonicalization the majority of the file is invisible to the dedup.
+	It("matches a generated https URI against the shorthand form recorded in the gallery", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"foo-gguf-q8-0": 0},
+			ByURI: map[string]string{
+				"huggingface://unsloth/Foo-GGUF/Foo-Q8_0.gguf": "foo-gguf-q8-0",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "foo-apex-q8-0",
+			Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Foo-GGUF/resolve/main/Foo-Q8_0.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("foo-apex-q8-0", "foo-gguf-q8-0"))
+	})
+
+	It("matches a generated shorthand URI against the https form recorded in the gallery", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"foo-gguf-q8-0": 0},
+			ByURI: map[string]string{
+				"https://huggingface.co/unsloth/Foo-GGUF/resolve/main/Foo-Q8_0.gguf": "foo-gguf-q8-0",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "foo-apex-q8-0",
+			Files: []EntryFile{{URI: "huggingface://unsloth/Foo-GGUF/Foo-Q8_0.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("foo-apex-q8-0", "foo-gguf-q8-0"))
+	})
+
+	// Sharded quants live under a subdirectory, so the file path carries slashes
+	// of its own and only the first two segments are the repo.
+	It("matches across both forms when the file path has a subdirectory", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"model-ud-q4-k-m": 0},
+			ByURI: map[string]string{
+				"huggingface://unsloth/Model-GGUF/UD-Q4_K_M/Model-UD-Q4_K_M-00001-of-00002.gguf": "model-ud-q4-k-m",
+			},
+		}
+		gen := []GalleryEntry{{
+			Name:  "model-apex-ud-q4-k-m",
+			Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Model-GGUF/resolve/main/UD-Q4_K_M/Model-UD-Q4_K_M-00001-of-00002.gguf"}},
+		}}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(BeEmpty())
+		Expect(reused).To(HaveKeyWithValue("model-apex-ud-q4-k-m", "model-ud-q4-k-m"))
+	})
+
+	// Several APEX repos share one base model, so the same unsloth rungs are
+	// generated more than once in a single batch.
+	It("adds only the first of two generated entries sharing a name", func() {
+		existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+		gen := []GalleryEntry{
+			{
+				Name:  "shared-rung-q8-0",
+				Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"}},
+			},
+			{
+				Name:  "shared-rung-q8-0",
+				Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Other-GGUF/resolve/main/Other-Q8_0.gguf"}},
+			},
+		}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(add[0].Files[0].URI).To(Equal("https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"))
+		Expect(reused).To(HaveKeyWithValue("shared-rung-q8-0", "shared-rung-q8-0"))
+	})
+
+	It("adds only the first of two generated entries sharing a primary URI", func() {
+		existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
+		gen := []GalleryEntry{
+			{
+				Name:  "shared-rung-from-apex",
+				Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"}},
+			},
+			{
+				Name:  "shared-rung-from-apex-mtp",
+				Files: []EntryFile{{URI: "huggingface://unsloth/Shared-GGUF/Shared-Q8_0.gguf"}},
+			},
+		}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(add[0].Name).To(Equal("shared-rung-from-apex"))
+		Expect(reused).To(HaveKeyWithValue("shared-rung-from-apex-mtp", "shared-rung-from-apex"))
+	})
+
+	// Anything that is not a HuggingFace URI must survive untouched, so an
+	// unrecognised scheme still dedups against the very same string.
+	It("leaves a URI in neither recognised form alone and still dedups it exactly", func() {
+		existing := &ExistingIndex{
+			ByName: map[string]int{"mirrored-model": 0},
+			ByURI: map[string]string{
+				"https://mirror.example.com/weights/Model-Q8_0.gguf": "mirrored-model",
+			},
+		}
+		gen := []GalleryEntry{
+			{
+				Name:  "mirrored-apex",
+				Files: []EntryFile{{URI: "https://mirror.example.com/weights/Model-Q8_0.gguf"}},
+			},
+			{
+				Name:  "elsewhere-apex",
+				Files: []EntryFile{{URI: "https://mirror.example.com/weights/Other-Q8_0.gguf"}},
+			},
+		}
+
+		add, reused := Merge(existing, gen)
+
+		Expect(add).To(HaveLen(1))
+		Expect(add[0].Name).To(Equal("elsewhere-apex"))
+		Expect(reused).To(HaveKeyWithValue("mirrored-apex", "mirrored-model"))
+	})
+})

.github/ci/apexentries/render.go

@@ -0,0 +1,175 @@
+package main
+
+import (
+	"fmt"
+	"path"
+	"strings"
+)
+
+// EntryFile is one downloadable file of a gallery entry.
+type EntryFile struct {
+	Filename string `yaml:"filename"`
+	SHA256   string `yaml:"sha256"`
+	URI      string `yaml:"uri"`
+}
+
+// GalleryEntry is the subset of a gallery entry this generator writes.
+//
+// Named GalleryEntry rather than Entry because the test files dot-import
+// Ginkgo, whose table DSL exports an Entry that a package-level Entry would
+// collide with. The yaml tags are what the gallery index sees, so the Go
+// identifier is free to differ.
+type GalleryEntry struct {
+	Name        string         `yaml:"name"`
+	URL         string         `yaml:"url"`
+	Description string         `yaml:"description,omitempty"`
+	Tags        []string       `yaml:"tags,omitempty"`
+	Overrides   map[string]any `yaml:"overrides,omitempty"`
+	Files       []EntryFile    `yaml:"files,omitempty"`
+	Variants    []VariantRef   `yaml:"variants,omitempty"`
+}
+
+// VariantRef mirrors the gallery's variant reference: a name and nothing else.
+type VariantRef struct {
+	Model string `yaml:"model"`
+}
+
+// ChildInput is everything needed to render one non-parent entry.
+type ChildInput struct {
+	Name string
+	Repo string
+	// DraftRepo is the repo publishing the drafter, when it is not the repo
+	// publishing the weights. Speculative pairings routinely cross repos, so
+	// the drafter cannot be assumed to sit next to the weights. Empty means
+	// same-repo, which is how the *-APEX-MTP-GGUF repos ship.
+	DraftRepo string
+	Template  string
+	Weights   []GGUFFile
+	MMProj    *GGUFFile
+	SpecType  string
+	DraftFile *GGUFFile
+	BaseTags  []string
+}
+
+// specTuning is the acceptance-window tuning each spec type ships with, copied
+// from the hand-written entries that already run these two mechanisms rather
+// than invented here. The two differ because the drafters differ: self-drafted
+// MTP heads produce a short, high-confidence proposal (15+ hand-written entries
+// use 6 with a 0.75 floor), while a separate DFlash drafter is cheap enough to
+// run far ahead unconditionally (the five hand-written dflash entries use 15 and
+// set no floor).
+var specTuning = map[string][]string{
+	"draft-mtp":    {"spec_n_max:6", "spec_p_min:0.75"},
+	"draft-dflash": {"spec_n_max:15"},
+}
+
+func hfURI(repo, file string) string {
+	return fmt.Sprintf("https://huggingface.co/%s/resolve/main/%s", repo, file)
+}
+
+// localPath is where a downloaded file lands.
+//
+// The hand-written entries namespace by the repo's BARE name
+// (llama-cpp/models/<repo>/<file>), which is not unique. LiquidAI/LFM2.5-8B-A1B-GGUF
+// and unsloth/LFM2.5-8B-A1B-GGUF share a basename, so both claim
+// llama-cpp/models/LFM2.5-8B-A1B-GGUF/, and installing the second after the first
+// either overwrites weights whose recorded sha256 belongs to the other file or is
+// skipped as already present. Two owners publishing the same model name is the
+// normal case for quantizers, not an edge case, so the owner has to be in the path.
+//
+// The owner becomes its own path segment rather than being folded into the
+// directory name: owner/repo is unique on HuggingFace and "/" cannot occur inside
+// either half, so this is the only form that is collision-proof by construction.
+// It still reads as the hand-written convention with the owner restored, and the
+// extra depth is already present in the index for sharded builds.
+func localPath(kind, repo, file string) string {
+	// path.Dir yields "." for a repo named without an owner, which path.Join
+	// drops, so such a caller keeps the historical two-segment layout.
+	return path.Join("llama-cpp", kind, path.Dir(repo), path.Base(repo), file)
+}
+
+// RenderChild builds one child entry.
+//
+// The dflash/mtp tag is added if and only if this entry sets a spec_type,
+// because variant ranking reads tags and nothing else, and a tag that does not
+// match what the entry configures either promotes a build that is no faster or
+// hides one that is.
+func RenderChild(in ChildInput) GalleryEntry {
+	e := GalleryEntry{
+		Name:      in.Name,
+		URL:       fmt.Sprintf("github:mudler/LocalAI/gallery/%s@master", in.Template),
+		Tags:      append([]string{}, in.BaseTags...),
+		Overrides: map[string]any{},
+	}
+
+	// gallery/virtual.yaml carries no backend, so nothing else would name an
+	// engine for these entries. Matching the hand-written entries on
+	// known_usecases too: LocalAI would fall back to the backend defaults, but
+	// generated entries should not read differently from their neighbours.
+	e.Overrides["backend"] = "llama-cpp"
+	e.Overrides["known_usecases"] = []string{"chat"}
+
+	options := []string{"use_jinja:true"}
+
+	for _, w := range in.Weights {
+		e.Files = append(e.Files, EntryFile{
+			Filename: localPath("models", in.Repo, w.Name),
+			SHA256:   w.SHA256,
+			URI:      hfURI(in.Repo, w.Name),
+		})
+	}
+	e.Overrides["parameters"] = map[string]any{
+		"model": localPath("models", in.Repo, in.Weights[0].Name),
+	}
+
+	if in.MMProj != nil {
+		// An explicit known_usecases SUPPRESSES the backend-default fallback in
+		// core/gallery/models_types.go, so a multimodal entry left at chat-only
+		// never matches FilterGalleryModelsByUsecase(FLAG_VISION) or
+		// FilterGalleryModelsByMultimodal and vanishes from the UI's vision and
+		// multimodal filters. 19 of the 45 APEX repos ship an mmproj.
+		e.Overrides["known_usecases"] = []string{"chat", "vision"}
+		e.Overrides["mmproj"] = localPath("mmproj", in.Repo, in.MMProj.Name)
+		e.Files = append(e.Files, EntryFile{
+			Filename: localPath("mmproj", in.Repo, in.MMProj.Name),
+			SHA256:   in.MMProj.SHA256,
+			URI:      hfURI(in.Repo, in.MMProj.Name),
+		})
+	}
+
+	// A spec type is configured independently of a drafter FILE. Weights that
+	// carry their own MTP heads need no second download, and requiring one left
+	// the *-APEX-MTP-GGUF builds shipping the larger heads-bearing weights with
+	// the heads switched off: a strictly bigger download at the same speed,
+	// ranked identically to the plain rung at the same tier.
+	if in.SpecType != "" {
+		options = append(options, "spec_type:"+in.SpecType)
+		options = append(options, specTuning[in.SpecType]...)
+		// The tag is derived from the spec type this entry sets and from nothing
+		// else. Variant ranking reads tags only, so a tag taken from a repo or
+		// entry NAME would promote a build that is no faster whenever the name
+		// and the configuration disagree.
+		e.Tags = append(e.Tags, strings.TrimPrefix(in.SpecType, "draft-"))
+	}
+
+	if in.SpecType != "" && in.DraftFile != nil {
+		// Fall back to the weights repo so pairings that publish the drafter
+		// alongside the weights keep working without restating the repo.
+		draftRepo := in.DraftRepo
+		if draftRepo == "" {
+			draftRepo = in.Repo
+		}
+		draftPath := localPath("models", draftRepo, in.DraftFile.Name)
+
+		e.Overrides["draft_model"] = draftPath
+		e.Overrides["flash_attention"] = "on"
+		e.Files = append(e.Files, EntryFile{
+			Filename: draftPath,
+			SHA256:   in.DraftFile.SHA256,
+			URI:      hfURI(draftRepo, in.DraftFile.Name),
+		})
+	}
+
+	e.Overrides["options"] = options
+	return e
+}

.github/ci/apexentries/render_test.go

@@ -0,0 +1,249 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("RenderChild", func() {
+	It("tags an entry that configures draft-dflash", func() {
+		e := RenderChild(ChildInput{
+			Name:      "qwen3.5-9b-dflash",
+			Repo:      "mudler/Example-APEX-GGUF",
+			Template:  "virtual.yaml",
+			Weights:   []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			SpecType:  "draft-dflash",
+			DraftFile: &GGUFFile{Name: "Example-DFlash.Q8_0.gguf", SHA256: "b"},
+			BaseTags:  []string{"llm", "gguf"},
+		})
+
+		Expect(e.Tags).To(ContainElement("dflash"))
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Overrides["options"]).To(ContainElement("spec_type:draft-dflash"))
+		Expect(e.Overrides["draft_model"]).ToNot(BeNil())
+	})
+
+	It("does not tag an MTP-named repo that configures no speculation", func() {
+		// mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF ships MTP-bearing weights. Weights
+		// that carry the heads are not an entry that enables them, and tagging it
+		// would win the feature axis without being any faster.
+		e := RenderChild(ChildInput{
+			Name:     "qwen3.6-35b-a3b-apex-mtp-i-quality",
+			Repo:     "mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Qwen3.6-35B-A3B-APEX-MTP-I-Quality.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Tags).ToNot(ContainElement("dflash"))
+		Expect(e.Overrides).ToNot(HaveKey("draft_model"))
+	})
+
+	It("lists every shard of a sharded build and points the model at the first", func() {
+		e := RenderChild(ChildInput{
+			Name:     "step-3.7-flash-ud-q4-k-m",
+			Repo:     "unsloth/Step-3.7-Flash-GGUF",
+			Template: "virtual.yaml",
+			Weights: []GGUFFile{
+				{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf", SHA256: "a"},
+				{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00002-of-00002.gguf", SHA256: "b"},
+			},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Files).To(HaveLen(2))
+		params, ok := e.Overrides["parameters"].(map[string]any)
+		Expect(ok).To(BeTrue())
+		Expect(params["model"]).To(HaveSuffix("00001-of-00002.gguf"))
+		Expect(e.Files[0].URI).To(Equal(
+			"https://huggingface.co/unsloth/Step-3.7-Flash-GGUF/resolve/main/UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf"))
+	})
+
+	It("wires mmproj when the repo publishes one", func() {
+		e := RenderChild(ChildInput{
+			Name:     "example-i-mini",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Mini.gguf", SHA256: "a"}},
+			MMProj:   &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["mmproj"]).ToNot(BeNil())
+		Expect(e.Files).To(HaveLen(2))
+	})
+
+	It("names the engine and the usecases the hand-written entries name", func() {
+		// gallery/virtual.yaml supplies no backend, so an entry that omits one
+		// names no engine at all and cannot load.
+		e := RenderChild(ChildInput{
+			Name:     "example-i-mini",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Mini.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["backend"]).To(Equal("llama-cpp"))
+		Expect(e.Overrides["known_usecases"]).To(ContainElement("chat"))
+	})
+
+	It("draws the drafter from DraftRepo when the pairing spans two repos", func() {
+		// unsloth/Qwen3-4B-GGUF pairs with a drafter published separately by
+		// AtomicChat, so a drafter URI built from the weights repo 404s.
+		e := RenderChild(ChildInput{
+			Name:      "qwen3-4b-dflash",
+			Repo:      "unsloth/Qwen3-4B-GGUF",
+			DraftRepo: "AtomicChat/Qwen3-4B-DFlash-GGUF",
+			Template:  "virtual.yaml",
+			Weights:   []GGUFFile{{Name: "Qwen3-4B-Q4_K_M.gguf", SHA256: "a"}},
+			SpecType:  "draft-dflash",
+			DraftFile: &GGUFFile{Name: "Qwen3-4B-DFlash.Q8_0.gguf", SHA256: "b"},
+			BaseTags:  []string{"llm", "gguf"},
+		})
+
+		Expect(e.Files[0].URI).To(Equal(
+			"https://huggingface.co/unsloth/Qwen3-4B-GGUF/resolve/main/Qwen3-4B-Q4_K_M.gguf"))
+		Expect(e.Files[1].URI).To(Equal(
+			"https://huggingface.co/AtomicChat/Qwen3-4B-DFlash-GGUF/resolve/main/Qwen3-4B-DFlash.Q8_0.gguf"))
+		Expect(e.Files[1].Filename).To(Equal(
+			"llama-cpp/models/AtomicChat/Qwen3-4B-DFlash-GGUF/Qwen3-4B-DFlash.Q8_0.gguf"))
+		Expect(e.Overrides["draft_model"]).To(Equal(
+			"llama-cpp/models/AtomicChat/Qwen3-4B-DFlash-GGUF/Qwen3-4B-DFlash.Q8_0.gguf"))
+	})
+
+	It("falls back to the weights repo for the drafter when DraftRepo is empty", func() {
+		// The *-APEX-MTP-GGUF repos ship the drafter alongside the weights.
+		e := RenderChild(ChildInput{
+			Name:      "example-apex-dflash",
+			Repo:      "mudler/Example-APEX-GGUF",
+			Template:  "virtual.yaml",
+			Weights:   []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			SpecType:  "draft-dflash",
+			DraftFile: &GGUFFile{Name: "Example-DFlash.Q8_0.gguf", SHA256: "b"},
+			BaseTags:  []string{"llm", "gguf"},
+		})
+
+		Expect(e.Files[1].URI).To(Equal(
+			"https://huggingface.co/mudler/Example-APEX-GGUF/resolve/main/Example-DFlash.Q8_0.gguf"))
+		Expect(e.Files[1].Filename).To(Equal(
+			"llama-cpp/models/mudler/Example-APEX-GGUF/Example-DFlash.Q8_0.gguf"))
+	})
+})
+
+var _ = Describe("RenderChild known_usecases", func() {
+	It("declares vision alongside chat when the entry carries an mmproj", func() {
+		// An explicit known_usecases suppresses the backend-default fallback, so a
+		// chat-only multimodal entry disappears from the UI's vision filter.
+		e := RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			MMProj:   &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["known_usecases"]).To(ConsistOf("chat", "vision"))
+	})
+
+	It("leaves a text-only entry at chat", func() {
+		e := RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["known_usecases"]).To(ConsistOf("chat"))
+	})
+})
+
+var _ = Describe("localPath", func() {
+	It("keeps two repos with the same basename but different owners apart", func() {
+		// LiquidAI and unsloth both publish LFM2.5-8B-A1B-GGUF. A path built from
+		// the bare repo name gives both the same local file, so installing the
+		// second overwrites or skips the first and one of them then serves bytes
+		// that do not match its recorded sha256.
+		liquid := RenderChild(ChildInput{
+			Name:     "lfm2.5-8b-a1b-i-quality",
+			Repo:     "LiquidAI/LFM2.5-8B-A1B-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "LFM2.5-8B-A1B-Q8_0.gguf", SHA256: "33ab3b8c"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+		unsloth := RenderChild(ChildInput{
+			Name:     "lfm2.5-8b-a1b-q8-0",
+			Repo:     "unsloth/LFM2.5-8B-A1B-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "LFM2.5-8B-A1B-Q8_0.gguf", SHA256: "ec11666b"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(liquid.Files[0].Filename).ToNot(Equal(unsloth.Files[0].Filename))
+		Expect(unsloth.Files[0].Filename).To(Equal(
+			"llama-cpp/models/unsloth/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf"))
+	})
+
+	It("namespaces the mmproj by owner too", func() {
+		e := RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     "mudler/Example-APEX-GGUF",
+			Template: "virtual.yaml",
+			Weights:  []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
+			MMProj:   &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
+			BaseTags: []string{"llm", "gguf"},
+		})
+
+		Expect(e.Overrides["mmproj"]).To(Equal(
+			"llama-cpp/mmproj/mudler/Example-APEX-GGUF/mmproj-F16.gguf"))
+	})
+})
+
+var _ = Describe("MTP builds", func() {
+	renderTier := func(repo string) GalleryEntry {
+		return RenderChild(ChildInput{
+			Name:     "example-i-quality",
+			Repo:     repo,
+			Template: "virtual.yaml",
+			SpecType: SpecTypeForRepo(repo),
+			Weights:  []GGUFFile{{Name: "Example-I-Quality.gguf", SHA256: "a"}},
+			BaseTags: []string{"llm", "gguf"},
+		})
+	}
+
+	It("turns MTP on for a build off an APEX-MTP repo", func() {
+		// These weights retain the model's own MTP heads, so shipping them with
+		// speculation off is a strictly larger download at the same speed,
+		// ranked identically to the plain rung at the same tier.
+		e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF")
+
+		Expect(e.Overrides["options"]).To(ContainElements(
+			"spec_type:draft-mtp", "spec_n_max:6", "spec_p_min:0.75"))
+		Expect(e.Tags).To(ContainElement("mtp"))
+	})
+
+	It("needs no drafter file, because the heads travel with the weights", func() {
+		e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF")
+
+		Expect(e.Overrides).ToNot(HaveKey("draft_model"))
+		Expect(e.Files).To(HaveLen(1))
+	})
+
+	It("leaves a build off a plain APEX repo alone", func() {
+		e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-GGUF")
+
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Overrides["options"]).To(ConsistOf("use_jinja:true"))
+	})
+
+	It("leaves an unsloth counterpart rung alone", func() {
+		// The counterpart quantizes the plain weights; nothing there carries heads.
+		e := renderTier("unsloth/Qwen3.6-35B-A3B-GGUF")
+
+		Expect(e.Tags).ToNot(ContainElement("mtp"))
+		Expect(e.Overrides["options"]).To(ConsistOf("use_jinja:true"))
+	})
+})

.github/ci/apexentries/unsloth.go

@@ -0,0 +1,71 @@
+package main
+
+import (
+	"regexp"
+	"sort"
+	"strings"
+)
+
+// WantedQuants is the fixed unsloth subset this generator emits. It is a
+// deliberate subset: unsloth publishes north of 20 quants per repo, and the
+// selector needs useful fitness points rather than every rung.
+var WantedQuants = []string{"UD-Q4_K_M", "UD-Q5_K_M", "UD-Q6_K", "Q8_0"}
+
+var shardRE = regexp.MustCompile(`-(\d{5})-of-(\d{5})\.gguf$`)
+
+// QuantBuild is one unsloth quantization, which may be a single file or an
+// ordered set of shards.
+type QuantBuild struct {
+	Quant   string
+	Files   []GGUFFile
+	Sharded bool
+}
+
+// CounterpartCandidates returns the unsloth repo base names worth probing, most
+// likely first. Both derivations are needed: the repo name finds
+// unsloth/gemma-4-26B-A4B-it-GGUF, while the file stem is what matches for
+// repos whose stem is the canonical model name.
+func CounterpartCandidates(repoName, fileStem string) []string {
+	clean := func(s string) string {
+		s = strings.TrimSuffix(s, "-GGUF")
+		s = regexp.MustCompile(`-(MTP|TQ)$`).ReplaceAllString(s, "")
+		s = strings.TrimSuffix(s, "-APEX")
+		return regexp.MustCompile(`-(MTP|TQ)$`).ReplaceAllString(s, "")
+	}
+
+	out := []string{clean(repoName)}
+	if stem := clean(fileStem); stem != out[0] {
+		out = append(out, stem)
+	}
+	return out
+}
+
+// DiscoverUnslothQuants returns the wanted quants a repo publishes, handling
+// both the flat single-file layout and the sharded layout where a quant lives
+// in its own subdirectory.
+func DiscoverUnslothQuants(files []GGUFFile) []QuantBuild {
+	var out []QuantBuild
+
+	for _, q := range WantedQuants {
+		var flat []GGUFFile
+		var shards []GGUFFile
+
+		for _, f := range files {
+			switch {
+			case !strings.Contains(f.Name, "/") && strings.HasSuffix(f.Name, "-"+q+".gguf"):
+				flat = append(flat, f)
+			case strings.HasPrefix(f.Name, q+"/") && shardRE.MatchString(f.Name):
+				shards = append(shards, f)
+			}
+		}
+
+		switch {
+		case len(flat) > 0:
+			out = append(out, QuantBuild{Quant: q, Files: flat})
+		case len(shards) > 0:
+			sort.Slice(shards, func(i, j int) bool { return shards[i].Name < shards[j].Name })
+			out = append(out, QuantBuild{Quant: q, Files: shards, Sharded: true})
+		}
+	}
+	return out
+}

.github/ci/apexentries/unsloth_test.go

@@ -0,0 +1,75 @@
+package main
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("CounterpartCandidates", func() {
+	It("offers both the repo-derived and stem-derived names", func() {
+		// mudler/gemma-4-26B-A4B-it-APEX-GGUF ships gemma-4-26B-A4B-APEX-*.gguf,
+		// and only the repo-derived name finds unsloth/gemma-4-26B-A4B-it-GGUF.
+		got := CounterpartCandidates("gemma-4-26B-A4B-it-APEX-GGUF", "gemma-4-26B-A4B-APEX")
+
+		Expect(got).To(Equal([]string{"gemma-4-26B-A4B-it", "gemma-4-26B-A4B"}))
+	})
+
+	It("strips the MTP marker", func() {
+		got := CounterpartCandidates("Qwopus3.6-35B-A3B-v1-APEX-MTP-GGUF", "Qwopus3.6-35B-A3B-v1-APEX-MTP")
+
+		Expect(got[0]).To(Equal("Qwopus3.6-35B-A3B-v1"))
+	})
+
+	It("strips the TQ marker", func() {
+		// This is the branch that folds mudler/Qwen3.5-35B-A3B-APEX-TQ-GGUF into
+		// the qwen3.5-35b-a3b hub. Without it the probe is
+		// unsloth/Qwen3.5-35B-A3B-TQ-GGUF, which does not exist, so the family
+		// silently loses every unsloth rung.
+		got := CounterpartCandidates("Qwen3.5-35B-A3B-APEX-TQ-GGUF", "Qwen3.5-35B-A3B-APEX-TQ")
+
+		Expect(got).To(Equal([]string{"Qwen3.5-35B-A3B"}))
+	})
+
+	It("does not repeat a candidate when both derivations agree", func() {
+		got := CounterpartCandidates("Qwen3.6-35B-A3B-APEX-GGUF", "Qwen3.6-35B-A3B-APEX")
+
+		Expect(got).To(Equal([]string{"Qwen3.6-35B-A3B"}))
+	})
+})
+
+var _ = Describe("DiscoverUnslothQuants", func() {
+	It("finds flat single-file quants", func() {
+		files := []GGUFFile{
+			{Name: "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf", SHA256: "a"},
+			{Name: "Qwen3.6-35B-A3B-UD-IQ1_M.gguf", SHA256: "b"},
+		}
+
+		got := DiscoverUnslothQuants(files)
+
+		Expect(got).To(HaveLen(1))
+		Expect(got[0].Quant).To(Equal("UD-Q4_K_M"))
+		Expect(got[0].Sharded).To(BeFalse())
+		Expect(got[0].Files).To(HaveLen(1))
+	})
+
+	It("collects a sharded quant from its subdirectory in shard order", func() {
+		files := []GGUFFile{
+			{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00002-of-00002.gguf", SHA256: "b"},
+			{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf", SHA256: "a"},
+		}
+
+		got := DiscoverUnslothQuants(files)
+
+		Expect(got).To(HaveLen(1))
+		Expect(got[0].Quant).To(Equal("UD-Q4_K_M"))
+		Expect(got[0].Sharded).To(BeTrue())
+		Expect(got[0].Files).To(HaveLen(2))
+		Expect(got[0].Files[0].Name).To(HaveSuffix("00001-of-00002.gguf"))
+	})
+
+	It("ignores quants outside the wanted subset", func() {
+		files := []GGUFFile{{Name: "Model-UD-IQ2_XXS.gguf", SHA256: "a"}}
+
+		Expect(DiscoverUnslothQuants(files)).To(BeEmpty())
+	})
+})

.github/ci/apexentries/verify.go

@@ -0,0 +1,312 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+type verifyEntry struct {
+	Name      string       `yaml:"name"`
+	Tags      []string     `yaml:"tags"`
+	Variants  []VariantRef `yaml:"variants"`
+	Overrides struct {
+		// Backend scopes the checks that only hold for one engine. An entry that
+		// declares none takes its configuration from the referenced url: template,
+		// which this verifier never reads, so it cannot be judged either way.
+		Backend string   `yaml:"backend"`
+		Options []string `yaml:"options"`
+		// MMProj and DraftModel name the files that are not weights. They are
+		// the only signal for it: a drafter lands in the same models/ prefix as
+		// the weights, so the path alone cannot tell them apart.
+		MMProj     string `yaml:"mmproj"`
+		DraftModel string `yaml:"draft_model"`
+	} `yaml:"overrides"`
+	Files []struct {
+		Filename string `yaml:"filename"`
+		SHA256   string `yaml:"sha256"`
+		URI      string `yaml:"uri"`
+	} `yaml:"files"`
+}
+
+// Verify checks the invariants the variants schema and the tagging rule
+// require. It returns every problem rather than the first, so one run tells the
+// author everything that needs fixing.
+func Verify(path string) []string {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return []string{fmt.Sprintf("reading %s: %v", path, err)}
+	}
+
+	var entries []verifyEntry
+	if err := yaml.Unmarshal(raw, &entries); err != nil {
+		return []string{fmt.Sprintf("parsing %s: %v", path, err)}
+	}
+
+	var problems []string
+
+	byName := map[string]verifyEntry{}
+	for _, e := range entries {
+		if _, seen := byName[e.Name]; seen {
+			problems = append(problems, fmt.Sprintf("duplicate entry name: %s", e.Name))
+			continue
+		}
+		byName[e.Name] = e
+	}
+
+	for _, e := range entries {
+		for _, v := range e.Variants {
+			target, ok := byName[v.Model]
+			if !ok {
+				problems = append(problems, fmt.Sprintf("%s: variant %q does not exist", e.Name, v.Model))
+				continue
+			}
+			if len(target.Variants) > 0 {
+				problems = append(problems, fmt.Sprintf("%s: variant %q declares variants of its own", e.Name, v.Model))
+			}
+		}
+
+		for _, f := range e.Files {
+			if requiresSHA256(f.Filename) && f.SHA256 == "" {
+				problems = append(problems, fmt.Sprintf("%s: file %s has no sha256", e.Name, f.Filename))
+			}
+		}
+
+		problems = append(problems, checkWeightCount(e)...)
+		problems = append(problems, checkFeatureTag(e, "dflash")...)
+		problems = append(problems, checkFeatureTag(e, "mtp")...)
+	}
+
+	problems = append(problems, checkPathCollisions(entries)...)
+
+	return problems
+}
+
+// checkPathCollisions catches two different upstream files claiming one local
+// path. The install layer keys on the local filename, so whichever entry is
+// installed second either overwrites weights the first entry recorded a
+// different sha256 for or is skipped as already present. Either way some entry
+// afterwards serves bytes that do not match its own checksum, and nothing at
+// install time says so.
+//
+// This is an index-wide invariant rather than a per-entry one: neither entry is
+// wrong on its own and the collision exists only in their pairing. The usual
+// source is a path scheme built from the repo's BARE name, because two owners
+// publishing the same model name is routine for quantizers.
+//
+// Sharing a path is fine when the uri is the same, which is how several entries
+// legitimately reuse one projector. Files with no uri are skipped: there is
+// nothing to compare.
+func checkPathCollisions(entries []verifyEntry) []string {
+	type source struct{ uri, entry string }
+
+	first := map[string]source{}
+	reported := map[string]bool{}
+
+	var problems []string
+	for _, e := range entries {
+		for _, f := range e.Files {
+			if f.Filename == "" || f.URI == "" {
+				continue
+			}
+			prev, seen := first[f.Filename]
+			if !seen {
+				first[f.Filename] = source{uri: f.URI, entry: e.Name}
+				continue
+			}
+			if prev.uri == f.URI || reported[f.Filename] {
+				continue
+			}
+			// Reported once per path however many entries pile onto it, so one
+			// heavily reused filename cannot bury the rest of the report.
+			reported[f.Filename] = true
+			problems = append(problems, fmt.Sprintf(
+				"local path %s is claimed by two different uris: %s (%s) and %s (%s)",
+				f.Filename, prev.uri, prev.entry, f.URI, e.Name))
+		}
+	}
+	return problems
+}
+
+// auxiliaryExtensions are the metadata formats an entry ships beside its
+// weights, where an unverified download is a nuisance rather than a hole.
+//
+// The exclusion is stated as a list of metadata formats on purpose. Requiring
+// the checksum only on a blessed list of weight formats would silently exempt
+// every format nobody has shipped yet, and it already exempted safetensors
+// weights, which are downloaded and loaded exactly like GGUF ones.
+var auxiliaryExtensions = []string{".json", ".txt", ".md"}
+
+// requiresSHA256 reports whether an unverified download of this file would be
+// a supply-chain hole rather than a cosmetic gap.
+func requiresSHA256(filename string) bool {
+	for _, ext := range auxiliaryExtensions {
+		if strings.HasSuffix(filename, ext) {
+			return false
+		}
+	}
+	return true
+}
+
+// checkWeightCount catches an entry carrying two whole models. The flat-match
+// branch in DiscoverUnslothQuants appends every match, so a quant label that is
+// a suffix of another one (Q8_0 of UD-Q8_0) collects both files into one build
+// while the rendered model: points at only the first. The result downloads
+// twice the bytes and serves whichever file sorted first, silently.
+//
+// Shards are exempt because a sharded build is legitimately many files.
+//
+// The collision is a property of llama-cpp quant discovery, so the check is
+// scoped to that backend. Multi-component TTS, ASR and diffusion engines ship an
+// encoder, a decoder and a vocoder as one model, and there the second GGUF is
+// the design rather than a bug.
+func checkWeightCount(e verifyEntry) []string {
+	if e.Overrides.Backend != "llama-cpp" {
+		return nil
+	}
+
+	var weights []string
+	for _, f := range e.Files {
+		switch {
+		case !strings.HasSuffix(f.Filename, ".gguf"):
+		case shardRE.MatchString(f.Filename):
+		case f.Filename == e.Overrides.MMProj:
+		case f.Filename == e.Overrides.DraftModel:
+		default:
+			weights = append(weights, f.Filename)
+		}
+	}
+
+	if len(weights) > 1 {
+		return []string{fmt.Sprintf("%s: more than one weight file: %s", e.Name, strings.Join(weights, ", "))}
+	}
+	return nil
+}
+
+// checkFeatureTag enforces the rule in both directions. A tag without the
+// configuration promotes a build that is no faster; configuration without the
+// tag leaves a genuinely faster build ranked as plain.
+//
+// It only speaks about backends whose declaration it can actually read, because
+// a rule applied where the evidence is invisible reports noise rather than bugs.
+func checkFeatureTag(e verifyEntry, feature string) []string {
+	decl, configured, judgeable := featureDeclaration(e, feature)
+	if !judgeable {
+		return nil
+	}
+
+	tagged := false
+	for _, t := range e.Tags {
+		if t == feature {
+			tagged = true
+			break
+		}
+	}
+
+	switch {
+	case tagged && !configured:
+		return []string{fmt.Sprintf("%s: tagged %s but sets no %s", e.Name, feature, decl)}
+	case configured && !tagged:
+		return []string{fmt.Sprintf("%s: sets %s but is not tagged %s", e.Name, decl, feature)}
+	}
+	return nil
+}
+
+// featureDeclaration implements the per-backend table in
+// .agents/adding-gallery-models.md. It returns the declaration the backend uses
+// to configure the feature, whether the entry carries it, and whether this
+// verifier is in a position to answer at all.
+func featureDeclaration(e verifyEntry, feature string) (decl string, configured, judgeable bool) {
+	switch e.Overrides.Backend {
+	case "llama-cpp":
+		decl = "spec_type:draft-" + feature
+		for _, o := range e.Overrides.Options {
+			if strings.TrimSpace(o) == decl {
+				return decl, true, true
+			}
+		}
+		return decl, false, true
+
+	case "ds4":
+		// ds4 carries the MTP heads in the weights and turns them on with
+		// mtp_path / mtp_draft. It has no dflash counterpart, so dflash is not a
+		// question that can be asked of a ds4 entry.
+		if feature != "mtp" {
+			return "", false, false
+		}
+		decl = "mtp_path:"
+		for _, o := range e.Overrides.Options {
+			o = strings.TrimSpace(o)
+			if strings.HasPrefix(o, "mtp_path:") || strings.HasPrefix(o, "mtp_draft:") {
+				return decl, true, true
+			}
+		}
+		return decl, false, true
+
+	default:
+		// sglang configures the feature with speculative_algorithm: in the
+		// referenced gallery/*.yaml, and an entry that declares no backend takes
+		// its whole configuration from its url: template. Verify reads one index
+		// file and follows neither, so it must not judge these in either
+		// direction.
+		return "", false, false
+	}
+}
+
+// UnaccountedQuants reports a wanted quant the repo demonstrably publishes but
+// that discovery produced no build for. The layout that triggers it today is
+// root-level shards, which match neither branch of DiscoverUnslothQuants; no
+// counterpart ships that way yet, but a batch generator must not drop a build
+// with nothing said about it.
+func UnaccountedQuants(files []GGUFFile, builds []QuantBuild) []string {
+	built := map[string]bool{}
+	for _, b := range builds {
+		built[b.Quant] = true
+	}
+
+	var problems []string
+	for _, q := range WantedQuants {
+		if built[q] {
+			continue
+		}
+		for _, f := range files {
+			if filePublishesQuant(f.Name, q) {
+				problems = append(problems, fmt.Sprintf("quant %s is published upstream (%s) but produced no build", q, f.Name))
+				break
+			}
+		}
+	}
+	return problems
+}
+
+// filePublishesQuant reports whether an upstream file is a publication of
+// quant q. It anchors on the quant label the way DiscoverUnslothQuants does,
+// as the trailing token of the base name or as the sharding subdirectory, so
+// the diagnostic and the discovery it audits cannot disagree about what a file
+// is.
+//
+// An unanchored match would reproduce the very collision this diagnostic warns
+// about: Q8_0 is a substring of UD-Q8_0, so a repo publishing only UD-Q8_0
+// would be reported as publishing an unbuilt Q8_0, which it does not, and
+// UD-Q8_0 is not a wanted quant at all.
+func filePublishesQuant(name, q string) bool {
+	if strings.HasPrefix(name, q+"/") {
+		return true
+	}
+
+	base := name[strings.LastIndex(name, "/")+1:]
+	// Shard numbering sits between the quant label and the extension, so it has
+	// to come off before the label can be read as the trailing token. Root-level
+	// shards are the layout that matches neither branch of
+	// DiscoverUnslothQuants, and so the layout this diagnostic mainly catches.
+	base = shardRE.ReplaceAllString(base, ".gguf")
+
+	if !strings.HasSuffix(base, "-"+q+".gguf") {
+		return false
+	}
+	// UD- is unsloth's dynamic-quant modifier, and UD-<q> is a distinct quant
+	// label rather than a publication of <q>.
+	return !strings.HasSuffix(base, "-UD-"+q+".gguf")
+}

.github/ci/apexentries/verify_test.go

@@ -0,0 +1,480 @@
+package main
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Verify", func() {
+	write := func(body string) string {
+		dir := GinkgoT().TempDir()
+		p := filepath.Join(dir, "index.yaml")
+		Expect(os.WriteFile(p, []byte(body), 0o600)).To(Succeed())
+		return p
+	}
+
+	It("passes a sound index", func() {
+		Expect(Verify(write(`
+- name: parent
+  variants:
+    - model: child
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: child
+  files:
+    - filename: b.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(BeEmpty())
+	})
+
+	It("reports a variant pointing at a missing entry", func() {
+		Expect(Verify(write(`
+- name: parent
+  variants:
+    - model: ghost
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("ghost")))
+	})
+
+	It("reports a variant that itself declares variants", func() {
+		Expect(Verify(write(`
+- name: parent
+  variants:
+    - model: child
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: child
+  variants:
+    - model: grandchild
+  files:
+    - filename: b.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+- name: grandchild
+  files:
+    - filename: c.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+`))).To(ContainElement(ContainSubstring("declares variants of its own")))
+	})
+
+	It("reports duplicate entry names", func() {
+		Expect(Verify(write(`
+- name: dup
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: dup
+  files:
+    - filename: b.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(ContainElement(ContainSubstring("duplicate entry name")))
+	})
+
+	It("reports a file with no sha256", func() {
+		Expect(Verify(write(`
+- name: one
+  files:
+    - filename: a.gguf
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("no sha256")))
+	})
+
+	It("reports an entry tagged dflash without a matching spec_type", func() {
+		Expect(Verify(write(`
+- name: liar
+  tags:
+    - dflash
+  overrides:
+    backend: llama-cpp
+    options:
+      - use_jinja:true
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("tagged dflash")))
+	})
+
+	It("reports an entry configuring spec_type without the tag", func() {
+		Expect(Verify(write(`
+- name: shy
+  overrides:
+    backend: llama-cpp
+    options:
+      - spec_type:draft-mtp
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("not tagged mtp")))
+	})
+
+	// ds4 carries the MTP heads in the weights and names them with mtp_path, so
+	// the rule holds there in a different vocabulary rather than not at all.
+	It("reports a ds4 entry configuring mtp_path without the tag", func() {
+		Expect(Verify(write(`
+- name: ds4-shy
+  overrides:
+    backend: ds4
+    options:
+      - mtp_path:model-mtp.gguf
+      - mtp_draft:2
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("not tagged mtp")))
+	})
+
+	It("reports a ds4 entry tagged mtp that configures no mtp_path", func() {
+		Expect(Verify(write(`
+- name: ds4-liar
+  tags:
+    - mtp
+  overrides:
+    backend: ds4
+    options:
+      - context_size:4096
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(ContainElement(ContainSubstring("tagged mtp")))
+	})
+
+	It("accepts a ds4 entry that both configures mtp_path and carries the tag", func() {
+		Expect(Verify(write(`
+- name: ds4-honest
+  tags:
+    - mtp
+  overrides:
+    backend: ds4
+    options:
+      - mtp_path:model-mtp.gguf
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(BeEmpty())
+	})
+
+	// sglang declares speculative_algorithm in the referenced gallery/*.yaml,
+	// which Verify never reads, so it may not judge such an entry either way.
+	It("says nothing about an sglang entry tagged mtp", func() {
+		Expect(Verify(write(`
+- name: sglang-mtp
+  tags:
+    - mtp
+  overrides:
+    backend: sglang
+  files: []
+`))).To(BeEmpty())
+	})
+
+	It("says nothing about the tag on an entry with no declared backend", func() {
+		Expect(Verify(write(`
+- name: templated
+  tags:
+    - mtp
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+`))).To(BeEmpty())
+	})
+
+	// The flat-match branch in unsloth.go appends every match, so a repo
+	// publishing both a plain and a UD Q8_0 renders one entry holding two full
+	// models while model: points at only the first.
+	It("reports an entry holding more than one non-shard weight file", func() {
+		Expect(Verify(write(`
+- name: greedy
+  overrides:
+    backend: llama-cpp
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/repo/Model-Q8_0.gguf
+  files:
+    - filename: llama-cpp/models/repo/Model-Q8_0.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: llama-cpp/models/repo/Model-UD-Q8_0.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(ContainElement(ContainSubstring("more than one weight file")))
+	})
+
+	It("accepts many shards alongside an mmproj and a drafter", func() {
+		Expect(Verify(write(`
+- name: sharded
+  tags:
+    - mtp
+  overrides:
+    backend: llama-cpp
+    options:
+      - spec_type:draft-mtp
+    mmproj: llama-cpp/mmproj/repo/mm.gguf
+    draft_model: llama-cpp/models/repo/Model-draft.gguf
+  files:
+    - filename: llama-cpp/models/repo/Model-00001-of-00002.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: llama-cpp/models/repo/Model-00002-of-00002.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+    - filename: llama-cpp/mmproj/repo/mm.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+    - filename: llama-cpp/models/repo/Model-draft.gguf
+      sha256: dd
+      uri: https://example.com/d.gguf
+`))).To(BeEmpty())
+	})
+
+	// Multi-component TTS and ASR engines legitimately ship an encoder, a
+	// tokenizer, a vocoder and so on as one model, so the collision the weight
+	// count catches does not exist for them.
+	It("accepts a multi-component non-llama-cpp entry declaring five weights", func() {
+		Expect(Verify(write(`
+- name: multi
+  overrides:
+    backend: qwen3-tts-cpp
+  files:
+    - filename: talker.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: tokenizer.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+    - filename: vocoder.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+    - filename: encoder.gguf
+      sha256: dd
+      uri: https://example.com/d.gguf
+    - filename: vae.gguf
+      sha256: ee
+      uri: https://example.com/e.gguf
+`))).To(BeEmpty())
+	})
+
+	It("says nothing about the weight count of an entry with no declared backend", func() {
+		Expect(Verify(write(`
+- name: templated-weights
+  files:
+    - filename: model-Q4_K_M.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: model-mmproj-f16.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+`))).To(BeEmpty())
+	})
+
+	It("says nothing about an auxiliary metadata file carrying no sha256", func() {
+		Expect(Verify(write(`
+- name: aux
+  files:
+    - filename: a.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+    - filename: params.json
+      sha256: ""
+      uri: https://example.com/params.json
+`))).To(BeEmpty())
+	})
+
+	// safetensors weights are downloaded and loaded exactly like GGUF weights,
+	// so an unverified one is the same supply-chain hole.
+	It("reports a safetensors weight carrying no sha256", func() {
+		Expect(Verify(write(`
+- name: vae
+  files:
+    - filename: wan_2.1_vae.safetensors
+      sha256: ""
+      uri: https://example.com/vae.safetensors
+`))).To(ContainElement(ContainSubstring("no sha256")))
+	})
+
+	It("says nothing about a txt or md file carrying no sha256", func() {
+		Expect(Verify(write(`
+- name: docs
+  files:
+    - filename: notes.txt
+      sha256: ""
+      uri: https://example.com/notes.txt
+    - filename: README.md
+      sha256: ""
+      uri: https://example.com/README.md
+`))).To(BeEmpty())
+	})
+})
+
+var _ = Describe("UnaccountedQuants", func() {
+	// A quant published only as root-level shards matches neither branch in
+	// DiscoverUnslothQuants, so without this diagnostic the build would vanish
+	// from a batch run with nothing said about it.
+	It("reports a wanted quant upstream publishes but discovery dropped", func() {
+		files := []GGUFFile{
+			{Name: "Model-UD-Q4_K_M-00001-of-00003.gguf", SHA256: "aa"},
+			{Name: "Model-UD-Q4_K_M-00002-of-00003.gguf", SHA256: "bb"},
+			{Name: "Model-UD-Q4_K_M-00003-of-00003.gguf", SHA256: "cc"},
+		}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).
+			To(ContainElement(ContainSubstring("UD-Q4_K_M")))
+	})
+
+	It("says nothing when every published wanted quant produced a build", func() {
+		files := []GGUFFile{
+			{Name: "Model-UD-Q4_K_M.gguf", SHA256: "aa"},
+			{Name: "UD-Q6_K/Model-UD-Q6_K-00001-of-00002.gguf", SHA256: "bb"},
+			{Name: "UD-Q6_K/Model-UD-Q6_K-00002-of-00002.gguf", SHA256: "cc"},
+		}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
+	})
+
+	It("says nothing about a wanted quant the repo does not publish at all", func() {
+		files := []GGUFFile{{Name: "Model-UD-Q4_K_M.gguf", SHA256: "aa"}}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
+	})
+
+	// UD-Q8_0 is its own quant label and is not a wanted one. Reading it as a
+	// publication of Q8_0 is the substring collision this diagnostic exists to
+	// warn about, and subdirectory-sharded UD quants are the normal unsloth
+	// layout for large repos, so the false positive would fire on every batch.
+	It("does not read a subdirectory-sharded UD-Q8_0 as a published Q8_0", func() {
+		files := []GGUFFile{
+			{Name: "UD-Q8_0/Model-UD-Q8_0-00001-of-00002.gguf", SHA256: "aa"},
+			{Name: "UD-Q8_0/Model-UD-Q8_0-00002-of-00002.gguf", SHA256: "bb"},
+		}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
+	})
+
+	// A quant in its own subdirectory but not shard-numbered matches neither
+	// branch of DiscoverUnslothQuants, so it is genuinely published and
+	// genuinely undiscovered.
+	It("reports a wanted quant published in its own subdirectory without shard numbering", func() {
+		files := []GGUFFile{{Name: "Q8_0/Model-Q8_0.gguf", SHA256: "aa"}}
+
+		Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).
+			To(ContainElement(ContainSubstring("quant Q8_0 is published upstream")))
+	})
+
+	// builds is empty on purpose: it isolates the file-to-quant match from
+	// whatever DiscoverUnslothQuants would have made of the same file.
+	It("matches the flat single-file layout", func() {
+		files := []GGUFFile{{Name: "Model-Q8_0.gguf", SHA256: "aa"}}
+
+		Expect(UnaccountedQuants(files, nil)).
+			To(ConsistOf(ContainSubstring("quant Q8_0 is published upstream")))
+	})
+})
+
+var _ = Describe("Verify local path collisions", func() {
+	write := func(body string) string {
+		dir := GinkgoT().TempDir()
+		p := filepath.Join(dir, "index.yaml")
+		Expect(os.WriteFile(p, []byte(body), 0o600)).To(Succeed())
+		return p
+	}
+
+	It("reports one local path claimed by two different uris", func() {
+		// The shape that shipped: LiquidAI and unsloth both publish
+		// LFM2.5-8B-A1B-GGUF, so a path built from the bare repo name gives both
+		// entries the same local file under two different checksums.
+		Expect(Verify(write(`
+- name: lfm2.5-8b-a1b
+  files:
+    - filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
+      sha256: 33ab3b8c
+      uri: https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q8_0.gguf
+- name: lfm2.5-8b-a1b-q8-0
+  files:
+    - filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
+      sha256: ec11666b
+      uri: https://huggingface.co/unsloth/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q8_0.gguf
+`))).To(ContainElement(SatisfyAll(
+			ContainSubstring("claimed by two different uris"),
+			ContainSubstring("lfm2.5-8b-a1b-q8-0"),
+		)))
+	})
+
+	It("accepts two entries reusing one file from the same uri", func() {
+		// Sibling builds of one repo legitimately share a projector.
+		Expect(Verify(write(`
+- name: a
+  files:
+    - filename: llama-cpp/mmproj/mudler/Example-GGUF/mmproj-F16.gguf
+      sha256: cc
+      uri: https://huggingface.co/mudler/Example-GGUF/resolve/main/mmproj-F16.gguf
+- name: b
+  files:
+    - filename: llama-cpp/mmproj/mudler/Example-GGUF/mmproj-F16.gguf
+      sha256: cc
+      uri: https://huggingface.co/mudler/Example-GGUF/resolve/main/mmproj-F16.gguf
+`))).To(BeEmpty())
+	})
+
+	It("reports a collision once however many entries pile onto the path", func() {
+		problems := Verify(write(`
+- name: a
+  files:
+    - filename: shared.gguf
+      sha256: aa
+      uri: https://example.com/a.gguf
+- name: b
+  files:
+    - filename: shared.gguf
+      sha256: bb
+      uri: https://example.com/b.gguf
+- name: c
+  files:
+    - filename: shared.gguf
+      sha256: cc
+      uri: https://example.com/c.gguf
+`))
+
+		var collisions int
+		for _, p := range problems {
+			if strings.Contains(p, "claimed by two different uris") {
+				collisions++
+			}
+		}
+		Expect(collisions).To(Equal(1))
+	})
+
+	It("says nothing about files that carry no uri", func() {
+		// A hand-written entry may record only a checksum. There is no upstream
+		// to compare, so the check cannot conclude anything either way.
+		Expect(Verify(write(`
+- name: a
+  files:
+    - filename: shared.gguf
+      sha256: aa
+- name: b
+  files:
+    - filename: shared.gguf
+      sha256: bb
+`))).To(BeEmpty())
+	})
+})

.github/ci/galleryedit/edit.go

@@ -0,0 +1,152 @@
+// Package galleryedit splices variant references into the LocalAI gallery index
+// as TEXT.
+//
+// Re-serialising the index through a YAML marshaller would reflow 40,000 lines,
+// drop the anchors and merge keys the gallery relies on, and produce a diff no
+// reviewer could read, which makes a pull request worthless even when the
+// content inside it is right. Every generator that adds variants to an entry the
+// gallery already ships therefore edits lines, and they share this package so
+// that two of them cannot drift apart on where a variants block belongs.
+package galleryedit
+
+import (
+	"fmt"
+	"regexp"
+	"sort"
+	"strings"
+)
+
+var (
+	entryStart   = regexp.MustCompile(`^-(?: |$)`)
+	inlineName   = regexp.MustCompile(`^- (?:&\S+ )?name:`)
+	keyName      = regexp.MustCompile(`^  name:`)
+	keyVariants  = regexp.MustCompile(`^  variants:\s*(.*)$`)
+	variantItem  = regexp.MustCompile(`^    - `)
+	unsafeInName = regexp.MustCompile(`[:#{}\[\],&*?|>'"%@` + "`" + `]|^\s|\s$`)
+)
+
+// Entry is the positional view of one gallery entry: what it is called and
+// which lines it occupies. Nothing about what the entry MEANS belongs here, so
+// each caller keeps its own semantic decode and only hands over the coordinates.
+type Entry struct {
+	Name string
+	// StartLine and EndLine bound the entry, zero based and half open.
+	StartLine int
+	EndLine   int
+}
+
+// Insert is one entry's pending variants addition. The caller owns the contents
+// of Variants: this package neither orders nor deduplicates them, because the
+// right order and the right dedup rule differ between generators.
+type Insert struct {
+	Entry    Entry
+	Variants []string
+}
+
+// Scan splits index text into lines and reports the line each top level list
+// item begins on.
+func Scan(text string) (lines []string, starts []int) {
+	lines = strings.Split(text, "\n")
+	for i, line := range lines {
+		if entryStart.MatchString(line) {
+			starts = append(starts, i)
+		}
+	}
+	return lines, starts
+}
+
+// Apply splices every insert into the index lines and returns the new text.
+func Apply(lines []string, inserts []Insert) ([]string, error) {
+	type edit struct {
+		at     int
+		remove int
+		insert []string
+	}
+	var edits []edit
+
+	for _, in := range inserts {
+		if len(in.Variants) == 0 {
+			continue
+		}
+
+		items := make([]string, 0, len(in.Variants))
+		for _, v := range in.Variants {
+			items = append(items, "    - model: "+QuoteName(v))
+		}
+
+		at, remove, err := insertionPoint(lines, in.Entry)
+		if err != nil {
+			return nil, err
+		}
+		block := items
+		if remove > 0 || !hasVariantsKey(lines, in.Entry) {
+			block = append([]string{"  variants:"}, items...)
+		}
+		edits = append(edits, edit{at: at, remove: remove, insert: block})
+	}
+
+	// Applying from the bottom up keeps every line number computed against the
+	// original text valid while earlier edits are still pending.
+	sort.Slice(edits, func(i, j int) bool { return edits[i].at > edits[j].at })
+
+	out := append([]string(nil), lines...)
+	for _, e := range edits {
+		tail := append([]string(nil), out[e.at+e.remove:]...)
+		out = append(out[:e.at], append(append([]string(nil), e.insert...), tail...)...)
+	}
+	return out, nil
+}
+
+func hasVariantsKey(lines []string, e Entry) bool {
+	for i := e.StartLine; i < e.EndLine; i++ {
+		if keyVariants.MatchString(lines[i]) {
+			return true
+		}
+	}
+	return false
+}
+
+// insertionPoint reports where new variant items belong, and how many existing
+// lines the insertion replaces.
+//
+// An entry with no variants key gets one right after its name, which is where
+// the hand-written families put it. An entry with an empty "variants: []" has
+// that line replaced by a block. An entry with a block gets its items appended.
+func insertionPoint(lines []string, e Entry) (at int, remove int, err error) {
+	for i := e.StartLine; i < e.EndLine; i++ {
+		m := keyVariants.FindStringSubmatch(lines[i])
+		if m == nil {
+			continue
+		}
+		if strings.TrimSpace(m[1]) == "[]" {
+			return i, 1, nil
+		}
+		if strings.TrimSpace(m[1]) != "" {
+			return 0, 0, fmt.Errorf("entry %q writes its variants inline (%q); this job only edits block lists", e.Name, strings.TrimSpace(m[1]))
+		}
+		last := i
+		for j := i + 1; j < e.EndLine && variantItem.MatchString(lines[j]); j++ {
+			last = j
+		}
+		return last + 1, 0, nil
+	}
+
+	if inlineName.MatchString(lines[e.StartLine]) {
+		return e.StartLine + 1, 0, nil
+	}
+	for i := e.StartLine; i < e.EndLine; i++ {
+		if keyName.MatchString(lines[i]) {
+			return i + 1, 0, nil
+		}
+	}
+	return 0, 0, fmt.Errorf("entry %q has no name line to anchor the insertion to", e.Name)
+}
+
+// QuoteName quotes a variant reference when the name would otherwise change
+// meaning as bare YAML. Config-suffixed names carry a ":" and always need it.
+func QuoteName(name string) string {
+	if unsafeInName.MatchString(name) {
+		return `"` + strings.ReplaceAll(name, `"`, `\"`) + `"`
+	}
+	return name
+}

.github/ci/gen-redirects.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+#
+# Generate client-side redirects for the documentation URLs that used to live at
+# the site root.
+#
+# Until this site existed, the Hugo docs site WAS localai.io, so pages
+# were published at /features/..., /getting-started/..., /faq/ and so on. The
+# docs now build under /docs/, and GitHub Pages serves static files only: there
+# is no server-side rewrite, no .htaccess, no _redirects. The only way to keep
+# every published, bookmarked and search-indexed URL alive is to leave a real
+# HTML file at the old address that sends the browser to the new one.
+#
+# Anything the main site already publishes wins: it owns /, /engines/,
+# /blog/ and friends, so an existing file is never replaced.
+#
+# Usage: gen-redirects.sh <public-dir> [base-url]
+#   public-dir  merged output directory (main site with docs/ inside it)
+#   base-url    absolute or root-relative prefix the deployment is served from,
+#               trailing slash optional (default "/")
+
+set -euo pipefail
+
+PUBLIC_DIR=${1:?usage: gen-redirects.sh <public-dir> [base-url]}
+BASE_URL=${2:-/}
+
+# Normalise to exactly one trailing slash so concatenation below is predictable.
+BASE_URL="${BASE_URL%/}/"
+
+DOCS_DIR="${PUBLIC_DIR}/docs"
+
+if [ ! -d "$DOCS_DIR" ]; then
+  echo "gen-redirects: no docs output at ${DOCS_DIR}" >&2
+  exit 1
+fi
+
+created=0
+skipped=0
+
+# Every .html file is a reachable old URL, not just directory indexes: the
+# generated model gallery ships as a bare gallery.html and used to sit at the
+# root too.
+while IFS= read -r src; do
+  rel=${src#"$DOCS_DIR"/}
+  dst="${PUBLIC_DIR}/${rel}"
+
+  if [ -e "$dst" ]; then
+    skipped=$((skipped + 1))
+    continue
+  fi
+
+  # Link to the directory, not to its index.html, so the redirect target is the
+  # canonical URL the docs site itself advertises.
+  target="${BASE_URL}docs/${rel%index.html}"
+
+  mkdir -p "$(dirname "$dst")"
+  printf '%s' '<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>Moved</title>
+<link rel="canonical" href="'"$target"'">
+<meta name="robots" content="noindex">
+<meta http-equiv="refresh" content="0; url='"$target"'">
+</head>
+<body>
+<p>This page moved to <a href="'"$target"'">'"$target"'</a>.</p>
+</body>
+</html>
+' > "$dst"
+
+  created=$((created + 1))
+done <<EOF
+$(find "$DOCS_DIR" -type f -name '*.html' | sort)
+EOF
+
+echo "gen-redirects: ${created} redirect(s) written, ${skipped} path(s) left to the main site"

.github/ci/variantproposals/edit.go

@@ -2,119 +2,41 @@ package main
 
 import (
 	"fmt"
-	"regexp"
-	"sort"
 	"strings"
-)
 
-var (
-	inlineName   = regexp.MustCompile(`^- (?:&\S+ )?name:`)
-	keyName      = regexp.MustCompile(`^  name:`)
-	keyVariants  = regexp.MustCompile(`^  variants:\s*(.*)$`)
-	variantItem  = regexp.MustCompile(`^    - `)
-	unsafeInName = regexp.MustCompile(`[:#{}\[\],&*?|>'"%@` + "`" + `]|^\s|\s$`)
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
 )
 
 // ApplyFamilies writes the proposed variant lists into the index text.
 //
-// The edit is textual on purpose. Re-serialising the index through a YAML
-// marshaller would reflow 40,000 lines, drop the anchors and merge keys the
-// gallery relies on, and produce a diff no reviewer could read, which would
-// make the pull request worthless even when the proposals inside it are right.
+// The line editing itself lives in galleryedit, shared with the apexentries
+// generator. Both jobs add variants to entries the gallery already ships, and a
+// second answer to "where does a variants block go" would drift from this one;
+// see that package for why the edit is textual rather than a YAML round trip.
 func ApplyFamilies(ix *Index, families []Family) ([]string, error) {
 	byName, _ := ix.ByName()
 
-	type edit struct {
-		at      int
-		remove  int
-		insert  []string
-		ordinal int
-	}
-	var edits []edit
-
+	var inserts []galleryedit.Insert
 	for _, f := range families {
 		entry, ok := byName[strings.ToLower(f.Parent)]
 		if !ok {
 			return nil, fmt.Errorf("parent %q is not in the index", f.Parent)
 		}
-		items := make([]string, 0, len(f.Proposals))
+
+		variants := make([]string, 0, len(f.Proposals))
 		for _, p := range f.Proposals {
-			items = append(items, "    - model: "+quoteName(p.Variant))
+			variants = append(variants, p.Variant)
 		}
 
-		at, remove, err := insertionPoint(ix, entry)
-		if err != nil {
-			return nil, err
-		}
-		insert := items
-		if remove > 0 || !hasVariantsKey(ix, entry) {
-			insert = append([]string{"  variants:"}, items...)
-		}
-		edits = append(edits, edit{at: at, remove: remove, insert: insert, ordinal: entry.Index})
+		inserts = append(inserts, galleryedit.Insert{
+			Entry: galleryedit.Entry{
+				Name:      entry.Name,
+				StartLine: entry.StartLine,
+				EndLine:   entry.EndLine,
+			},
+			Variants: variants,
+		})
 	}
 
-	// Applying from the bottom up keeps every line number computed against the
-	// original text valid while earlier edits are still pending.
-	sort.Slice(edits, func(i, j int) bool { return edits[i].at > edits[j].at })
-
-	lines := append([]string(nil), ix.Lines...)
-	for _, e := range edits {
-		tail := append([]string(nil), lines[e.at+e.remove:]...)
-		lines = append(lines[:e.at], append(append([]string(nil), e.insert...), tail...)...)
-	}
-	return lines, nil
-}
-
-func hasVariantsKey(ix *Index, e *GalleryEntry) bool {
-	for i := e.StartLine; i < e.EndLine; i++ {
-		if keyVariants.MatchString(ix.Lines[i]) {
-			return true
-		}
-	}
-	return false
-}
-
-// insertionPoint reports where new variant items belong, and how many existing
-// lines the insertion replaces.
-//
-// An entry with no variants key gets one right after its name, which is where
-// the hand-written families put it. An entry with an empty "variants: []" has
-// that line replaced by a block. An entry with a block gets its items appended.
-func insertionPoint(ix *Index, e *GalleryEntry) (at int, remove int, err error) {
-	for i := e.StartLine; i < e.EndLine; i++ {
-		m := keyVariants.FindStringSubmatch(ix.Lines[i])
-		if m == nil {
-			continue
-		}
-		if strings.TrimSpace(m[1]) == "[]" {
-			return i, 1, nil
-		}
-		if strings.TrimSpace(m[1]) != "" {
-			return 0, 0, fmt.Errorf("entry %q writes its variants inline (%q); this job only edits block lists", e.Name, strings.TrimSpace(m[1]))
-		}
-		last := i
-		for j := i + 1; j < e.EndLine && variantItem.MatchString(ix.Lines[j]); j++ {
-			last = j
-		}
-		return last + 1, 0, nil
-	}
-
-	if inlineName.MatchString(ix.Lines[e.StartLine]) {
-		return e.StartLine + 1, 0, nil
-	}
-	for i := e.StartLine; i < e.EndLine; i++ {
-		if keyName.MatchString(ix.Lines[i]) {
-			return i + 1, 0, nil
-		}
-	}
-	return 0, 0, fmt.Errorf("entry %q has no name line to anchor the insertion to", e.Name)
-}
-
-// quoteName quotes a variant reference when the name would otherwise change
-// meaning as bare YAML. Config-suffixed names carry a ":" and always need it.
-func quoteName(name string) string {
-	if unsafeInName.MatchString(name) {
-		return `"` + strings.ReplaceAll(name, `"`, `\"`) + `"`
-	}
-	return name
+	return galleryedit.Apply(ix.Lines, inserts)
 }

.github/ci/variantproposals/index.go

@@ -8,6 +8,8 @@ import (
 	"strings"
 
 	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/.github/ci/galleryedit"
 )
 
 // File is the subset of a gallery file entry the proposer reads.
@@ -57,7 +59,6 @@ type Index struct {
 }
 
 var (
-	entryStart  = regexp.MustCompile(`^-(?: |$)`)
 	anchorStart = regexp.MustCompile(`^- &(\S+)`)
 	mergeStart  = regexp.MustCompile(`^- !!merge <<: \*(\S+)`)
 )
@@ -82,13 +83,7 @@ func ParseIndex(text string) (*Index, error) {
 		return nil, fmt.Errorf("decoding gallery index: %w", err)
 	}
 
-	lines := strings.Split(text, "\n")
-	var starts []int
-	for i, line := range lines {
-		if entryStart.MatchString(line) {
-			starts = append(starts, i)
-		}
-	}
+	lines, starts := galleryedit.Scan(text)
 	if len(starts) != len(entries) {
 		return nil, fmt.Errorf("gallery index has %d decoded entries but %d top level list items; refusing to edit by line number", len(entries), len(starts))
 	}

.github/gh_curl.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+# Shared curl wrapper for the nightly dependency-bump scripts.
+#
+# The bump workflow fans out to ~25 parallel matrix jobs, each querying
+# api.github.com. Anonymous API calls are capped at 60/hour per source IP and
+# GitHub-hosted runners egress through shared NAT addresses, so a random handful
+# of jobs were getting rate-limited (HTTP 403 -> curl exit 22, empty response)
+# every single night. Authenticating with GITHUB_TOKEN lifts the ceiling to
+# 1000/hour; the retries absorb whatever transient blips remain.
+
+# Wraps curl with GitHub auth (when a token is present) plus retry/timeout
+# hardening. Callers pass their own headers and the URL.
+gh_curl() {
+    # The bump scripts run under `set -x`; without this the Authorization header
+    # would be echoed into the job log on every call.
+    local had_xtrace=0
+    case "$-" in
+        *x*) had_xtrace=1; set +x ;;
+    esac
+
+    local args=(
+        --silent --show-error --location --fail
+        # --retry-all-errors so 403 rate-limit responses are retried too; plain
+        # --retry only covers 408/429/5xx. curl honours Retry-After when sent.
+        --retry 5 --retry-delay 3 --retry-all-errors
+        --connect-timeout 15 --max-time 60
+    )
+    if [ -n "${GITHUB_TOKEN:-}" ]; then
+        args+=(--header "Authorization: Bearer ${GITHUB_TOKEN}")
+    fi
+
+    curl "${args[@]}" "$@"
+    local rc=$?
+
+    if [ "$had_xtrace" -eq 1 ]; then
+        set -x
+    fi
+    return $rc
+}

.github/workflows/backend_build_darwin.yml

@@ -244,8 +244,19 @@ jobs:
           make protogen-go
           make backends/privacy-filter-darwin
 
+      # audio-cpp is a C++/ggml backend like ds4 and privacy-filter - a single
+      # grpc-server with otool dylib bundling, plus bundled VAD assets - so it
+      # gets its own bespoke darwin script rather than the generic
+      # build-darwin-go-backend path, which would look for a backend/go/audio-cpp
+      # that does not exist. Keep this set in sync with DARWIN_BESPOKE_BUILDERS
+      # in scripts/lib/backend-filter.mjs.
+      - name: Build audio-cpp backend (Darwin Metal)
+        if: inputs.backend == 'audio-cpp'
+        run: |
+          make backends/audio-cpp-darwin
+
       - name: Build ${{ inputs.backend }}-darwin
-        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4' && inputs.backend != 'privacy-filter'
+        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4' && inputs.backend != 'privacy-filter' && inputs.backend != 'audio-cpp'
         run: |
           make protogen-go
           BACKEND=${{ inputs.backend }} BUILD_TYPE=${{ inputs.build-type }} USE_PIP=${{ inputs.use-pip }} make build-darwin-${{ inputs.lang }}-backend

.github/workflows/build-test.yaml

@@ -5,6 +5,32 @@ on:
     branches:
       - master
   pull_request:
+    # GoReleaser and the darwin launcher take no gallery, docs or markdown
+    # input, so a diff confined to these paths cannot change either binary.
+    # The darwin job matters here: macOS is the scarcest runner class.
+    #
+    # backend/{cpp,go,python}/**: GoReleaser builds ./cmd/local-ai and the
+    # launcher builds ./cmd/launcher; neither compiles a backend. The
+    # before-hooks still matter, but their input is backend/backend.proto
+    # (protogen-go) and go.mod/go.sum (go mod tidy), none of which live under
+    # these prefixes, so a change to any of those still triggers a full run.
+    # See .agents/ci-caching.md.
+    paths-ignore:
+      - 'gallery/**'
+      - 'docs/**'
+      - 'examples/**'
+      - '**/*.md'
+      - 'backend/cpp/**'
+      - 'backend/go/**'
+      - 'backend/python/**'
+
+# Supersede an in-flight run when a PR gets a new push. Keyed on the PR number
+# so every push to the same PR shares a group; on a master push the key falls
+# back to github.sha (unique per commit) and cancel-in-progress is false, so
+# master runs never cancel each other -- each commit is built on its own.
+concurrency:
+  group: ci-build-test-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
   build-test:
@@ -18,9 +44,15 @@ jobs:
         uses: actions/setup-go@v5
         with:
           go-version: 1.25
+      # A PR builds only the host target. The three-platform cross-compile
+      # (linux/amd64, linux/arm64, darwin/arm64) is the bulk of this job's
+      # ~6.6min median and no PR consumes the resulting binaries. The
+      # before-hooks (protogen-go, react-ui, go mod tidy) run either way, so the
+      # "is the release build broken" signal is unchanged. master and tags still
+      # build everything.
       - name: Run GoReleaser
         run: |
-          make dev-dist
+          make ${{ github.event_name == 'pull_request' && 'dev-dist-single' || 'dev-dist' }}
   launcher-build-darwin:
     runs-on: macos-latest
     steps:

.github/workflows/bump_deps.yaml

@@ -30,6 +30,10 @@ jobs:
             variable: "DS4_VERSION"
             branch: "main"
             file: "backend/cpp/ds4/Makefile"
+          - repository: "0xShug0/audio.cpp"
+            variable: "AUDIO_CPP_VERSION"
+            branch: "main"
+            file: "backend/cpp/audio-cpp/Makefile"
           - repository: "meituan-longcat/LongCat-Video"
             variable: "LONGCAT_VIDEO_VERSION"
             branch: "main"
@@ -50,6 +54,10 @@ jobs:
             variable: "PARAKEET_VERSION"
             branch: "master"
             file: "backend/go/parakeet-cpp/Makefile"
+          - repository: "mudler/vllm.cpp"
+            variable: "VLLM_CPP_VERSION"
+            branch: "main"
+            file: "backend/go/vllm-cpp/Makefile"
           - repository: "localai-org/moss-transcribe.cpp"
             variable: "MOSS_VERSION"
             branch: "master"
@@ -74,6 +82,10 @@ jobs:
             variable: "STABLEDIFFUSION_GGML_VERSION"
             branch: "master"
             file: "backend/go/stablediffusion-ggml/Makefile"
+          - repository: "localai-org/trellis2cpp"
+            variable: "TRELLIS2CPP_VERSION"
+            branch: "pbr-textures"
+            file: "backend/go/trellis2cpp/Makefile"
           - repository: "mudler/go-piper"
             variable: "PIPER_VERSION"
             branch: "master"
@@ -110,11 +122,21 @@ jobs:
             variable: "VIBEVOICE_CPP_VERSION"
             branch: "master"
             file: "backend/go/vibevoice-cpp/Makefile"
+          - repository: "mudler/magpie-tts.cpp"
+            variable: "MAGPIETTS_CPP_VERSION"
+            branch: "main"
+            file: "backend/go/magpie-tts-cpp/Makefile"
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v7
       - name: Bump dependencies 🔧
         id: bump
+        env:
+          # This job fans out to ~25 parallel matrix entries, all querying
+          # api.github.com from runner IPs that share the 60/hour anonymous
+          # rate limit. Authenticating raises it to 1000/hour, which is what
+          # kept a random handful of these red every night.
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_deps.sh ${{ matrix.repository }} ${{ matrix.branch }} ${{ matrix.variable }} ${{ matrix.file }}
           {
@@ -151,6 +173,8 @@ jobs:
       - uses: actions/checkout@v7
       - name: Bump vLLM cu130 wheel pin 🔧
         id: bump
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_vllm_wheel.sh vllm-project/vllm backend/python/vllm/requirements-cublas13-after.txt VLLM_VERSION
           {
@@ -187,6 +211,8 @@ jobs:
       - uses: actions/checkout@v7
       - name: Bump vllm-metal pin 🔧
         id: bump
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_vllm_metal.sh vllm-project/vllm-metal backend/python/vllm/install.sh VLLM_METAL_VERSION
           {

.github/workflows/bump_docs.yaml

@@ -15,6 +15,10 @@ jobs:
     steps:
       - uses: actions/checkout@v7
       - name: Bump dependencies 🔧
+        env:
+          # Authenticated API calls get 1000 req/hour instead of the 60/hour
+          # anonymous cap that is shared across every job on the runner's IP.
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           bash .github/bump_docs.sh ${{ matrix.repository }}
       - name: Create Pull Request

.github/workflows/ci-tools-tests.yaml

@@ -0,0 +1,39 @@
+---
+# The packages under .github/ci/ are invisible to `go list ./...`, so neither
+# `make lint` nor the repository test run ever touches them. Their specs are
+# dead weight until a workflow names each package explicitly.
+name: 'CI tool tests'
+on:
+  pull_request:
+    paths:
+      - '.github/ci/**'
+      - '.github/workflows/ci-tools-tests.yaml'
+  push:
+    branches:
+      - master
+    paths:
+      - '.github/ci/**'
+jobs:
+  ci-tools:
+    name: 'Test the .github/ci generators'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-go@v5
+        with:
+          go-version-file: go.mod
+          cache: false
+
+      # The discovery heuristics are the risky part of these tools. A regression
+      # produces confident, wrong gallery entries, which is worse than no tool.
+      - name: 'Test the APEX entry generator'
+        run: go test ./.github/ci/apexentries/
+
+      - name: 'Test the variant proposer'
+        run: go test ./.github/ci/variantproposals/
+
+      # Shared by both generators above. Its behaviour is exercised through their
+      # specs; this step exists so a break in the shared package fails under its
+      # own name rather than as a puzzling failure in whichever caller ran first.
+      - name: 'Test the shared gallery editor'
+        run: go test ./.github/ci/galleryedit/

.github/workflows/gh-pages.yml

@@ -1,4 +1,4 @@
-name: Deploy docs to GitHub Pages
+name: Deploy site to GitHub Pages
 
 on:
   push:
@@ -6,9 +6,11 @@ on:
       - master
     paths:
       - 'docs/**'
+      - 'website/**'
       - 'gallery/**'
       - 'images/**'
       - '.github/ci/modelslist.go'
+      - '.github/ci/gen-redirects.sh'
       - '.github/workflows/gh-pages.yml'
   workflow_dispatch:
 
@@ -23,7 +25,20 @@ concurrency:
 
 jobs:
   build:
-    runs-on: ubuntu-latest
+    # Self-hosted. This workflow is push-to-master + workflow_dispatch only, so
+    # it never executes pull-request code and a fork cannot reach the runner
+    # with untrusted changes. The repository guard keeps forks (whose own master
+    # pushes would otherwise queue forever against a label they do not have) on
+    # the hosted pool.
+    #
+    # Why: the GitHub-hosted pool is shared account-wide and has repeatedly
+    # starved (2026-07-31: 35 consecutive minutes at zero scheduled jobs, while
+    # arc-runner-set kept completing work throughout). Publishing the site is
+    # small, frequent, and must not sit behind a saturated hosted queue.
+    #
+    # Needs only git, tar and curl on the runner: setup-go and actions-hugo
+    # fetch their own toolchains, and no step uses sudo, apt, make or unzip.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     env:
       HUGO_VERSION: "0.146.3"
     steps:
@@ -49,25 +64,46 @@ jobs:
         id: pages
         uses: actions/configure-pages@v6
 
+      # The gallery page is generated from the model index and shipped as a
+      # static asset of the docs site, so it has to exist before Hugo runs.
       - name: Generate gallery
         run: go run ./.github/ci/modelslist.go ./gallery/index.yaml > docs/static/gallery.html
 
-      - name: Build site
+      # Two Hugo sites, one Pages artifact: the main site owns the root,
+      # the docs site is nested under /docs/.
+      - name: Build the main site
+        working-directory: website
+        run: hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+
+      - name: Build documentation site
         working-directory: docs
         run: |
           mkdir -p layouts/_default
-          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/docs/"
+
+      - name: Merge documentation into the main site
+        run: |
+          mkdir -p website/public/docs
+          cp -R docs/public/. website/public/docs/
+
+      # Keeps the pre-split URLs alive; see the script header.
+      - name: Generate legacy URL redirects
+        run: .github/ci/gen-redirects.sh website/public "${{ steps.pages.outputs.base_url }}/"
 
       - name: Upload artifact
         uses: actions/upload-pages-artifact@v5
         with:
-          path: docs/public
+          path: website/public
 
   deploy:
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}
-    runs-on: ubuntu-latest
+    # Same routing as build: a hosted slot for a ~10s deploy is exactly the kind
+    # of job that should not block on a starved pool. deploy-pages authenticates
+    # with the job's OIDC token (id-token: write above), which self-hosted
+    # runners issue the same way hosted ones do.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     needs: build
     steps:
       - name: Deploy to GitHub Pages

.github/workflows/image-pr.yml

@@ -3,7 +3,28 @@
   
   on:
     pull_request:
-  
+      # None of these seven image builds can observe a diff confined to these
+      # paths. Gallery metadata is parsed at runtime and never copied into an
+      # image; docs and markdown never enter one at all. Gallery content is
+      # still checked by yaml-check.yml and by
+      # core/gallery/variants_lint_test.go under 'tests'.
+      #
+      # backend/{cpp,go,python}/**: this workflow builds the core image, whose
+      # only compiled output is `make build` -> `go build ./cmd/local-ai`. The
+      # per-backend trees are copied into the builder but nothing in them reaches
+      # the binary or the final stage. backend/backend.proto is deliberately not
+      # listed: it feeds protogen-go and so does change the binary, and it does
+      # not live under any of these prefixes, so it still triggers a full run.
+      # See .agents/ci-caching.md.
+      paths-ignore:
+        - 'gallery/**'
+        - 'docs/**'
+        - 'examples/**'
+        - '**/*.md'
+        - 'backend/cpp/**'
+        - 'backend/go/**'
+        - 'backend/python/**'
+
   concurrency:
     group: ci-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
     cancel-in-progress: ${{ github.event_name == 'pull_request' }}
@@ -52,7 +73,7 @@
               tag-latest: 'false'
               tag-suffix: '-gpu-nvidia-cuda-13'
               runs-on: 'ubuntu-latest'
-              base-image: "ubuntu:22.04"
+              base-image: "ubuntu:24.04"
               makeflags: "--jobs=3 --output-sync=target"
               ubuntu-version: '2404'
             - build-type: 'hipblas'

.github/workflows/image.yml

@@ -13,8 +13,53 @@
     cancel-in-progress: ${{ github.event_name == 'pull_request' }}
   
   jobs:
-    hipblas-jobs:
+    # Decide once whether this push can change any image. Gallery metadata is
+    # fetched at runtime and never baked into an image, and docs/markdown never
+    # enter one, so a push confined to those paths produces byte-identical
+    # images. On 2026-07-30, 12 of the 23 queued runs of this workflow were
+    # commits like "add 1 new model to gallery" or a docs fix, each rebuilding
+    # all 18 images.
+    #
+    # A job-level gate rather than `paths-ignore` on the trigger: paths-ignore
+    # would also apply to tag pushes, and a tag created on an existing commit
+    # carries an empty commits list, which would silently skip the release image
+    # build. Tags short-circuit to "build" below, as does a push whose base
+    # commit cannot be resolved -- the same run-everything posture the backend
+    # matrix filter takes for a truncated diff.
+    changes:
       if: github.repository == 'mudler/LocalAI'
+      runs-on: ubuntu-latest
+      outputs:
+        build: ${{ steps.decide.outputs.build }}
+      steps:
+        - uses: actions/checkout@v7
+          with:
+            fetch-depth: 0
+        - id: decide
+          env:
+            BEFORE: ${{ github.event.before }}
+            AFTER: ${{ github.sha }}
+          run: |
+            set -euo pipefail
+            emit() { echo "$2"; echo "build=$1" >> "$GITHUB_OUTPUT"; exit 0; }
+            case "${GITHUB_REF}" in
+              refs/tags/*) emit true "tag push: building every image" ;;
+            esac
+            if [ -z "${BEFORE:-}" ] || [ "${BEFORE}" = "0000000000000000000000000000000000000000" ] \
+               || ! git cat-file -e "${BEFORE}^{commit}" 2>/dev/null; then
+              emit true "no resolvable base commit: building every image"
+            fi
+            files="$(git diff --name-only "${BEFORE}" "${AFTER}")"
+            echo "changed files:"; echo "${files:-<none>}"
+            [ -z "${files}" ] && emit true "empty diff: building every image"
+            if echo "${files}" | grep -qvE '^(gallery/|docs/|examples/)|\.md$'; then
+              emit true "push touches image-visible content: building"
+            fi
+            emit false "only gallery/docs/markdown changed: images identical, skipping"
+
+    hipblas-jobs:
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}
@@ -47,7 +92,8 @@
               ubuntu-codename: 'noble'
 
     core-image-build:
-      if: github.repository == 'mudler/LocalAI'
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}
@@ -113,7 +159,7 @@
               tag-latest: 'auto'
               tag-suffix: '-gpu-nvidia-cuda-13'
               runs-on: 'ubuntu-latest'
-              base-image: "ubuntu:22.04"
+              base-image: "ubuntu:24.04"
               skip-drivers: 'false'
               makeflags: "--jobs=4 --output-sync=target"
               ubuntu-version: '2404'
@@ -155,8 +201,8 @@
       # merge whenever any matrix cell of the parent build fails or is
       # cancelled. Same fix as backend.yml's merge jobs — we still want to
       # publish the manifest list for tag-suffixes whose legs all succeeded.
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -168,8 +214,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-vulkan-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -187,8 +233,8 @@
     # Each merge job needs only its parent build matrix and is filtered by
     # tag-suffix in image_merge.yml's artifact-download pattern.
     gpu-nvidia-cuda-12-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -200,8 +246,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-nvidia-cuda-13-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -213,8 +259,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-intel-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -226,8 +272,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-hipblas-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: hipblas-jobs
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, hipblas-jobs]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -239,8 +285,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     nvidia-l4t-arm64-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: gh-runner
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, gh-runner]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -252,8 +298,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     nvidia-l4t-arm64-cuda-13-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: gh-runner
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, gh-runner]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -265,7 +311,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gh-runner:
-      if: github.repository == 'mudler/LocalAI'
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}

.github/workflows/lint.yml

@@ -8,6 +8,9 @@ on:
       - 'examples/**'
       - 'README.md'
       - '**/*.md'
+      # golangci-lint runs new-from-merge-base, so a diff with no touched Go
+      # lines can only ever be a no-op. See .agents/ci-caching.md.
+      - 'gallery/**'
   push:
     branches:
       - master
@@ -18,8 +21,36 @@ concurrency:
 
 jobs:
   golangci-lint:
-    runs-on: ubuntu-latest
+    # Self-hosted for PUSH only, and only in the canonical repo.
+    #
+    # This workflow also runs on pull_request, which for a fork PR means
+    # executing untrusted contributor code. That must never land on a
+    # self-hosted runner, so anything that is not a push to mudler/LocalAI stays
+    # on the ephemeral hosted pool. Pushes to master are trusted code that has
+    # already been reviewed and merged.
+    #
+    # Why at all: the hosted pool is shared account-wide and starved for 35
+    # straight minutes on 2026-07-31 while arc-runner-set kept completing jobs.
+    # Lint is small and runs on every commit, so it is a good candidate to move
+    # off the contended pool.
+    runs-on: ${{ (github.event_name == 'push' && github.repository == 'mudler/LocalAI') && 'arc-runner-set' || 'ubuntu-latest' }}
     steps:
+      - name: Preflight - required host tools
+        # The hosted images ship these; a self-hosted container image may not.
+        # Check up front so a missing tool reports itself by name instead of
+        # surfacing as an opaque failure inside `make protogen-go` (which needs
+        # curl + unzip for protoc) or `make lint`.
+        run: |
+          missing=""
+          for t in git curl unzip make tar; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
       - uses: actions/checkout@v7
         with:
           # Full history so golangci-lint's new-from-merge-base can reach
@@ -52,8 +83,25 @@ jobs:
     # container build (a missing transitive dep, a partial cuDNN family). Their
     # shell tests need nothing but bash + gcc + ldd, so run them on every PR
     # rather than waiting on a multi-GB cross-arch backend image build.
-    runs-on: ubuntu-latest
+    #
+    # Push-only self-hosted routing, same fork-safety reasoning as
+    # golangci-lint above.
+    runs-on: ${{ (github.event_name == 'push' && github.repository == 'mudler/LocalAI') && 'arc-runner-set' || 'ubuntu-latest' }}
     steps:
+      - name: Preflight - required host tools
+        # This job additionally needs a C toolchain: the packaging-script tests
+        # compile a throwaway binary and inspect it with ldd.
+        run: |
+          missing=""
+          for t in git make gcc ldd python3; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
       - uses: actions/checkout@v7
       - name: run packaging script tests
         run: make test-build-scripts
@@ -61,8 +109,14 @@ jobs:
       # The backend matrix path filter fails silently: a miss emits an empty
       # matrix, every job goes green, and the change reaches no image (#10946).
       # Its tests need only node, so they ride along with this job.
-      - uses: actions/setup-node@v4
+      - uses: actions/setup-node@v7
         with:
           node-version: '20'
       - name: run CI script tests
         run: make test-ci-scripts
+
+      # The shared python backend helpers (Options[] parsing, engine-arg
+      # mapping, model reference resolution) are stdlib-only, so their tests
+      # ride along here instead of waiting on a multi-GB backend image build.
+      - name: run shared python backend helper tests
+        run: make test-python-helpers

.github/workflows/secscan.yaml

@@ -7,6 +7,19 @@ on:
   schedule:
   - cron: '0 0 * * 0'
 
+# `push:` is deliberately unfiltered, so this fires on every push to every
+# branch and there is no pull_request event to key on -- the usual
+# `github.event.pull_request.number || github.sha` idiom used elsewhere would
+# key on the unique-per-commit sha and dedup nothing. Group on the ref instead
+# so successive pushes to the same feature branch supersede one another.
+#
+# Cancelling is safe here: the only output is a SARIF upload, and code scanning
+# tracks the latest result per ref, so a superseded scan has nothing to lose.
+# master is excluded anyway -- every commit on master gets its own scan.
+concurrency:
+  group: ci-secscan-${{ github.ref }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.ref != 'refs/heads/master' }}
+
 jobs:
   tests:
     runs-on: ubuntu-latest
@@ -15,9 +28,9 @@ jobs:
     steps:
       - name: Checkout Source
         uses: actions/checkout@v7
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
       - name: Run Gosec Security Scanner
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
         uses: securego/gosec@v2.27.1
         with:
           # we let the report trigger content trigger a failure using the GitHub Security features.
@@ -26,7 +39,7 @@ jobs:
           # noise, G104 unhandled errors) are inherent to that upstream code, not ours to rewrite.
           args: '-no-fail -exclude-dir=backend/go/supertonic -fmt sarif -out results.sarif ./...'
       - name: Upload SARIF file
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
         uses: github/codeql-action/upload-sarif@v4
         with:
           # Path to SARIF file relative to the root of the repository

.github/workflows/test-extra.yml

@@ -37,6 +37,8 @@ jobs:
       sglang: ${{ steps.detect.outputs.sglang }}
       acestep-cpp: ${{ steps.detect.outputs.acestep-cpp }}
       qwen3-tts-cpp: ${{ steps.detect.outputs.qwen3-tts-cpp }}
+      magpie-tts-cpp: ${{ steps.detect.outputs.magpie-tts-cpp }}
+      trellis2cpp: ${{ steps.detect.outputs.trellis2cpp }}
       rfdetr-cpp: ${{ steps.detect.outputs.rfdetr-cpp }}
       locate-anything-cpp: ${{ steps.detect.outputs.locate-anything-cpp }}
       vibevoice-cpp: ${{ steps.detect.outputs.vibevoice-cpp }}
@@ -866,6 +868,38 @@ jobs:
       - name: Test qwen3-tts-cpp
         run: |
           make --jobs=5 --output-sync=target -C backend/go/qwen3-tts-cpp test
+  tests-magpie-tts-cpp:
+    needs: detect-changes
+    if: needs.detect-changes.outputs.magpie-tts-cpp == 'true' || needs.detect-changes.outputs.run-all == 'true'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          submodules: true
+      - name: Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential cmake curl libopenblas-dev ffmpeg
+      - name: Setup Go
+        uses: actions/setup-go@v5
+      - name: Display Go version
+        run: go version
+      - name: Proto Dependencies
+        run: |
+          # Install protoc
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
+          unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Build magpie-tts-cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/magpie-tts-cpp
+      - name: Test magpie-tts-cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/magpie-tts-cpp test
   # Per-backend smoke for rfdetr-cpp: builds the .so + Go binary and runs
   # `make -C backend/go/rfdetr-cpp test`. test.sh fetches the small (~20 MB)
   # rfdetr-nano-q8_0 GGUF from the published mudler/rfdetr-cpp-nano HF repo
@@ -902,6 +936,41 @@ jobs:
       - name: Test rfdetr-cpp
         run: |
           make --jobs=5 --output-sync=target -C backend/go/rfdetr-cpp test
+  # Weight-free packaged-backend smoke for trellis2cpp. Starting run.sh loads
+  # libtrellis2 + ggml, resolves the complete C ABI (including remeshing), and
+  # answers gRPC Health without downloading or loading the multi-GB model set.
+  tests-trellis2cpp:
+    needs: detect-changes
+    if: needs.detect-changes.outputs.trellis2cpp == 'true' || needs.detect-changes.outputs.run-all == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          submodules: true
+      - name: Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential cmake curl unzip
+      - name: Setup Go
+        uses: actions/setup-go@v5
+      - name: Display Go version
+        run: go version
+      - name: Proto Dependencies
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
+          unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Build trellis2cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/trellis2cpp
+      - name: Test trellis2cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/trellis2cpp test
   # Per-backend e2e for locate-anything-cpp: builds the .so + Go binary and
   # runs `make -C backend/go/locate-anything-cpp test`. test.sh fetches the
   # locate-anything-q8_0 GGUF (~6.3 GB, NVIDIA LocateAnything-3B) from the

.github/workflows/test.yml

@@ -24,8 +24,13 @@ jobs:
         uses: actions/checkout@v7
         with:
           submodules: true
-      - name: Free disk space
-        uses: ./.github/actions/free-disk-space
+      # No free-disk-space step here on purpose. That action exists to make room
+      # for docker buildx layers, and this job runs no buildx step. It was also
+      # sized for a `make test` that downloaded multi-GB GGUF/whisper fixtures
+      # and built llama-cpp/whisper/stablediffusion-ggml; after the test-suite
+      # reorg it does neither (see the Makefile test target). It cost ~3min of
+      # every run, and its tool-cache:true wipe also forced setup-go and
+      # setup-node to re-download toolchains that ship preinstalled.
       - name: Setup Go ${{ matrix.go-version }}
         uses: actions/setup-go@v5
         with:

.github/workflows/tests-e2e.yml

@@ -3,6 +3,14 @@ name: 'E2E Backend Tests'
 
 on:
   pull_request:
+    # The e2e suite drives backends over gRPC directly and reads none of these
+    # paths, so a diff confined to them cannot move it.
+    # See .agents/ci-caching.md.
+    paths-ignore:
+      - 'gallery/**'
+      - 'docs/**'
+      - 'examples/**'
+      - '**/*.md'
   push:
     branches:
       - master

.github/workflows/yaml-check.yml

@@ -1,6 +1,11 @@
 name: 'Yamllint GitHub Actions'
 on:
   - pull_request
+
+concurrency:
+  group: ci-yamllint-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
 jobs:
   yamllint:
     name: 'Yamllint'

.gitignore

@@ -30,6 +30,7 @@ LocalAI
 # Go backend packages whose main lives under backend/go/.
 /cloud-proxy
 /local-store
+/valkey-store
 # prevent above rules from omitting the helm chart
 !charts/*
 # prevent above rules from omitting the api/localai folder
@@ -61,6 +62,11 @@ prepare
 /ggml-metal.metal
 docs/static/gallery.html
 
+# Hugo build output and lock files (docs/ and website/)
+docs/public/
+website/public/
+.hugo_build.lock
+
 # Protobuf generated files
 *.pb.go
 *pb2.py
@@ -111,3 +117,10 @@ core/http/react-ui/test-results/
 # the realtime-conformance gate; only the .fizz sources are authoritative.
 formal-verification/*.json
 formal-verification/out/
+
+# `go build ./.github/ci/apexentries` drops a binary of the package name into
+# whatever directory it runs in, one `git add -A` away from being committed.
+# Both paths are anchored: an unanchored `apexentries` would also match the
+# package directory itself and untrack the source.
+/apexentries
+/.github/ci/apexentries/apexentries

ADOPTERS.md

@@ -0,0 +1,48 @@
+# Adopters
+
+Organisations running LocalAI, listed by the people who run it.
+
+If your organisation uses LocalAI and you are happy to say so publicly, open a
+pull request adding a row to the table below. That pull request is how we know
+we have permission to list you, which is why we do not add anybody ourselves.
+
+You do not need to be a large company, and you do not need to disclose anything
+sensitive. A sentence on what you use it for is more useful to other readers
+than a logo.
+
+## How to add yourself
+
+1. Add a row to the table, in alphabetical order.
+2. Use your organisation's usual name and a link to your site.
+3. Say briefly what you use LocalAI for, and whether it is in production.
+4. Open the pull request from an account that makes it plausible you speak for
+   the organisation, or say in the description who you are. We may ask.
+
+To be removed, open a pull request deleting your row, or email
+[info@localai.io](mailto:info@localai.io). We will not ask why.
+
+## Who is using LocalAI
+
+<!-- Keep alphabetical. Columns: Organisation | What for | Status -->
+
+| Organisation | What they use it for | Status |
+|---|---|---|
+| _Your organisation here_ | | |
+
+## What this list is not
+
+This is not a list of everyone who has ever starred the repository, and it is
+not a list of the employers of people who have contributed a patch. Both of
+those are easy to scrape and neither means what a logo wall implies.
+
+The website shows two separate things, both of which are checkable without
+anybody's permission:
+
+- **Engineers from these companies have contributed code.** Evidence is the
+  commit history plus the employer on that person's public GitHub profile. It
+  is a claim about a person, not about their employer.
+- **These projects integrate LocalAI.** Evidence is a reference to LocalAI in
+  that project's own repository or documentation.
+
+Those two lists live in [`website/data/ecosystem.yaml`](website/data/ecosystem.yaml).
+This file is the third, stronger thing: organisations that chose to say so.

AGENTS.md

@@ -32,16 +32,18 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 | [.agents/adding-gallery-models.md](.agents/adding-gallery-models.md) | Adding GGUF models from HuggingFace to the model gallery |
 | [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) | LocalAI Assistant chat modality — adding admin tools to the in-process MCP server, editing skill prompts, keeping REST + MCP + skills in sync |
 | [.agents/backend-signing.md](.agents/backend-signing.md) | Backend OCI image signing (keyless cosign + sigstore-go) — producer-side CI setup, consumer-side gallery `verification:` block, strict mode (`LOCALAI_REQUIRE_BACKEND_INTEGRITY`), revocation via `not_before` |
+| [.agents/preparing-a-release.md](.agents/preparing-a-release.md) | Cutting a release: PR labels, `RELEASE_NOTES_vX.Y.Z.md`, the blog post under `website/content/blog/`, and the demo clips under `website/static/media/` |
 
 ## Quick Reference
 
-- **Git hooks & coverage gates**: Run `make install-hooks` once per clone so the pre-commit lint + coverage gates run. **Never bypass them with `git commit --no-verify`, and never lower a coverage baseline or widen a gate's tolerance to turn a red gate green** — the coverage ratchet only moves up. If a change drops coverage, add tests to raise it (e.g. render-smoke specs). See [.agents/building-and-testing.md](.agents/building-and-testing.md).
+- **Coverage gates**: Never lower a coverage baseline or widen a gate's tolerance to turn a red gate green — the coverage ratchet only moves up. If a change drops coverage, add tests to raise it (e.g. render-smoke specs). See [.agents/building-and-testing.md](.agents/building-and-testing.md).
 - **Logging**: Use `github.com/mudler/xlog` (same API as slog)
 - **Go style**: Prefer `any` over `interface{}`
 - **Comments**: Explain *why*, not *what*
 - **Docs (docs-with-code rule)**: When you change user-facing behavior (API endpoints, CLI flags, config keys, or features), update the corresponding page under `docs/content/` in the SAME change, not as a follow-up. A user-facing change without a matching docs update is incomplete. See also the documentation conventions in [.agents/coding-style.md](.agents/coding-style.md).
 - **New API endpoints**: LocalAI advertises its capability surface in several independent places — swagger `@Tags`, `/api/instructions` registry, auth `RouteFeatureRegistry`, React UI `capabilities.js`, docs. Read [.agents/api-endpoints-and-auth.md](.agents/api-endpoints-and-auth.md) and follow its checklist — missing any surface means clients, admins, and the UI won't know the endpoint exists.
 - **Admin endpoints → MCP tool**: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in `pkg/mcp/localaitools/`. The LocalAI Assistant chat modality and the standalone `local-ai mcp-server` consume that package; drift between REST and MCP is a real risk. Read [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) — the `TestToolHTTPRouteMappingComplete` test fails until you wire the new tool and update the route map.
+- **Releases ship with a post and clips**: a release is not done at the tag. It needs labelled PRs, `RELEASE_NOTES_vX.Y.Z.md`, a blog post under `website/content/blog/`, and a short demo clip in `website/static/media/` for each notable feature. See [.agents/preparing-a-release.md](.agents/preparing-a-release.md).
 - **Build**: Inspect `Makefile` and `.github/workflows/` — ask the user before running long builds
 - **Backend OS coverage**: a new backend must target every OS it can build for, not just Linux. `.github/backend-matrix.yml` has two matrices — `include:` (Linux) and `includeDarwin:` (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the `includeDarwin` entry + `backend/index.yaml` `metal:` entries, or say in the PR why an OS is unsupported. See the darwin checklist in [.agents/adding-backends.md](.agents/adding-backends.md).
 - **Gallery variant ranking**: a gallery entry can declare `variants` (alternative builds of the same weights), and LocalAI ranks the ones a host can run by engine preference first, size second. A new backend that should be preferred on some hardware must be listed in `engineNamePreferenceRules` in `pkg/system/capabilities.go`; the sibling `backendBuildTagPreferenceRules` speaks build tags rather than engine names, and using the wrong table matches nothing without erroring. See [.agents/adding-backends.md](.agents/adding-backends.md).

CONTRIBUTING.md

@@ -198,7 +198,6 @@ For AI-assisted development, see [`AGENTS.md`](AGENTS.md) (or the equivalent [`C
 
 - Prefer modern Go idioms — for example, use `any` instead of `interface{}`.
 - Use [`golangci-lint`](https://golangci-lint.run) to catch common issues before submitting a PR.
-- Run `make install-hooks` once per clone to enable the pre-commit hook: Go changes run `make lint` + the coverage gate (`make test-coverage-check`); `core/http/react-ui/` changes run the Playwright e2e suite (`make test-ui`). Bypass a single commit with `git commit --no-verify`.
 - Use [`github.com/mudler/xlog`](https://github.com/mudler/xlog) for logging (same API as `slog`). Do not use `fmt.Println` or the standard `log` package for operational logging.
 - Use tab indentation for Go files (as defined in `.editorconfig`).
 
@@ -268,7 +267,7 @@ make test-e2e
 
 ### React UI tests and coverage
 
-The React UI (`core/http/react-ui/`) is covered by Playwright e2e specs, gated by a **monotonic line-coverage ratchet** (`make test-ui-coverage-check`, run in CI and pre-commit). The metric is non-deterministic — a fast local box reads higher than a slow CI runner for the same code — so a small tolerance is unavoidable.
+The React UI (`core/http/react-ui/`) is covered by Playwright e2e specs, gated by a **monotonic line-coverage ratchet** (`make test-ui-coverage-check`, run in CI). The metric is non-deterministic — a fast local box reads higher than a slow CI runner for the same code — so a small tolerance is unavoidable.
 
 **If your change lowers UI coverage, raise it back by adding specs — do not widen the tolerance or hand-lower the baseline.** A *render-smoke* spec (navigate to a page, assert its header is visible) cheaply covers an entire lazy page. See `core/http/react-ui/e2e/page-render-smoke.spec.js` and the full policy in [.agents/building-and-testing.md](.agents/building-and-testing.md#react-ui-coverage).
 

Makefile

@@ -1,5 +1,5 @@
 # Disable parallel execution for backend builds
-.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
+.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/trellis2cpp backends/trellis2cpp-darwin backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/valkey-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/magpie-tts-cpp backends/vllm-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin backends/audio-cpp backends/audio-cpp-darwin
 
 GOCMD=go
 GOTEST=$(GOCMD) test
@@ -69,7 +69,7 @@ else
 	GORELEASER=$(shell which goreleaser)
 endif
 
-TEST_PATHS?=./api/... ./pkg/... ./core/... ./backend/go/cloud-proxy/... ./backend/go/local-store/...
+TEST_PATHS?=./api/... ./pkg/... ./core/... ./backend/go/cloud-proxy/... ./backend/go/local-store/... ./backend/go/valkey-store/...
 
 ## Coverage output and the committed baseline that CI compares against.
 ## The gate is strict: total coverage must never decrease (no tolerance).
@@ -103,7 +103,7 @@ COVERAGE_E2E_LABELS?=!real-models
 COVERAGE_EXCLUDE_RE?=grpc/proto/.*[.]pb[.]go
 
 
-.PHONY: all test test-coverage test-coverage-baseline test-coverage-check test-backend-cpp test-build-scripts test-ui test-ui-coverage-baseline test-ui-coverage-check install-hooks build vendor lint lint-all
+.PHONY: all test test-coverage test-coverage-baseline test-coverage-check test-backend-cpp test-build-scripts test-ui test-ui-coverage-baseline test-ui-coverage-check build vendor lint lint-all
 
 all: help
 
@@ -172,6 +172,15 @@ build-dev: ## Run LocalAI in dev mode with live reload
 dev-dist:
 	$(GORELEASER) build --snapshot --clean
 
+## PR-time variant of dev-dist: builds only the host platform instead of all
+## three release targets (linux/amd64, linux/arm64, darwin/arm64). The point of
+## running goreleaser on a PR is to catch a broken config or a broken
+## before-hook (protogen-go, react-ui, go mod tidy), and --single-target still
+## exercises every one of those. Nothing consumes a PR's cross-compiled
+## binaries. master pushes and tags still run the full dev-dist/dist.
+dev-dist-single:
+	$(GORELEASER) build --snapshot --clean --single-target
+
 dist:
 	$(GORELEASER) build --clean
 
@@ -222,6 +231,14 @@ test-build-scripts:
 test-ci-scripts:
 	@set -e; for t in scripts/lib/*_test.mjs; do echo "== $$t"; node --test "$$t"; done
 
+## Runs the unit tests for the shared python backend helpers. These modules are
+## pure stdlib on purpose so they run without any backend venv; the list is
+## explicit because their siblings (model_identity_test) import grpc and the
+## generated protobufs, which only exist inside a built backend.
+PYTHON_HELPER_TESTS?=python_utils_test vllm_utils_test model_utils_test mlx_utils_test parent_watch_test
+test-python-helpers:
+	cd backend/python/common && python3 -m unittest $(PYTHON_HELPER_TESTS)
+
 ## Runs the core suite ($(TEST_PATHS)) with statement-coverage instrumentation
 ## and writes a merged profile to $(COVERAGE_PROFILE). Deliberately omits
 ## --fail-fast so a single failure doesn't truncate the coverage number, and
@@ -269,8 +286,7 @@ LINT_EXCLUDE_DIRS_RE=/(backend/go/(piper|silero-vad|llm)|cmd/launcher)(/|$$)
 
 ## Set LINT_NEW_FROM to a git ref to override .golangci.yml's
 ## new-from-merge-base (origin/master). Useful from a fork clone where
-## origin/master is stale relative to the canonical repo — the pre-commit
-## hook passes the resolved upstream ref here so local lint matches CI.
+## origin/master is stale relative to the canonical repo.
 LINT_NEW_FROM?=
 lint:
 	@command -v golangci-lint >/dev/null 2>&1 || { \
@@ -289,17 +305,6 @@ lint-all:
 	}
 	golangci-lint run --new=false --new-from-merge-base= --new-from-rev= $$(go list -e -f '{{.Dir}}' ./... | grep -vE '$(LINT_EXCLUDE_DIRS_RE)')
 
-########################################################
-## Git hooks
-########################################################
-## Points git at the versioned .githooks/ directory so the pre-commit hook
-## (lint + coverage gate) runs locally. Run once per clone. Undo with:
-## `git config --unset core.hooksPath`. Skip a single commit with
-## `git commit --no-verify`.
-install-hooks:
-	git config core.hooksPath .githooks
-	@echo 'Installed git hooks: core.hooksPath -> .githooks (pre-commit runs lint + test-coverage-check on Go changes)'
-
 ########################################################
 ## E2E AIO tests (uses standard image with pre-configured models)
 ########################################################
@@ -398,6 +403,15 @@ test-stores: backends/local-store
 	BACKENDS_PATH=$(abspath ./)/backends \
 	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) -v -r tests/integration
 
+## Valkey-backed vector-store integration. Requires a running Valkey Search
+## server (valkey/valkey-bundle:9.1.0) reachable at $$VALKEY_ADDR — the suite
+## skips itself when VALKEY_ADDR is unset. Builds the backend on demand and
+## points the model loader at it via BACKENDS_PATH. Label-filtered to the
+## valkey specs so it does not also run the in-memory local-store suite.
+test-valkey-store: backends/valkey-store
+	BACKENDS_PATH=$(abspath ./)/backends \
+	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) --label-filter='valkey' -v -r tests/integration
+
 test-opus:
 	@echo 'Running opus backend tests'
 	$(MAKE) -C backend/go/opus libopusshim.so
@@ -606,6 +620,8 @@ prepare-test-extra: protogen-python
 	$(MAKE) -C backend/rust/kokoros kokoros-grpc
 	$(MAKE) -C backend/go/rfdetr-cpp
 	$(MAKE) -C backend/go/locate-anything-cpp
+	$(MAKE) -C backend/go/trellis2cpp
+	$(MAKE) -C backend/go/valkey-store
 
 test-extra: prepare-test-extra
 	$(MAKE) -C backend/python/transformers test
@@ -637,6 +653,9 @@ test-extra: prepare-test-extra
 	$(MAKE) -C backend/go/locate-anything-cpp test
 	$(MAKE) -C backend/go/depth-anything-cpp test
 	$(MAKE) -C backend/go/supertonic test
+	$(MAKE) -C backend/go/vllm-cpp test
+	$(MAKE) -C backend/go/trellis2cpp test
+	$(MAKE) -C backend/go/valkey-store test
 
 ##
 ## End-to-end gRPC tests that exercise a built backend container image.
@@ -1199,6 +1218,10 @@ backends/privacy-filter-darwin: build
 	bash ./scripts/build/privacy-filter-darwin.sh
 	./local-ai backends install "ocifile://$(abspath ./backend-images/privacy-filter.tar)"
 
+backends/audio-cpp-darwin: build
+	bash ./scripts/build/audio-cpp-darwin.sh
+	./local-ai backends install "ocifile://$(abspath ./backend-images/audio-cpp.tar)"
+
 build-darwin-python-backend: build
 	bash ./scripts/build/python-darwin.sh
 
@@ -1229,6 +1252,10 @@ backends/stablediffusion-ggml-darwin:
 	BACKEND=stablediffusion-ggml BUILD_TYPE=metal $(MAKE) build-darwin-go-backend
 	./local-ai backends install "ocifile://$(abspath ./backend-images/stablediffusion-ggml.tar)"
 
+backends/trellis2cpp-darwin:
+	BACKEND=trellis2cpp BUILD_TYPE=metal $(MAKE) build-darwin-go-backend
+	./local-ai backends install "ocifile://$(abspath ./backend-images/trellis2cpp.tar)"
+
 backend-images:
 	mkdir -p backend-images
 
@@ -1252,14 +1279,21 @@ BACKEND_DS4 = ds4|ds4|.|false|false
 # openai-privacy-filter PII/NER token classifier) — the TokenClassify RPC for
 # the PII redactor tier, on stock ggml with no llama.cpp carry-patches.
 BACKEND_PRIVACY_FILTER = privacy-filter|privacy-filter|.|false|false
+# audio-cpp wraps 0xShug0/audio.cpp, a multi-family ggml audio inference engine
+# (TTS, ASR, VAD, diarization, source separation, music generation). Builds
+# against apt gRPC/protobuf rather than a prebuilt base-grpc image; the reason
+# is on the audio-cpp block in .github/backend-matrix.yml.
+BACKEND_AUDIO_CPP = audio-cpp|audio-cpp|.|false|false
 
 # Golang backends
 BACKEND_PIPER = piper|golang|.|false|true
 BACKEND_LOCAL_STORE = local-store|golang|.|false|true
+BACKEND_VALKEY_STORE = valkey-store|golang|.|false|true
 BACKEND_CLOUD_PROXY = cloud-proxy|golang|.|false|true
 BACKEND_HUGGINGFACE = huggingface|golang|.|false|true
 BACKEND_SILERO_VAD = silero-vad|golang|.|false|true
 BACKEND_STABLEDIFFUSION_GGML = stablediffusion-ggml|golang|.|--progress=plain|true
+BACKEND_TRELLIS2CPP = trellis2cpp|golang|.|--progress=plain|true
 BACKEND_WHISPER = whisper|golang|.|false|true
 BACKEND_CRISPASR = crispasr|golang|.|false|true
 BACKEND_PARAKEET_CPP = parakeet-cpp|golang|.|false|true
@@ -1269,6 +1303,8 @@ BACKEND_VOXTRAL = voxtral|golang|.|false|true
 BACKEND_ACESTEP_CPP = acestep-cpp|golang|.|false|true
 BACKEND_QWEN3_TTS_CPP = qwen3-tts-cpp|golang|.|false|true
 BACKEND_MOSS_TTS_CPP = moss-tts-cpp|golang|.|false|true
+BACKEND_MAGPIE_TTS_CPP = magpie-tts-cpp|golang|.|false|true
+BACKEND_VLLM_CPP = vllm-cpp|golang|.|false|true
 BACKEND_OMNIVOICE_CPP = omnivoice-cpp|golang|.|false|true
 BACKEND_VIBEVOICE_CPP = vibevoice-cpp|golang|.|false|true
 BACKEND_LOCALVQE = localvqe|golang|.|false|true
@@ -1351,12 +1387,15 @@ $(eval $(call generate-docker-build-target,$(BACKEND_TURBOQUANT)))
 $(eval $(call generate-docker-build-target,$(BACKEND_BONSAI)))
 $(eval $(call generate-docker-build-target,$(BACKEND_DS4)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PRIVACY_FILTER)))
+$(eval $(call generate-docker-build-target,$(BACKEND_AUDIO_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PIPER)))
 $(eval $(call generate-docker-build-target,$(BACKEND_LOCAL_STORE)))
+$(eval $(call generate-docker-build-target,$(BACKEND_VALKEY_STORE)))
 $(eval $(call generate-docker-build-target,$(BACKEND_CLOUD_PROXY)))
 $(eval $(call generate-docker-build-target,$(BACKEND_HUGGINGFACE)))
 $(eval $(call generate-docker-build-target,$(BACKEND_SILERO_VAD)))
 $(eval $(call generate-docker-build-target,$(BACKEND_STABLEDIFFUSION_GGML)))
+$(eval $(call generate-docker-build-target,$(BACKEND_TRELLIS2CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_WHISPER)))
 $(eval $(call generate-docker-build-target,$(BACKEND_CRISPASR)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PARAKEET_CPP)))
@@ -1396,6 +1435,8 @@ $(eval $(call generate-docker-build-target,$(BACKEND_ACE_STEP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_ACESTEP_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_QWEN3_TTS_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_MOSS_TTS_CPP)))
+$(eval $(call generate-docker-build-target,$(BACKEND_MAGPIE_TTS_CPP)))
+$(eval $(call generate-docker-build-target,$(BACKEND_VLLM_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_OMNIVOICE_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_VIBEVOICE_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_LOCALVQE)))
@@ -1415,7 +1456,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_SUPERTONIC)))
 docker-save-%: backend-images
 	docker save local-ai-backend:$* -o backend-images/$*.tar
 
-docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter
+docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-magpie-tts-cpp docker-build-vllm-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter docker-build-trellis2cpp docker-build-valkey-store docker-build-audio-cpp
 
 ########################################################
 ### Mock Backend for E2E Tests
@@ -1450,7 +1491,7 @@ test-ui-e2e: build-ui-test-server
 UI_TEST_WORKERS ?=
 PLAYWRIGHT_WORKERS_FLAG = $(if $(UI_TEST_WORKERS),--workers=$(UI_TEST_WORKERS),)
 
-## Fast Playwright e2e run used by the pre-commit hook on React UI changes.
+## Fast Playwright e2e run for local React UI validation.
 ## Force-rebuilds the (non-instrumented) dist so the suite tests the working
 ## tree — not a stale dist the `react-ui` skip-guard would leave — re-embeds
 ## it into ui-test-server, and runs the specs. Uses the nix-provided browser
@@ -1507,7 +1548,12 @@ swagger:
 gen-assets:
 	$(GOCMD) run core/dependencies_manager/manager.go webui_static.yaml core/http/static/assets
 
-## Documentation
+## Documentation and website
+# The published site is two Hugo sites: website/ owns the root, docs/ is nested
+# under /docs/. Serve them separately while editing; use `make site` to get the
+# merged tree (including the legacy URL redirects) that GitHub Pages deploys.
+SITE_BASE_URL?=http://localhost:8000
+
 docs/layouts/_default:
 	mkdir -p docs/layouts/_default
 
@@ -1519,12 +1565,30 @@ docs/public: docs/layouts/_default docs/static/gallery.html
 
 docs-clean:
 	rm -rf docs/public
+	rm -rf website/public
 	rm -rf docs/static/gallery.html
 
 .PHONY: docs
 docs: docs/static/gallery.html
 	cd docs && hugo serve
 
+.PHONY: website
+website:
+	cd website && hugo serve
+
+.PHONY: site
+site: docs/static/gallery.html
+	rm -rf website/public docs/public
+	cd website && hugo --minify --baseURL "$(SITE_BASE_URL)/"
+	cd docs && hugo --minify --baseURL "$(SITE_BASE_URL)/docs/"
+	mkdir -p website/public/docs
+	cp -R docs/public/. website/public/docs/
+	./.github/ci/gen-redirects.sh website/public "$(SITE_BASE_URL)/"
+
+.PHONY: site-serve
+site-serve: site
+	cd website/public && python3 -m http.server 8000
+
 ########################################################
 ## Platform-specific builds
 ########################################################

README.md

@@ -231,18 +231,21 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native
 
 | Backend | What it does |
 |---------|-------------|
+| [vllm.cpp](https://github.com/mudler/vllm.cpp) | From-scratch C++20 port of vLLM for text generation: paged KV cache, continuous batching, prefix caching, safetensors + GGUF loading, engine-enforced structured output, on CPU, CUDA, Metal and Vulkan |
 | [parakeet.cpp](https://github.com/mudler/parakeet.cpp) | C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription |
 | [moss-transcribe.cpp](https://github.com/localai-org/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass |
 | [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | C++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec |
+| [magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp) | C++/GGML port of NVIDIA's Magpie TTS Multilingual 357M: 22.05 kHz mono text-to-speech in 5 voices and 9+ languages, with the NanoCodec neural codec and tokenizer/G2P embedded in a single GGUF |
 | [ced.cpp](https://github.com/localai-org/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
 | [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
-| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Voxtral Realtime 4B speech-to-text in pure C |
+| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Mistral Voxtral-4B-TTS text-to-speech in pure C: 20 preset voices across 9 languages, 24 kHz WAV output, no dependencies beyond libc |
 | [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization |
 | [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
 | [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) |
 | [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose estimation |
 | [face-detect.cpp](https://github.com/mudler/face-detect.cpp) | Face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace), replacing the Python insightface backend |
 | [free-splatter.cpp](https://github.com/localai-org/free-splatter.cpp) | Pose-free 3D reconstruction (FreeSplatter): turns a handful of plain photos into 3D Gaussians, no camera poses or GPU required |
+| [trellis2.cpp](https://github.com/localai-org/trellis2cpp) | C++/GGML port of Microsoft TRELLIS.2: single-image to textured 3D mesh (GLB with PBR materials) |
 | [privacy-filter.cpp](https://github.com/localai-org/privacy-filter.cpp) | Standalone GGML PII/NER token-classification engine powering LocalAI's PII redaction tier |
 | [LocalVQE](https://github.com/localai-org/LocalVQE) | Joint acoustic echo cancellation, noise suppression, and dereverberation |
 | [local-store](https://github.com/mudler/LocalAI) | Local-first vector database for embeddings (shipped in-tree) |

backend/Dockerfile.audio-cpp

@@ -0,0 +1,120 @@
+ARG BASE_IMAGE=ubuntu:24.04
+ARG APT_MIRROR=""
+ARG APT_PORTS_MIRROR=""
+
+# audio-cpp: 0xShug0/audio.cpp, a ggml audio inference framework covering TTS,
+# ASR, VAD, diarization, source separation and music generation, wrapped as a
+# LocalAI gRPC backend.
+#
+# BASE_IMAGE is ubuntu:24.04 for cpu and vulkan builds, or
+# nvidia/cuda:<ver>-devel-ubuntu24.04 for cublas builds; both ship apt and
+# Ubuntu Noble packages, and the CUDA base additionally provides
+# /usr/local/cuda. BUILD_TYPE selects the engine backend in the Makefile:
+# "" = portable CPU with all ggml CPU variants, "cublas" ->
+# -DENGINE_ENABLE_CUDA=ON, "vulkan" -> -DENGINE_ENABLE_VULKAN=ON. Darwin
+# (Metal) builds bypass this Dockerfile entirely.
+#
+# Upstream needs GCC 13 or newer, which ubuntu:24.04 and the CUDA 12/13
+# devel-ubuntu24.04 images all provide.
+#
+# THIS BACKEND CANNOT USE .docker/install-base-deps.sh OR THE PREBUILT
+# quay.io/go-skynet/ci-cache:base-grpc-* IMAGES, AND THAT IS NOT A STYLE CHOICE.
+#
+# Both supply gRPC v1.65 built from source at /opt/grpc, which downstream
+# Dockerfiles copy to /usr/local. That gRPC vendors protobuf v26, and protobuf
+# has depended on abseil since v22: google/protobuf/message_lite.h includes
+# absl/strings/cord.h. audio.cpp links sentencepiece, and our CMakeLists sets
+# SPM_PROTOBUF_PROVIDER=package so sentencepiece uses the same protobuf the
+# generated backend.pb.cc was built against (the alternative broke every
+# nested-message parse; the full account is in backend/cpp/audio-cpp/CMakeLists.txt).
+# That makes sentencepiece's init.h include the external message_lite.h while it
+# still includes its own vendored mini-abseil from third_party/absl. The vendored
+# copy declares `namespace absl { namespace internal { ... } }` and real abseil
+# declares `namespace absl { inline namespace lts_20240116 { namespace internal
+# { ... } } }`, so every `absl::internal::` reference becomes ambiguous and the
+# compile dies in absl/base/casts.h. Verified, not theorised: building this image
+# against the base-grpc-amd64 prebuilt fails at
+# sentencepiece-static/error.cc.o with "reference to 'internal' is ambiguous".
+#
+# Ubuntu Noble's apt protobuf is 3.21.12, which predates the abseil dependency,
+# so message_lite.h pulls in no abseil and the vendored copy is the only one in
+# scope. That is also the exact protobuf/gRPC pair every unit and end-to-end run
+# of this backend has been verified against. Keep it: a from-source gRPC here
+# does not buy a faster build, it buys a broken one.
+#
+# The install-base-deps path is additionally unsafe because it drops protoc 27.1
+# into /usr/local/bin, which shadows apt's protoc on PATH and would generate
+# protobuf-27 sources to be compiled against 3.21 headers.
+FROM ${BASE_IMAGE} AS builder
+ARG BUILD_TYPE
+ARG TARGETARCH
+ARG TARGETVARIANT
+ARG APT_MIRROR
+ARG APT_PORTS_MIRROR
+# Selects the CUDA architecture list in backend/cpp/audio-cpp/Makefile. It has
+# to be forwarded: upstream compiles engine_runtime for `native` when
+# CMAKE_CUDA_ARCHITECTURES is unset, and no CI runner has a GPU to enumerate.
+# The value is the same cuda-major-version the matrix entry declares.
+ARG CUDA_MAJOR_VERSION
+
+ENV BUILD_TYPE=${BUILD_TYPE} \
+    CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} \
+    APT_MIRROR=${APT_MIRROR} \
+    APT_PORTS_MIRROR=${APT_PORTS_MIRROR} \
+    DEBIAN_FRONTEND=noninteractive \
+    PATH=/usr/local/cuda/bin:${PATH}
+
+WORKDIR /build
+
+# gRPC/protobuf from apt, deliberately; see the block above. libgrpc++-dev ships
+# a CMake config so find_package(gRPC CONFIG) resolves, and libprotobuf-dev
+# lands in the layout CMake's FindProtobuf module expects, which matters because
+# sentencepiece runs a bare find_package(Protobuf REQUIRED) with no CONFIG
+# fallback of its own.
+#
+# BUILD_TYPE=vulkan additionally needs the loader headers and glslc; both are in
+# Noble. The CUDA toolkit for BUILD_TYPE=cublas comes from BASE_IMAGE.
+RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mirror \
+    sh /usr/local/sbin/apt-mirror && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends \
+        git cmake build-essential pkg-config ca-certificates \
+        libgrpc++-dev libprotobuf-dev protobuf-compiler protobuf-compiler-grpc && \
+    if [ "${BUILD_TYPE}" = "vulkan" ]; then \
+        apt-get install -y --no-install-recommends libvulkan-dev glslc; \
+    fi && \
+    if [ "${TARGETARCH}" = "arm64" ]; then \
+        apt-get install -y --no-install-recommends gcc-14 g++-14; \
+    fi && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+COPY . /LocalAI
+
+# gcc-14 on arm64, for the same reason llama-cpp does it in
+# .docker/llama-cpp-compile.sh: ggml's CPU_ALL_VARIANTS table includes armv9.2
+# variants built with -march=...+sme, and Noble's default gcc-13 rejects that
+# feature modifier outright ("invalid feature modifier 'sme'"). Every variant in
+# the table has to COMPILE even though a host only ever dlopens the one its own
+# CPU supports, so one unbuildable variant fails the whole image.
+#
+# ON EVERY arm64 BUILD_TYPE, which is where this differs from llama-cpp's script
+# and why that difference is spelled out rather than assumed. llama-cpp only
+# needs gcc-14 for its pure-CPU image because its GPU builds run
+# llama-cpp-fallback, which has no variant table at all. This backend's Makefile
+# sets ENGINE_ENABLE_CPU_ALL_VARIANTS for every non-Darwin build, GPU included,
+# so an arm64 GPU image would hit the identical compile error. Gating this on an
+# empty BUILD_TYPE would leave that trap armed for the first arm64 GPU entry
+# added to the matrix, which today has none.
+RUN --mount=type=cache,target=/root/.ccache,id=audio-cpp-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
+    if [ "${TARGETARCH}" = "arm64" ]; then \
+        export CC=gcc-14 CXX=g++-14; \
+    fi && \
+    make -C /LocalAI/backend/cpp/audio-cpp BUILD_TYPE=${BUILD_TYPE} \
+        CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} NATIVE=false grpc-server package
+
+# The package directory is the whole image: run.sh, grpc-server, the dlopened
+# ggml CPU variants, the bundled loader and its library closure, and the
+# bundled silero_vad / marblenet_vad assets. Nothing else exists at run time.
+FROM scratch
+COPY --from=builder /LocalAI/backend/cpp/audio-cpp/package/. ./

backend/Dockerfile.golang

@@ -221,10 +221,75 @@ RUN if [ "${BACKEND}" = "crispasr" ]; then \
     apt-get clean && rm -rf /var/lib/apt/lists/*; \
 fi
 
-COPY . /LocalAI
+# sherpa-onnx links onnxruntime's CUDA execution provider, and
+# libonnxruntime_providers_cuda.so has cuDNN as a hard DT_NEEDED. The
+# onnxruntime GPU tarball does not ship cuDNN itself, so without this the
+# builder has none (the arm64 + CUDA 13 branch above is the only other place
+# that installs it) and package-gpu-libs.sh correctly refuses to produce a
+# package that references cuDNN with no cuDNN available to it.
+#
+# Installed per-backend rather than for every cublas build: the auto-detection
+# in package-gpu-libs.sh bundles only what a package actually references, so
+# the ggml backends would not grow either way, but they would all pay ~1.1 GB
+# of builder layer and registry cache for a library they never call.
+#
+# Runtime package only, no -dev: sherpa-onnx consumes onnxruntime's prebuilt
+# CUDA provider and never compiles against cuDNN headers. libcudnn9-cuda-N
+# carries the dispatcher plus all seven dlopen()ed sublibraries, which is what
+# complete_cudnn_family needs to assemble a whole bundle.
+RUN <<EOT bash
+    if [ "${BACKEND}" = "sherpa-onnx" ] && [ "${BUILD_TYPE}" = "cublas" ] && [ "${SKIP_DRIVERS}" = "false" ]; then
+        apt-get update && \
+        apt-get install -y --no-install-recommends \
+            libcudnn9-cuda-${CUDA_MAJOR_VERSION} && \
+        ldconfig && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists/*
+    fi
+EOT
 
 RUN git config --global --add safe.directory /LocalAI
 
+# Prebuild the native engine from a layer that depends on this backend's own
+# directory and nothing else.
+#
+# The expensive part of a C++ backend build is the engine: each of these
+# Makefiles clones an upstream repo at a pinned SHA and compiles it once per
+# SIMD variant (depth-anything-cpp builds four: avx, avx2, avx512, fallback),
+# and those variant targets depend only on the clone. They cannot observe a
+# change anywhere else in the LocalAI tree. Building them below `COPY . /LocalAI`
+# threw that away: any Go-side edit invalidated the layer and recompiled C++ that
+# had not changed. Measured on 2026-07-30, that is a 100+ minute rebuild for the
+# larger engines.
+#
+# Copying only this backend's directory first keeps the compile in a layer that
+# survives any change elsewhere in the tree, so `cache-from: type=registry`
+# restores it. That covers the expensive cases directly: a shared-build-input or
+# backend.proto change, the weekly full-matrix cron and a tag push all rebuild
+# every backend while touching none of their directories. This is the mechanism
+# behind base-grpc-* applied one level down; unlike a --mount=type=cache it is a
+# real layer, which is what actually survives to the registry.
+#
+# The whole directory rather than just the Makefile: the CMake targets also need
+# CMakeLists.txt, and the file list differs per backend. The cost is that editing
+# this backend's Go sources also invalidates the engine layer.
+#
+# Backends whose Makefile has no `engine` target are unaffected: the guard skips
+# the prebuild and their engine still compiles in the `build` step below.
+COPY backend/go/${BACKEND}/ /LocalAI/backend/go/${BACKEND}/
+RUN cd /LocalAI/backend/go/${BACKEND} && \
+    if make -n engine >/dev/null 2>&1; then \
+        echo "==> prebuilding engine for ${BACKEND} (cacheable layer)" && \
+        make engine; \
+    else \
+        echo "==> ${BACKEND} has no engine target; it builds with the backend"; \
+    fi
+
+COPY . /LocalAI
+
+# The engine variants built above survive this COPY (they are build outputs, not
+# tracked files) and are newer than the pinned clone, so make treats them as up
+# to date and goes straight to the Go binary.
 RUN cd /LocalAI && make protogen-go && make -C /LocalAI/backend/go/${BACKEND} build
 
 FROM scratch

backend/Dockerfile.llama-cpp

@@ -111,6 +111,10 @@ RUN make -BC /LocalAI/backend/cpp/llama-cpp package
 # ============================================================================
 FROM ${BUILDER_BASE_IMAGE} AS builder-prebuilt
 
+ARG APT_MIRROR
+ENV APT_MIRROR=${APT_MIRROR}
+ARG APT_PORTS_MIRROR
+ENV APT_PORTS_MIRROR=${APT_PORTS_MIRROR}
 ARG BUILD_TYPE
 ENV BUILD_TYPE=${BUILD_TYPE}
 ARG CUDA_DOCKER_ARCH

backend/README.md

@@ -56,6 +56,7 @@ The backend system provides language-specific Dockerfiles that handle the build
 - **stablediffusion-ggml**: Stable Diffusion in Go with GGML Cpp backend
 - **piper**: Text-to-speech synthesis Golang with C bindings using rhaspy/piper
 - **local-store**: Vector storage backend
+- **valkey-store**: Durable vector storage backend backed by Valkey Search (FT.*)
 
 #### C++ Backends (`cpp/`)
 - **llama-cpp**: Llama.cpp integration

backend/backend.proto

@@ -16,6 +16,7 @@ service Backend {
   rpc Embedding(PredictOptions) returns (EmbeddingResult) {}
   rpc GenerateImage(GenerateImageRequest) returns (Result) {}
   rpc GenerateVideo(GenerateVideoRequest) returns (Result) {}
+  rpc Generate3D(Generate3DRequest) returns (Result) {}
   rpc AudioTranscription(TranscriptRequest) returns (TranscriptResult) {}
   rpc AudioTranscriptionStream(TranscriptRequest) returns (stream TranscriptStreamResponse) {}
   // AudioTranscriptionLive is the bidirectional live-microphone ASR RPC. The
@@ -34,6 +35,7 @@ service Backend {
   rpc TTSStream(TTSRequest) returns (stream Reply) {}
   rpc SoundGeneration(SoundGenerationRequest) returns (Result) {}
   rpc TokenizeString(PredictOptions) returns (TokenizationResponse) {}
+  rpc Detokenize(DetokenizeRequest) returns (DetokenizeResponse) {}
   rpc Status(HealthMessage) returns (StatusResponse) {}
   rpc Detect(DetectOptions) returns (DetectResponse) {}
   // SoundDetection runs an audio-tagging / sound-event-classification model
@@ -181,6 +183,13 @@ message ScoreRequest {
   // PredictOptions.ModelIdentity for the full rationale. Empty means "no
   // identity supplied" and backends MUST skip the check.
   string ModelIdentity = 5;
+  // Byte length of the prompt prefix that stays identical across
+  // repeated scoring calls (e.g. a classifier's option-list system
+  // prompt — everything before the per-turn probe text). Backends that
+  // snapshot state (hybrid/recurrent models cannot rewind otherwise)
+  // use it to place a reuse point exactly at the boundary, so the next
+  // call re-processes only the tokens after it. 0 means unknown.
+  int32 stable_prefix_len = 6;
 }
 
 // CandidateScore is one row in the ScoreResponse, matching by index
@@ -493,6 +502,11 @@ message ModelOptions {
   // Proxy carries the cloud-proxy backend's per-model configuration.
   // Empty for non-proxy backends.
   ProxyOptions Proxy = 74;
+
+  // EnableScore reserves backend resources for the Score RPC. It is derived
+  // from the model's explicit `known_usecases: [score]` declaration so models
+  // that never score retain their ordinary serving footprint.
+  bool EnableScore = 75;
 }
 
 // ProxyOptions configures the cloud-proxy backend. UpstreamURL and
@@ -508,6 +522,12 @@ message ProxyOptions {
   string api_key_file = 5;
   string upstream_model = 6;
   int32 request_timeout_seconds = 7;
+  // cache_prompt enables automatic Anthropic prompt-cache breakpoints
+  // (cache_control: ephemeral) on the stable prefix — system, tools, and
+  // the last message block — when translating to the Anthropic provider.
+  // Cuts input cost on repeated/agentic calls (cache read = 0.1x). Only
+  // meaningful for mode=translate + provider=anthropic; ignored otherwise.
+  bool cache_prompt = 8;
 }
 
 message Result {
@@ -640,6 +660,20 @@ message GenerateVideoRequest {
   string ModelIdentity = 15;
 }
 
+message Generate3DRequest {
+  string src = 1;          // Path to the staged conditioning image (3D generation is image-conditioned)
+  string dst = 2;          // Output path for the generated binary glTF (.glb) asset
+  int32 seed = 3;          // <=0 lets the backend pick a random seed
+  int32 step = 4;          // Flow sampling steps; <=0 uses the backend default
+  float cfg_scale = 5;     // Classifier-free guidance scale; <=0 uses the backend default
+  int32 texture_steps = 6; // Texture flow sampling steps; <=0 uses the backend default
+  string quality = 7;      // Mesh pipeline: ""|"auto"|"coarse"|"512"|"1024"
+  string background = 8;   // Conditioning-image background handling: ""|"auto"|"keep"|"black"|"white"
+  // Backend-specific per-request generation parameters. Values are strings
+  // and are validated/coerced by the selected backend.
+  map<string, string> params = 9;
+}
+
 message TTSRequest {
   string text = 1;
   string model = 2;
@@ -763,6 +797,14 @@ message TokenizationResponse {
   repeated int32 tokens = 2;
 }
 
+message DetokenizeRequest {
+  repeated int32 tokens = 1;
+}
+
+message DetokenizeResponse {
+  string content = 1;
+}
+
 message MemoryUsageData {
   uint64 total = 1;
   map<string, uint64> breakdown = 2;
@@ -1089,11 +1131,28 @@ message AudioTransformRequest {
   string ModelIdentity = 5;
 }
 
+// One named output of a transform that produces several from a single run.
+// Source separation is the case that needs it: htdemucs yields drums, bass,
+// other and vocals from one pass over the input.
+message AudioTransformStem {
+  string name = 1;                   // the model's own stem id, e.g. "vocals"
+  string dst = 2;                    // path of the file written for that stem
+}
+
 message AudioTransformResult {
   string dst = 1;
   int32  sample_rate = 2;
   int32  samples = 3;
   bool   reference_provided = 4;
+  // Every named output the run produced, in the model's own order, including
+  // the one copied into dst. Empty for a transform with a single output.
+  //
+  // It exists because dst carries one file while separation produces several,
+  // and running the model once per stem would cost four full separations of
+  // the same audio. The backend runs once, writes each stem beside dst, and
+  // names them here; without this field the other stems are on disk but no
+  // caller can find them, which is the same as not having produced them.
+  repeated AudioTransformStem stems = 5;
 }
 
 // Bidirectional streaming audio transform. The first message MUST carry a

backend/cpp/audio-cpp/.gitignore

@@ -0,0 +1,8 @@
+audio.cpp/
+build/
+package/
+grpc-server
+backend.pb.cc
+backend.pb.h
+backend.grpc.pb.cc
+backend.grpc.pb.h

backend/cpp/audio-cpp/CMakeLists.txt

@@ -0,0 +1,331 @@
+cmake_minimum_required(VERSION 3.20)
+project(audio-cpp-grpc-server LANGUAGES C CXX)
+
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(TARGET grpc-server)
+
+set(AUDIO_CPP_DIR "${CMAKE_CURRENT_SOURCE_DIR}/audio.cpp"
+    CACHE PATH "Path to the pinned audio.cpp checkout")
+option(AUDIO_CPP_GRPC_BUILD_TESTS "Build engine-linked ctest binaries" OFF)
+
+if(NOT EXISTS "${AUDIO_CPP_DIR}/CMakeLists.txt")
+    message(FATAL_ERROR
+        "AUDIO_CPP_DIR does not contain an audio.cpp checkout: ${AUDIO_CPP_DIR}. "
+        "Run 'make audio.cpp' first.")
+endif()
+
+if(APPLE)
+    # Homebrew installs protobuf/grpc under a non-default prefix.
+    if(CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "arm64")
+        set(HOMEBREW_DEFAULT_PREFIX "/opt/homebrew")
+    else()
+        set(HOMEBREW_DEFAULT_PREFIX "/usr/local")
+    endif()
+    link_directories("${HOMEBREW_DEFAULT_PREFIX}/lib")
+    include_directories("${HOMEBREW_DEFAULT_PREFIX}/include")
+endif()
+
+find_package(Threads REQUIRED)
+find_package(Protobuf CONFIG QUIET)
+if(NOT Protobuf_FOUND)
+    find_package(Protobuf REQUIRED)
+endif()
+find_package(gRPC CONFIG QUIET)
+if(NOT gRPC_FOUND)
+    # Reached only on distros whose grpc++ packaging ships no CMake config.
+    # Ubuntu's libgrpc-dev does ship one, so this is dead code on LocalAI's own
+    # build distro. Kept for the distros that do not.
+    find_library(GRPCPP_LIB grpc++ REQUIRED)
+    find_library(GRPCPP_REFLECTION_LIB grpc++_reflection REQUIRED)
+    add_library(gRPC::grpc++ INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++ PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_LIB}")
+    add_library(gRPC::grpc++_reflection INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++_reflection PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_REFLECTION_LIB}")
+endif()
+
+find_program(_PROTOC NAMES protoc REQUIRED)
+find_program(_GRPC_CPP_PLUGIN NAMES grpc_cpp_plugin REQUIRED)
+
+get_filename_component(HW_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/../../backend.proto" ABSOLUTE)
+get_filename_component(HW_PROTO_PATH "${HW_PROTO}" PATH)
+
+set(HW_PROTO_SRCS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.cc")
+set(HW_PROTO_HDRS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.h")
+set(HW_GRPC_SRCS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.cc")
+set(HW_GRPC_HDRS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.h")
+
+add_custom_command(
+    OUTPUT "${HW_PROTO_SRCS}" "${HW_PROTO_HDRS}" "${HW_GRPC_SRCS}" "${HW_GRPC_HDRS}"
+    COMMAND ${_PROTOC}
+    ARGS --grpc_out "${CMAKE_CURRENT_BINARY_DIR}"
+         --cpp_out  "${CMAKE_CURRENT_BINARY_DIR}"
+         -I "${HW_PROTO_PATH}"
+         --plugin=protoc-gen-grpc="${_GRPC_CPP_PLUGIN}"
+         "${HW_PROTO}"
+    DEPENDS "${HW_PROTO}")
+
+add_library(hw_grpc_proto STATIC
+    ${HW_GRPC_SRCS} ${HW_GRPC_HDRS}
+    ${HW_PROTO_SRCS} ${HW_PROTO_HDRS})
+target_include_directories(hw_grpc_proto PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
+# Required on macOS: without these the Homebrew protobuf/grpc include dirs never
+# reach this target and google/protobuf/runtime_version.h is not found.
+target_link_libraries(hw_grpc_proto PUBLIC protobuf::libprotobuf gRPC::grpc++)
+
+# TWO PROTOBUF RUNTIMES IN ONE BINARY, AND THE ONE THAT WON WAS THE WRONG ONE.
+#
+# engine_runtime links sentencepiece, whose default SPM_PROTOBUF_PROVIDER
+# ("internal") builds the protobuf-lite 3.14.0 sources vendored under
+# external/sentencepiece/third_party/protobuf-lite. Our generated backend.pb.cc
+# is compiled against the toolchain's protobuf 3.21.12 headers and links
+# libprotobuf.so 3.21.12. Both used to end up in the executable: 476
+# google::protobuf:: symbols from that archive, 278 of them also defined by
+# libprotobuf.so.
+#
+# The binding is decided at STATIC LINK time. Once ld pulls a sentencepiece
+# member in for sentencepiece's own code, that member's protobuf definitions are
+# in the executable and references from libhw_grpc_proto.a bind to them. Do NOT
+# reach for -Wl,--exclude-libs: it flips those symbols to LOCAL in .dynsym and
+# the breakage is unchanged, because no visibility flag revisits a static
+# binding already made.
+#
+# What broke, measured rather than assumed:
+# google::protobuf::internal::ParseContext::ParseMessage(MessageLite*, const char*)
+# is what every generated _InternalParse calls for a SUBMESSAGE field and for
+# nothing else. Bound to the 3.14 definition it fails, so a flat message parsed
+# and every nested one did not: a TranscriptResult carrying segments serialized
+# to correct bytes that the same process could not read back, and
+# TranscriptLiveRequest, a oneof of submessages, could not have been parsed at
+# all. 3.21 generated code was also running 3.14 arena, ArenaStringPtr and
+# ExtensionSet code, which is an ABI mismatch rather than a missing feature, so
+# "not observed to bite yet" was never a reason to leave it.
+#
+# "package" makes sentencepiece use the protobuf found above, which is the one
+# the generated code was built against. It must be set before add_subdirectory,
+# since that is when sentencepiece's own cache entry is created.
+#
+# WHAT THAT BUYS IS ONE PROTOBUF RUNTIME, not an executable free of protobuf
+# symbols, and the difference matters to whoever checks this next. Measured with
+# nm -C --defined-only on the linked grpc-server, 2515 google::protobuf::
+# symbols are still DEFINED in it, and that is what should be there: they are
+# generated code, sentencepiece::ModelProto's own _InternalParse and
+# CheckTypeAndMergeFrom among them, which name protobuf types in their
+# signatures and are compiled into every user of a .proto. Expecting zero would
+# send a reader looking for a regression that is not one.
+#
+# The claim that decides whether the ABI mismatch above is gone is the RUNTIME
+# one, and it holds: google::protobuf::internal::ParseContext::ParseMessage is
+# UNDEFINED in the executable, so every generated _InternalParse resolves it to
+# libprotobuf.so at load instead of to a vendored 3.14 copy. No vendored
+# protobuf-lite archive is pulled in at all, and citrinet_asr, which parses a
+# SentencePiece ModelProto at load, tokenizes correctly as a result.
+set(SPM_PROTOBUF_PROVIDER "package" CACHE STRING
+    "Make sentencepiece use the found protobuf, not its vendored 3.14 copy" FORCE)
+
+# Upstream's global add_compile_options(-Wall -Wextra -Wpedantic -pedantic-errors)
+# is a directory property of the subdirectory and does not reach our targets.
+#
+# EXCLUDE_FROM_ALL is load-bearing, do not drop it: upstream's default target set
+# includes its CLI, server, converter and test binaries, none of which we ship.
+# Without it every build would compile all of them. The targets we do name in
+# target_link_libraries below are still built on demand, so nothing is lost.
+add_subdirectory("${AUDIO_CPP_DIR}" "${CMAKE_CURRENT_BINARY_DIR}/audio-cpp" EXCLUDE_FROM_ALL)
+
+add_executable(${TARGET}
+    grpc-server.cpp
+    model_options.cpp
+    capability_routing.cpp
+    family_gate.cpp
+    loaded_model.cpp
+    audio_io.cpp
+    audio_units.cpp
+    transcript_assembly.cpp
+    result_map.cpp
+    stem_selection.cpp
+    generation_request.cpp
+    stream_delta.cpp
+    wav_header.cpp
+    inference_lane.cpp
+    live_watchdog.cpp
+)
+
+# Two files carry a switch over an enum with no `default:` label, deliberately,
+# so that -Wswitch reports an enumerator nobody handled. -Wswitch is only a
+# warning by default, and a warning in a 600-file build log is a warning nobody
+# reads, so it is promoted to an error on exactly these two translation units.
+# Not project-wide: upstream's own sources are not held to this, and they are
+# where the churn is.
+#
+# loaded_model.cpp mirrors engine::runtime::VoiceTaskKind onto its own Task enum.
+# Its static_asserts catch an insertion or a reorder, but an enumerator APPENDED
+# after the last one shifts no value, so no assertion can see it. What does see
+# it is from_engine_task's switch over the engine enum. This is the difference
+# between a build failure and a backend that silently runs the wrong task.
+#
+# capability_routing.cpp's unsupported_surface() switches UnsupportedRpc onto the
+# row of unsupported_surfaces() that explains it. Left as a warning, a sixth
+# enumerator added without a row BUILDS AND SHIPS, and its trailing
+# `return surfaces[0];` then answers the new RPC with AudioEncode's codec reason:
+# a confident, specific and false statement about audio.cpp, on the wire, on the
+# one code path whose entire job is to be truthful about what this backend
+# cannot do. Verified rather than assumed: adding a sixth enumerator and building
+# the shipping target produced exit 0, a binary, and one warning. A compile-time
+# check is the better trade than the runtime fallback it replaced only if it is
+# fatal, so here it is fatal.
+if(NOT MSVC)
+    set_source_files_properties(loaded_model.cpp capability_routing.cpp
+        PROPERTIES COMPILE_OPTIONS "-Werror=switch")
+endif()
+
+target_include_directories(${TARGET} PRIVATE
+    "${AUDIO_CPP_DIR}/include"
+    "${CMAKE_CURRENT_SOURCE_DIR}")
+
+# The shipping binary is held to the same bar as the tests below. Upstream's own
+# add_compile_options is a property of its directory and never reached this
+# target, so until now "the build was clean" meant only that nothing was being
+# checked.
+if(NOT MSVC)
+    target_compile_options(${TARGET} PRIVATE -Wall -Wextra -Wpedantic)
+endif()
+
+target_link_libraries(${TARGET} PRIVATE
+    hw_grpc_proto
+    engine_runtime
+    ggml
+    gRPC::grpc++
+    gRPC::grpc++_reflection
+    protobuf::libprotobuf
+    Threads::Threads)
+
+# ENGINE_ENABLE_CPU_ALL_VARIANTS builds ggml backends as shared objects that sit
+# next to the binary in the package, so the binary must search its own directory.
+# BUILD_WITH_INSTALL_RPATH keeps the build-tree binary at exactly "$ORIGIN".
+# Upstream sets CMAKE_BUILD_WITH_INSTALL_RPATH in its own directory scope, which
+# does not reach ours, so without this CMake also appends its build-tree library
+# directory. That absolute build-host path would survive into the copied binary
+# and let a package.sh that forgot to bundle libggml*.so still pass on the build
+# machine while failing everywhere else.
+set_target_properties(${TARGET} PROPERTIES
+    BUILD_RPATH "$ORIGIN"
+    INSTALL_RPATH "$ORIGIN"
+    BUILD_WITH_INSTALL_RPATH TRUE)
+
+if(AUDIO_CPP_GRPC_BUILD_TESTS)
+    enable_testing()
+
+    # These are the units whose tests CANNOT run under
+    # backend/cpp/run-unit-tests.sh, because that script compiles each
+    # *_test.cpp standalone with no protobuf and no audio.cpp include path.
+    # They are named *_ctest.cpp so the script's glob does not pick them up and
+    # fail every backend's suite; everything that can be stdlib-only still is,
+    # and still lives in a *_test.cpp beside its unit.
+    add_executable(result_map_ctest
+        result_map_ctest.cpp
+        result_map.cpp
+        transcript_assembly.cpp
+        audio_units.cpp)
+    target_include_directories(result_map_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h. Every other target
+        # here inherits that directory from the ggml target it links; this one
+        # links no ggml, so it has to name it.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    # No engine_runtime: result_map touches only the plain structs in
+    # engine/framework/runtime/session.h, so the header is all it needs.
+    target_link_libraries(result_map_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(result_map_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME result_map COMMAND result_map_ctest)
+
+    # Same shape as result_map_ctest: generation_request touches only the plain
+    # structs in engine/framework/runtime/session.h plus the generated protobuf
+    # messages, so the headers are all it needs and no engine_runtime is linked.
+    add_executable(generation_request_ctest
+        generation_request_ctest.cpp
+        generation_request.cpp)
+    target_include_directories(generation_request_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h, and this target links
+        # no ggml, so it has to name the include directory itself.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    target_link_libraries(generation_request_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(generation_request_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME generation_request COMMAND generation_request_ctest)
+
+    add_executable(audio_io_ctest
+        audio_io_ctest.cpp
+        audio_io.cpp)
+    target_include_directories(audio_io_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(audio_io_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(audio_io_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    set_target_properties(audio_io_ctest PROPERTIES
+        BUILD_RPATH "$ORIGIN"
+        INSTALL_RPATH "$ORIGIN"
+        BUILD_WITH_INSTALL_RPATH TRUE)
+    add_test(NAME audio_io COMMAND audio_io_ctest)
+
+    # The streaming drivers live in loaded_model.cpp, which links the engine, so
+    # this cannot be a standalone *_test.cpp. It builds no model and reads no
+    # file: LoadedModel::Session is a plain struct holding a pointer to an
+    # engine interface, so the drivers are exercised against fake sessions.
+    add_executable(streaming_driver_ctest
+        streaming_driver_ctest.cpp
+        loaded_model.cpp
+        capability_routing.cpp
+        family_gate.cpp
+        model_options.cpp
+        inference_lane.cpp)
+    target_include_directories(streaming_driver_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(streaming_driver_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(streaming_driver_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    # No "$ORIGIN" rpath override here, unlike the shipping target and unlike
+    # audio_io_ctest. loaded_model.cpp reaches make_default_registry, so this
+    # binary genuinely links libggml, and CMake's own build-tree rpath is what
+    # finds it: the ggml shared objects land in ${CMAKE_CURRENT_BINARY_DIR}/bin
+    # while the test binary sits one directory up. A build-host absolute path in
+    # a test binary is harmless, since package.sh ships only grpc-server, and
+    # forcing "$ORIGIN" here means ctest cannot start the binary at all.
+    add_test(NAME streaming_driver COMMAND streaming_driver_ctest)
+
+    # Asserts that the upstream ABSENCES capability_routing.cpp's refusal
+    # messages rest on are still absences, by querying make_default_registry()
+    # rather than by re-reading upstream. This is what makes an AUDIO_CPP_VERSION
+    # bump that adds a codec task kind, an spk family or a streaming converter
+    # fail the build instead of leaving a false statement on the wire.
+    #
+    # It links engine_runtime purely to run that query, which is why it lives
+    # here rather than with the standalone *_test.cpp files, and it needs no
+    # "$ORIGIN" rpath override for the same reason streaming_driver_ctest does
+    # not: see the note above.
+    add_executable(upstream_absence_ctest upstream_absence_ctest.cpp)
+    target_include_directories(upstream_absence_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(upstream_absence_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(upstream_absence_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME upstream_absence COMMAND upstream_absence_ctest)
+endif()

backend/cpp/audio-cpp/Makefile

@@ -0,0 +1,172 @@
+# audio.cpp backend Makefile.
+#
+# Upstream pin lives below in the AUDIO_CPP_VERSION variable, so
+# .github/bump_deps.sh can find and update it, matching the llama-cpp / ds4
+# convention. That script seds every line matching the variable name followed by
+# an assignment, so this comment deliberately spells the name on its own: a
+# comment repeating the full assignment token gets rewritten and mangled by the
+# first auto-bump (backend/cpp/ds4/Makefile shows the damage). The clone
+# recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
+# rebuild and so the bump bot can see the pin.
+
+AUDIO_CPP_VERSION?=f32876cfb45732dd4f43264e9104d229e95b0bc3
+AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
+
+CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+BUILD_DIR := build
+
+BUILD_TYPE ?=
+NATIVE ?= false
+JOBS ?= $(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
+
+UNAME_S := $(shell uname -s)
+
+# AUDIOCPP_DEPLOYMENT_BUILD compiles the model_specs/*.json catalog into
+# engine_runtime, so the shipped package needs no model_specs directory and a
+# safetensors model tree still resolves its family spec.
+CMAKE_ARGS ?= -DCMAKE_BUILD_TYPE=Release -DAUDIOCPP_DEPLOYMENT_BUILD=ON
+
+# CMAKE_CUDA_ARCHITECTURES must be set explicitly for a cublas build, and this
+# is not a tuning knob: upstream's CMakeLists sets CUDA_ARCHITECTURES to
+# `native` on the engine_runtime target whenever the root-scope variable is
+# unset (audio.cpp/CMakeLists.txt, the `if (CMAKE_CUDA_ARCHITECTURES)` branch
+# next to the istft/torch_random .cu sources), and docs/build/linux.md says so
+# outright: "Leave CMAKE_CUDA_ARCHITECTURES unset to build for the GPUs present
+# at build time (native)". No CI runner has a GPU, so `native` has nothing to
+# enumerate. ggml's own default (external/ggml/src/ggml-cuda/CMakeLists.txt)
+# does not rescue this: it list(APPEND)s in the ggml subdirectory scope, which
+# never reaches the root scope where the engine_runtime property is decided.
+#
+# The values below are ggml's list for the matching toolkit, copied rather than
+# invented, so the two targets compile for exactly the same set:
+#   - CUDA 13 drops the Maxwell/Pascal/Volta virtual archs (50/61/70).
+#   - 121a-real needs CUDA >= 12.9, so the CUDA 12 list (built against 12.8)
+#     stops at 120a-real.
+#   - `a`-suffixed archs are used rather than ggml's rejected 120f-virtual: the
+#     `f` suffix needs CMake >= 3.31.8, and Ubuntu Noble ships 3.28.3. The
+#     3.28 validator (Modules/Internal/CMakeCUDAArchitecturesValidate.cmake)
+#     accepts `[0-9]+a?(-real|-virtual)?`.
+#
+# Setting it here also pins ggml's copy, since its default is guarded by
+# `if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES)`. CUDA_MAJOR_VERSION is the CI
+# build-arg, forwarded by Dockerfile.audio-cpp.
+#
+# An EMPTY major maps to `native`, NOT to the CUDA 12 list. Only CI declares a
+# major; a developer running `BUILD_TYPE=cublas make` locally declares none, and
+# the CUDA 12 list contains 120a-real, which needs nvcc >= 12.8. Falling through
+# to it turned every local build on a CUDA 12.0-12.7 host into a compile error,
+# where upstream's documented behaviour ("Leave CMAKE_CUDA_ARCHITECTURES unset
+# to build for the GPUs present at build time") worked. `native` restores that.
+# It does require a GPU to enumerate, so the escape hatch for a GPU-less local
+# cross-build is to set CUDA_ARCHITECTURES on the command line, which the ?=
+# assignments below leave untouched.
+CUDA_MAJOR_VERSION ?=
+ifeq ($(CUDA_MAJOR_VERSION),13)
+    CUDA_ARCHITECTURES ?= 75-virtual;80-virtual;86-real;89-real;120a-real;121a-real
+else ifeq ($(CUDA_MAJOR_VERSION),12)
+    CUDA_ARCHITECTURES ?= 50-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-real;89-real;120a-real
+else ifeq ($(CUDA_MAJOR_VERSION),)
+    CUDA_ARCHITECTURES ?= native
+else ifeq ($(BUILD_TYPE),cublas)
+    # Gated on cublas because the variable means nothing to any other build, so a
+    # stray CUDA_MAJOR_VERSION in the environment must not break `make clean` or
+    # a CPU build. It does still error for `BUILD_TYPE=cublas make clean`, which
+    # is the right trade: that invocation is asking about a CUDA build tree.
+    $(error CUDA_MAJOR_VERSION=$(CUDA_MAJOR_VERSION) has no architecture list here (12 and 13 do). Leave it empty for a native build, or pass CUDA_ARCHITECTURES explicitly.)
+endif
+
+ifeq ($(BUILD_TYPE),cublas)
+    CMAKE_ARGS += -DENGINE_ENABLE_CUDA=ON "-DCMAKE_CUDA_ARCHITECTURES=$(CUDA_ARCHITECTURES)"
+else ifeq ($(BUILD_TYPE),vulkan)
+    CMAKE_ARGS += -DENGINE_ENABLE_VULKAN=ON
+else ifeq ($(UNAME_S),Darwin)
+    # Metal. ggml embeds the shader library by default (GGML_METAL_EMBED_LIBRARY
+    # defaults to GGML_METAL), so the package needs no .metallib beside the
+    # binary. Darwin builds go through scripts/build/audio-cpp-darwin.sh.
+    CMAKE_ARGS += -DENGINE_ENABLE_METAL=ON
+    # AppleClang ships no OpenMP runtime and Homebrew's libomp is keg-only, so
+    # neither libomp.dylib nor omp.h is symlinked into /opt/homebrew and CMake's
+    # FindOpenMP cannot find them on its own (the workflow's `brew link libomp`
+    # is a no-op for a keg-only formula, and its failure is swallowed).
+    # audio.cpp calls find_package(OpenMP REQUIRED COMPONENTS CXX) whenever
+    # ENGINE_ENABLE_OPENMP is ON, so with no hint the macOS build dies at
+    # configure time before compiling anything. OpenMP_ROOT is honoured by the
+    # find_library/find_path calls inside FindOpenMP under CMP0074, which is NEW
+    # here because audio.cpp requires CMake 3.20.
+    #
+    # If the keg is absent, turn OpenMP off rather than fail: the tree's only
+    # <omp.h> include is guarded by #ifdef _OPENMP and a #pragma omp without
+    # -fopenmp is simply ignored, so an OpenMP-less build is CORRECT. It is not
+    # cheap, though: 108 `#pragma omp` directives across ~30 files (roformer,
+    # demucs, chatterbox, moss, supertonic, seed_vc, framework/audio/dsp) are
+    # compiled out, and clang says nothing about an ignored omp pragma unless
+    # -Wsource-uses-openmp is on. A green package that is quietly single-threaded
+    # in every host DSP loop gets blamed on Metal, not on packaging, so the
+    # fallback announces itself.
+    ifeq ($(origin LIBOMP_PREFIX),undefined)
+        LIBOMP_PREFIX := $(shell brew --prefix libomp 2>/dev/null)
+    endif
+    # Nested ifneq rather than $(and ...): $(and) needs GNU make 3.81, and while
+    # that is what Apple ships, an older make expands it to empty and would take
+    # the OpenMP-OFF branch with no way to tell that from a genuinely missing
+    # keg. Two plain conditionals cannot fail that way.
+    LIBOMP_USABLE :=
+    ifneq ($(wildcard $(LIBOMP_PREFIX)/lib/libomp.dylib),)
+        ifneq ($(wildcard $(LIBOMP_PREFIX)/include/omp.h),)
+            LIBOMP_USABLE := yes
+        endif
+    endif
+    ifeq ($(LIBOMP_USABLE),yes)
+        CMAKE_ARGS += "-DOpenMP_ROOT=$(LIBOMP_PREFIX)"
+    else
+        $(warning audio-cpp: libomp not found at '$(LIBOMP_PREFIX)'; building without OpenMP (single-threaded host DSP). Install it with `brew install libomp`, or set LIBOMP_PREFIX.)
+        CMAKE_ARGS += -DENGINE_ENABLE_OPENMP=OFF
+    endif
+else
+    # Portable Linux CPU. Upstream wires this to GGML_BACKEND_DL +
+    # GGML_CPU_ALL_VARIANTS + $ORIGIN rpath, so one build serves every CPU
+    # tier instead of an AVX-tier image fan-out.
+    CMAKE_ARGS += -DENGINE_ENABLE_CPU_ALL_VARIANTS=ON
+endif
+
+ifneq ($(NATIVE),true)
+    CMAKE_ARGS += -DENGINE_ENABLE_NATIVE_CPU=OFF
+endif
+
+.PHONY: all grpc-server package test test-engine clean purge
+all: grpc-server
+
+# Clone the upstream source at the pinned commit. The directory is the target
+# so make only re-clones when it is missing. After bumping AUDIO_CPP_VERSION,
+# run 'make purge && make' to refetch.
+audio.cpp:
+	mkdir -p audio.cpp
+	cd audio.cpp && \
+	git init -q && \
+	git remote add origin $(AUDIO_CPP_REPO) && \
+	git fetch --depth 1 origin $(AUDIO_CPP_VERSION) && \
+	git checkout FETCH_HEAD
+
+grpc-server: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS)
+	cp $(BUILD_DIR)/grpc-server grpc-server
+
+package: grpc-server
+	bash package.sh
+
+test:
+	@echo "audio-cpp: standalone unit tests run from the repo root via 'make test-backend-cpp'"
+
+# Engine-linked tests. Needs the upstream checkout and a full engine build.
+test-engine: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) -DAUDIO_CPP_GRPC_BUILD_TESTS=ON $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS) && ctest --output-on-failure --no-tests=error
+
+clean:
+	rm -rf $(BUILD_DIR) grpc-server package
+
+purge: clean
+	rm -rf audio.cpp

backend/cpp/audio-cpp/audio_io.cpp

@@ -0,0 +1,124 @@
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include "engine/framework/audio/conversion.h"
+#include "engine/framework/audio/wav_reader.h"
+#include "engine/framework/audio/wav_writer.h"
+
+#include <filesystem>
+#include <string>
+#include <utility>
+
+namespace audiocpp_backend {
+
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no input audio path was supplied");
+    }
+    std::error_code ec;
+    const bool present = std::filesystem::exists(std::filesystem::path(path), ec);
+    if (ec) {
+        // exists() returning false with ec set does NOT mean the file is
+        // absent, it means the question could not be answered: most often a
+        // parent directory is not searchable. Reporting that as "does not
+        // exist" sends the operator after the file when the fault is the
+        // permissions on the directory above it.
+        throw ConfigError("audio-cpp: cannot stat input audio " + path + ": " +
+                          ec.message());
+    }
+    if (!present) {
+        throw ConfigError("audio-cpp: input audio does not exist: " + path);
+    }
+    engine::audio::WavData wav;
+    try {
+        wav = engine::audio::read_wav_f32(std::filesystem::path(path));
+    } catch (const std::exception &err) {
+        throw ConfigError("audio-cpp: cannot read " + path +
+                          " as WAV: " + err.what());
+    }
+    if (wav.sample_rate <= 0) {
+        throw ConfigError("audio-cpp: " + path +
+                          " declares a non-positive sample rate; every "
+                          "timestamp derived from it would be zero");
+    }
+    // AudioBuffer's own default is 1, and a reader that reports 0 channels
+    // still gave us an interleaving of one. Normalised before the conversion
+    // below rather than after, because mixdown_interleaved_to_mono_average
+    // throws on a non-positive channel count.
+    if (wav.channels <= 0) {
+        wav.channels = 1;
+    }
+
+    engine::runtime::AudioBuffer buffer;
+    if (target_sample_rate <= 0) {
+        buffer.sample_rate = wav.sample_rate;
+        buffer.channels = wav.channels;
+        buffer.samples = std::move(wav.samples);
+        return buffer;
+    }
+
+    buffer.sample_rate = target_sample_rate;
+    buffer.channels = 1;
+    try {
+        // A no-op copy when the rates already match, so the common 16 kHz
+        // upload pays only the mono mixdown it would have paid inside the
+        // family anyway.
+        buffer.samples =
+            engine::audio::convert_wav_to_mono_linear_resampled(wav, target_sample_rate);
+    } catch (const std::exception &err) {
+        // ConfigError, so this is INVALID_ARGUMENT rather than INTERNAL. What
+        // reaches here is a malformed input: a sample count that is not a whole
+        // number of frames is the realistic one, and it is the uploader's file
+        // that is truncated, not this backend that is broken.
+        throw ConfigError("audio-cpp: cannot resample " + path + " from " +
+                          std::to_string(wav.sample_rate) + " Hz to " +
+                          std::to_string(target_sample_rate) +
+                          " Hz: " + err.what());
+    }
+    return buffer;
+}
+
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no output path was supplied");
+    }
+    const std::filesystem::path destination(path);
+    if (destination.has_parent_path()) {
+        // Best effort: a failure here shows up as a write failure below, with a
+        // message naming the file the caller actually asked for.
+        std::error_code ec;
+        std::filesystem::create_directories(destination.parent_path(), ec);
+    }
+    try {
+        engine::audio::write_pcm16_wav(destination, audio.sample_rate,
+                                       audio.channels > 0 ? audio.channels : 1,
+                                       audio.samples);
+    } catch (const std::exception &err) {
+        // NOT a ConfigError, and the distinction is not cosmetic. The
+        // destination is chosen by LocalAI rather than by the caller: it is a
+        // unique name inside GeneratedContentDir. A failure to write it is a
+        // full disk, a permission fault on the server's own directory, or a bad
+        // mount, none of which the caller can fix or is to blame for. As a
+        // ConfigError this surfaced as INVALID_ARGUMENT, which tells a client
+        // its request was wrong and not to retry; a plain runtime_error maps to
+        // INTERNAL, which is both true and retryable. The empty path above
+        // stays INVALID_ARGUMENT, because that one really is a malformed
+        // request.
+        throw std::runtime_error("audio-cpp: cannot write " + path + ": " +
+                                 err.what());
+    }
+}
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = 1;
+    buffer.samples = std::move(samples);
+    return buffer;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io.h

@@ -0,0 +1,61 @@
+#pragma once
+
+// Thin wrappers over the framework's public audio IO. Engine-linked, so this
+// unit is built and tested through the CMake target rather than by
+// backend/cpp/run-unit-tests.sh. The pure part of the arithmetic these
+// wrappers feed lives in audio_units, which is stdlib-only and does have a
+// standalone test.
+
+#include "engine/framework/runtime/session.h"
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Reads a WAV file. Throws ConfigError when the file is missing, is not
+// readable as WAV, or declares a non-positive sample rate: all three are
+// user-fixable input problems rather than backend faults.
+//
+// A declared sample rate of zero is refused rather than passed on, because
+// every downstream conversion in audio_units answers 0 for a non-positive rate.
+// Accepting it would turn a corrupt header into a response full of zero
+// timestamps, which reads as a real answer.
+//
+// `target_sample_rate` is the rate the CALLER needs, in Hz:
+//
+//   0 (or negative)  keep the file's own rate and channel count.
+//   positive         downmix to mono and resample to that rate. Resampling is
+//                    skipped when the file already declares it, so passing the
+//                    rate a route needs costs nothing on the common input.
+//
+// It is a parameter, and not a constant inside this function, because the
+// routes that read audio do not agree on an answer. Speech routes want 16 kHz
+// mono; source separation does not, and folding a 44.1 kHz stereo input to
+// 16 kHz mono for demucs or roformer would destroy the very thing they separate
+// (both refuse a rate other than their own outright). Making the caller name
+// the rate keeps that decision where the route is known.
+//
+// Downmixing along with the resample is not an extra liberty: every family a
+// positive rate is used for (silero_vad, sortformer_diar and every ASR family)
+// begins by calling the same mixdown_interleaved_to_mono_average on whatever it
+// is given. Doing it once here produces the identical samples and halves the
+// buffer that is then moved through the request.
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate);
+
+// Writes 16-bit PCM WAV, creating parent directories.
+//
+// Throws ConfigError, i.e. INVALID_ARGUMENT, ONLY for an empty path, which is a
+// malformed request. Every other failure throws a plain runtime_error, i.e.
+// INTERNAL: the destination is LocalAI's own generated-content directory and
+// not anything the caller named, so a full disk or a permission fault there is
+// a server fault and is worth retrying, which is the opposite of what
+// INVALID_ARGUMENT tells a client.
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio);
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io_ctest.cpp

@@ -0,0 +1,217 @@
+// Tests for audio_io's reading contract, and in particular for the resampling
+// that keeps a 44.1 or 48 kHz upload from reaching a family that only accepts
+// 16 kHz.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE: see the note at the top of
+// result_map_ctest.cpp. This file links the audio.cpp engine, so it is built
+// and run by ctest, not by backend/cpp/run-unit-tests.sh.
+//
+//   make -C backend/cpp/audio-cpp test-engine
+
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include <algorithm>
+#include <cmath>
+#include <cstdio>
+#include <filesystem>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// A one-second tone, interleaved across `channels`. Real audio rather than
+// silence so a resample that dropped its input would be visible as a flat
+// buffer, not just as a different length.
+static engine::runtime::AudioBuffer tone(int sample_rate, int channels,
+                                         float seconds) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = channels;
+    const auto frames =
+        static_cast<size_t>(static_cast<double>(sample_rate) * seconds);
+    buffer.samples.reserve(frames * static_cast<size_t>(channels));
+    for (size_t frame = 0; frame < frames; ++frame) {
+        const float value = 0.5f * std::sin(2.0f * 3.14159265f * 220.0f *
+                                            static_cast<float>(frame) /
+                                            static_cast<float>(sample_rate));
+        for (int channel = 0; channel < channels; ++channel) {
+            buffer.samples.push_back(value);
+        }
+    }
+    return buffer;
+}
+
+static float peak(const std::vector<float> &samples) {
+    float highest = 0.0f;
+    for (const float sample : samples) {
+        highest = std::max(highest, std::abs(sample));
+    }
+    return highest;
+}
+
+static std::filesystem::path scratch_dir() {
+    const auto dir = std::filesystem::temp_directory_path() / "audiocpp-io-ctest";
+    std::filesystem::create_directories(dir);
+    return dir;
+}
+
+// The I2 fixture. Before the resample this returned a 44.1 kHz buffer, which
+// silero_vad and sortformer_diar both reject with a plain runtime_error, which
+// the server maps to INTERNAL. A 44.1 kHz WAV is an ordinary upload.
+static void test_441k_stereo_is_read_as_16k_mono() {
+    const auto path = scratch_dir() / "input-44100-stereo.wav";
+    write_audio_file(path.string(), tone(44100, 2, 1.0f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "44.1 kHz input is resampled to 16 kHz");
+    check(audio.channels == 1, "stereo input is downmixed to mono");
+    // Linear resampling lands within a sample or two of the exact ratio.
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 15990 && frames < 16010,
+          "one second in stays one second out");
+    check(peak(audio.samples) > 0.2f,
+          "the resampled buffer still carries the signal");
+}
+
+static void test_48k_is_read_as_16k() {
+    const auto path = scratch_dir() / "input-48000-mono.wav";
+    write_audio_file(path.string(), tone(48000, 1, 0.5f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "48 kHz input is resampled to 16 kHz");
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 7990 && frames < 8010, "half a second in, half a second out");
+}
+
+// The common case: the upload is already 16 kHz mono, and nothing is resampled.
+static void test_16k_mono_passes_through_unchanged() {
+    const auto path = scratch_dir() / "input-16000-mono.wav";
+    const auto source = tone(16000, 1, 1.0f);
+    write_audio_file(path.string(), source);
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "16 kHz stays 16 kHz");
+    check(audio.channels == 1, "mono stays mono");
+    check(audio.samples.size() == source.samples.size(),
+          "a matching rate resamples nothing");
+}
+
+// Rate 0 means "give me the file as it is", which is what a source separation
+// route needs: demucs and roformer refuse anything but their own 44.1 kHz and
+// work on stereo, so the reader must not force them to 16 kHz mono.
+static void test_zero_target_keeps_the_native_format() {
+    const auto path = scratch_dir() / "input-native.wav";
+    write_audio_file(path.string(), tone(44100, 2, 0.25f));
+
+    const auto audio = read_audio_file(path.string(), 0);
+    check(audio.sample_rate == 44100, "a zero target keeps the file's rate");
+    check(audio.channels == 2, "a zero target keeps the file's channels");
+}
+
+static void test_missing_file_is_a_config_error() {
+    bool threw_config_error = false;
+    try {
+        read_audio_file((scratch_dir() / "does-not-exist.wav").string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+        // Any other type maps to INTERNAL, which is what this asserts against.
+    }
+    check(threw_config_error, "a missing input file is INVALID_ARGUMENT, not INTERNAL");
+}
+
+static void test_unreadable_file_is_a_config_error() {
+    const auto path = scratch_dir() / "not-a-wav.wav";
+    {
+        FILE *file = fopen(path.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("this is not a RIFF header", file);
+            fclose(file);
+        }
+    }
+    bool threw_config_error = false;
+    try {
+        read_audio_file(path.string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "a non-WAV input is INVALID_ARGUMENT, not INTERNAL");
+}
+
+// The write side of the same distinction. The destination is LocalAI's own
+// generated-content directory, not a caller-supplied path, so a failure to
+// write it is a server fault: INTERNAL, which a client may retry, and not
+// INVALID_ARGUMENT, which tells it the request itself was wrong.
+static void test_write_failure_is_not_a_config_error() {
+    // A regular file where a directory has to be. ENOTDIR defeats root as well
+    // as an ordinary user, unlike a chmod, which CI running as root would walk
+    // straight through.
+    const auto blocker = scratch_dir() / "blocking-file";
+    {
+        FILE *file = fopen(blocker.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("not a directory", file);
+            fclose(file);
+        }
+    }
+    const auto path = blocker / "nested" / "out.wav";
+
+    bool threw_config_error = false;
+    bool threw_something = false;
+    try {
+        write_audio_file(path.string(), tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+        threw_something = true;
+    } catch (const std::exception &) {
+        threw_something = true;
+    }
+    check(threw_something, "an unwritable destination is reported at all");
+    check(!threw_config_error,
+          "a failed write is INTERNAL, not INVALID_ARGUMENT: the caller did not "
+          "choose the destination and cannot fix it");
+    check(!std::filesystem::exists(path), "and nothing was written");
+}
+
+static void test_empty_output_path_is_a_config_error() {
+    // The one write failure that IS the caller's: no path at all.
+    bool threw_config_error = false;
+    try {
+        write_audio_file("", tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "an empty output path stays INVALID_ARGUMENT");
+}
+
+int main() {
+    test_441k_stereo_is_read_as_16k_mono();
+    test_48k_is_read_as_16k();
+    test_16k_mono_passes_through_unchanged();
+    test_zero_target_keeps_the_native_format();
+    test_missing_file_is_a_config_error();
+    test_unreadable_file_is_a_config_error();
+    test_write_failure_is_not_a_config_error();
+    test_empty_output_path_is_a_config_error();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_io checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/audio_units.cpp

@@ -0,0 +1,139 @@
+#include "audio_units.h"
+
+#include <algorithm>
+#include <cmath>
+#include <limits>
+
+namespace audiocpp_backend {
+
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels) {
+    const std::size_t lanes = channels > 0 ? static_cast<std::size_t>(channels)
+                                           : static_cast<std::size_t>(1);
+    // Truncating division is deliberate: a trailing partial frame is not a
+    // position every channel reached, so counting it would overstate the length.
+    return static_cast<std::int64_t>(sample_count / lanes);
+}
+
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0;
+    }
+    // Split into whole seconds plus a remainder so the intermediate product
+    // cannot overflow on long recordings, and so rates like 44100 stay exact.
+    // The remainder division truncates deliberately: that matches Go's
+    // time.Duration conventions and keeps successive sample indices monotonic.
+    const std::int64_t rate = static_cast<std::int64_t>(sample_rate);
+    const std::int64_t whole_seconds = samples / rate;
+    const std::int64_t remainder = samples % rate;
+    return whole_seconds * 1000000000LL + (remainder * 1000000000LL) / rate;
+}
+
+float samples_to_seconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0.0f;
+    }
+    return static_cast<float>(static_cast<double>(samples) /
+                              static_cast<double>(sample_rate));
+}
+
+std::int64_t seconds_to_samples(double seconds, int sample_rate) {
+    // !(seconds > 0.0) rather than seconds <= 0.0: every comparison against NaN
+    // is false, so the <= form lets NaN reach the cast below, which is undefined
+    // behaviour and lands on INT64_MIN in practice. This is the one entry point
+    // fed by untrusted-shaped input (a float-seconds timestamp off the wire, or
+    // a boundary from a model that diverged), and a hugely negative sample index
+    // used later as an offset or a length is a wild pointer rather than merely a
+    // wrong timestamp.
+    if (sample_rate <= 0 || !(seconds > 0.0)) {
+        return 0;
+    }
+    const double scaled = seconds * static_cast<double>(sample_rate);
+    // Bound before the cast for the same reason: converting a double at or above
+    // 2^63 (infinity included) is undefined behaviour, so saturate instead.
+    const double limit =
+        static_cast<double>(std::numeric_limits<std::int64_t>::max());
+    if (scaled >= limit) {
+        return std::numeric_limits<std::int64_t>::max();
+    }
+    // Round rather than truncate: these functions exist to cross the float
+    // seconds boundary the VAD and diarize messages use, so a value that came
+    // from samples_to_seconds converts back to the sample it started as.
+    // Truncation lost one sample about half the time, starting at n=1.
+    //
+    // That round trip is exact only below roughly 2^23 samples. Past that the
+    // float samples_to_seconds returns can no longer resolve adjacent indices
+    // and the trip fails whatever the rounding. Both the first failing INDEX
+    // and the duration it stands for depend on the rate, so they are listed per
+    // rate rather than folded into one range; measured:
+    //
+    //   16 kHz    16384001 samples   17.1 min
+    //   44.1 kHz  11289602 samples    4.3 min
+    //   48 kHz    12288002 samples    4.3 min
+    //   96 kHz    12288002 samples    2.1 min
+    //
+    // The shortest recording this bites is therefore a couple of minutes of
+    // 96 kHz audio. It is a property of the float seconds API itself, not of
+    // the rounding here, and it is why nothing should use these to carry a
+    // sample-accurate position in a long recording.
+    return static_cast<std::int64_t>(std::llround(scaled));
+}
+
+std::vector<float> s16le_to_f32(const std::string &bytes) {
+    std::vector<float> samples;
+    const size_t count = bytes.size() / 2;
+    samples.reserve(count);
+    for (size_t i = 0; i < count; ++i) {
+        const auto low = static_cast<unsigned char>(bytes[i * 2]);
+        const auto high = static_cast<unsigned char>(bytes[i * 2 + 1]);
+        const auto raw = static_cast<std::int16_t>(
+            static_cast<std::uint16_t>(low) |
+            (static_cast<std::uint16_t>(high) << 8));
+        // 32768 on decode against 32767 on encode is deliberate, not a typo.
+        // 32768 is what keeps INT16_MIN at exactly -1.0 and every other code
+        // inside the [-1, 1] range this header promises; dividing by 32767
+        // would decode INT16_MIN to -1.00003. See f32_to_s16le for the other
+        // half of the pair. The cost is that a round trip shrinks a sample by
+        // 32767/32768, well under one LSB.
+        samples.push_back(static_cast<float>(raw) / 32768.0f);
+    }
+    return samples;
+}
+
+std::string f32_to_s16le(const std::vector<float> &samples) {
+    std::string bytes;
+    bytes.reserve(samples.size() * 2);
+    for (const float sample : samples) {
+        // NaN maps to silence. A NaN sample rendered as a full-scale click is
+        // worse audio than a dropped one, and this unit converts audio that may
+        // have originated off the wire.
+        //
+        // This guard also removes what used to be a spelling hazard in the
+        // clamp below. std::min and std::max return their first argument when
+        // the comparison is false, and every comparison against NaN is false,
+        // so before this branch existed the choice of spelling silently decided
+        // whether a NaN reached std::lround, whose result is unspecified for
+        // NaN. These three leaked it, the last being the idiomatic C++17 way to
+        // write a clamp and so the likeliest future edit:
+        //   std::min(std::max(sample, -1.0f), 1.0f)
+        //   std::max(std::min(sample, 1.0f), -1.0f)
+        //   std::clamp(sample, -1.0f, 1.0f)
+        // The order is no longer load-bearing now that the guard runs first,
+        // but the history is why the guard is here, so do not drop it.
+        if (std::isnan(sample)) {
+            bytes.push_back(0);
+            bytes.push_back(0);
+            continue;
+        }
+        const float clamped = std::max(-1.0f, std::min(1.0f, sample));
+        // 32767 rather than 32768 so +1.0 saturates at INT16_MAX instead of
+        // overflowing to INT16_MIN. See s16le_to_f32 for why decode differs.
+        const auto value =
+            static_cast<std::int16_t>(std::lround(clamped * 32767.0f));
+        const auto raw = static_cast<std::uint16_t>(value);
+        bytes.push_back(static_cast<char>(raw & 0xFF));
+        bytes.push_back(static_cast<char>((raw >> 8) & 0xFF));
+    }
+    return bytes;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units.h

@@ -0,0 +1,50 @@
+#pragma once
+
+// Time and sample-format conversion between audio.cpp's runtime types (sample
+// indices, float PCM) and LocalAI's proto types. Standard library only.
+//
+// LocalAI uses three different time units:
+//   TranscriptSegment / TranscriptWord start,end : int64 nanoseconds
+//   VADSegment start,end                         : float seconds
+//   DiarizeSegment start,end                     : float seconds
+
+#include <cstddef>
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Frames in an interleaved buffer of `sample_count` floats laid out across
+// `channels` channels. A frame is one per-channel position, which is the unit
+// every duration and every span boundary in this backend is expressed in, so a
+// stereo buffer must not report twice its real length: feeding sample_count
+// straight to samples_to_seconds makes a 3 second stereo clip come back as 6.
+//
+// A non-positive channel count is treated as mono, matching
+// engine::runtime::AudioBuffer's own default of 1 and keeping a reader that
+// reports 0 channels from dividing by zero.
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels);
+
+// Returns 0 when sample_rate is not positive rather than dividing by zero.
+// Uses integer arithmetic so 44.1 kHz does not lose precision.
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate);
+
+float samples_to_seconds(std::int64_t samples, int sample_rate);
+
+// Rounds to nearest. Negative seconds and NaN both yield 0, and a value too
+// large to convert saturates at INT64_MAX rather than overflowing. Round trips
+// with samples_to_seconds only below roughly 2^23 samples, past which the float
+// seconds can no longer resolve adjacent sample indices.
+std::int64_t seconds_to_samples(double seconds, int sample_rate);
+
+// Decodes little-endian signed 16-bit PCM. A trailing odd byte is dropped.
+std::vector<float> s16le_to_f32(const std::string &bytes);
+
+// Encodes to little-endian signed 16-bit PCM, clamping to [-1, 1] first so an
+// overshooting sample saturates instead of wrapping to the opposite sign.
+// A NaN sample encodes to 0, on the grounds that silence beats a full-scale
+// click.
+std::string f32_to_s16le(const std::vector<float> &samples);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units_test.cpp

@@ -0,0 +1,226 @@
+// Unit tests for audio_units. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+
+#include "audio_units.cpp"
+
+#include <cfenv>
+#include <cmath>
+#include <cstdio>
+#include <limits>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static bool close_to(float a, float b, float tol) { return std::fabs(a - b) <= tol; }
+
+using namespace audiocpp_backend;
+
+static void test_nanoseconds() {
+    // LocalAI TranscriptSegment/TranscriptWord times are nanoseconds
+    // (Go reads them as time.Duration).
+    check(samples_to_nanoseconds(16000, 16000) == 1000000000LL, "1s at 16k is 1e9 ns");
+    check(samples_to_nanoseconds(8000, 16000) == 500000000LL, "0.5s at 16k");
+    check(samples_to_nanoseconds(0, 16000) == 0, "zero samples is zero ns");
+    check(samples_to_nanoseconds(1000, 0) == 0, "zero sample rate yields zero, not UB");
+    // 44.1 kHz must not lose precision to float arithmetic.
+    check(samples_to_nanoseconds(44100, 44100) == 1000000000LL, "1s at 44.1k");
+    check(samples_to_nanoseconds(22050, 44100) == 500000000LL, "0.5s at 44.1k");
+    // The cases above all land on values a float happens to hold exactly, so
+    // they do not actually rule float arithmetic out. These do:
+    // a fraction that does not divide evenly, and a duration whose magnitude
+    // exceeds a float's 24-bit mantissa at nanosecond resolution.
+    check(samples_to_nanoseconds(44099, 44100) == 999977324LL,
+          "44.1k fraction is exact, not rounded through a float");
+    check(samples_to_nanoseconds(44100LL * 3600, 44100) == 3600000000000LL,
+          "one hour at 44.1k is exact to the nanosecond");
+    // A naive samples * 1e9 would overflow int64 here; the split into whole
+    // seconds plus a remainder is what keeps this correct.
+    check(samples_to_nanoseconds(44100LL * 360000, 44100) == 360000000000000LL,
+          "100 hours at 44.1k does not overflow");
+    // Double arithmetic is close enough to pass everything above, but still
+    // truncates this one a nanosecond short. Integer division does not.
+    check(samples_to_nanoseconds(4004, 8000) == 500500000LL,
+          "0.5005s at 8k is exact to the nanosecond");
+    // Truncation, not rounding: this matches Go's time.Duration conventions and
+    // keeps successive sample indices monotonic. The exact value here is
+    // 22675.7...; rounding to nearest would give 22676.
+    check(samples_to_nanoseconds(1, 44100) == 22675LL,
+          "a sub-nanosecond fraction truncates rather than rounding up");
+}
+
+static void test_seconds() {
+    check(close_to(samples_to_seconds(24000, 24000), 1.0f, 1e-6f), "1s at 24k");
+    check(close_to(samples_to_seconds(12000, 24000), 0.5f, 1e-6f), "0.5s at 24k");
+    check(close_to(samples_to_seconds(100, 0), 0.0f, 1e-6f), "zero sample rate is 0s");
+    check(seconds_to_samples(1.0, 16000) == 16000, "1s to samples at 16k");
+    check(seconds_to_samples(0.5, 16000) == 8000, "0.5s to samples at 16k");
+    check(seconds_to_samples(1.0, 0) == 0, "zero sample rate yields zero samples");
+    check(seconds_to_samples(-1.0, 16000) == 0, "negative seconds clamps to zero");
+
+    // seconds_to_samples is the one entry point fed by untrusted-shaped input:
+    // a float-seconds timestamp off the wire, or a VAD boundary from a model
+    // that diverged. A hugely negative sample index used later as an offset or
+    // a length is a wild pointer, not merely a wrong timestamp.
+    const double nan_seconds = std::numeric_limits<double>::quiet_NaN();
+    const double inf_seconds = std::numeric_limits<double>::infinity();
+    const std::int64_t max_samples = std::numeric_limits<std::int64_t>::max();
+    check(seconds_to_samples(nan_seconds, 16000) == 0, "NaN seconds yields zero");
+    check(seconds_to_samples(inf_seconds, 16000) == max_samples,
+          "infinite seconds saturates instead of overflowing");
+    check(seconds_to_samples(1e30, 16000) == max_samples,
+          "out of range seconds saturates instead of overflowing");
+    check(seconds_to_samples(-inf_seconds, 16000) == 0,
+          "negative infinity clamps to zero");
+
+    // Crossing the float-seconds boundary and back is the expected round trip
+    // for the VAD and diarize messages, so it must not lose a sample.
+    // Truncation loses one about half the time, starting at n=1.
+    check(seconds_to_samples(samples_to_seconds(1, 44100), 44100) == 1,
+          "one sample survives the seconds round trip at 44.1k");
+    check(seconds_to_samples(samples_to_seconds(1, 16000), 16000) == 1,
+          "one sample survives the seconds round trip at 16k");
+    check(seconds_to_samples(samples_to_seconds(4001, 8000), 8000) == 4001,
+          "4001 samples survive the seconds round trip at 8k");
+}
+
+static void test_s16le_round_trip() {
+    const std::vector<float> original = {0.0f, 0.5f, -0.5f, 1.0f, -1.0f};
+    const std::string encoded = f32_to_s16le(original);
+    check(encoded.size() == original.size() * 2, "two bytes per sample");
+
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == original.size(), "round trip keeps the sample count");
+    for (size_t i = 0; i < original.size(); ++i) {
+        // 16-bit quantisation: one LSB is ~3.05e-5. Guard the index so a short
+        // result reports a named failure instead of aborting the whole suite.
+        check(i < decoded.size() && close_to(decoded[i], original[i], 1e-4f),
+              "round trip preserves sample " + std::to_string(i));
+    }
+}
+
+static void test_s16le_endianness() {
+    // 0.5 encodes to 16384 = 0x4000, little endian is 0x00 0x40.
+    const std::string encoded = f32_to_s16le({0.5f});
+    check(encoded.size() == 2, "one sample is two bytes");
+    check(static_cast<unsigned char>(encoded[0]) == 0x00, "low byte first");
+    check(static_cast<unsigned char>(encoded[1]) == 0x40, "high byte second");
+}
+
+static void test_s16le_clamping() {
+    // Values outside [-1, 1] must clamp, not wrap around to the opposite sign.
+    const std::string encoded = f32_to_s16le({2.0f, -2.0f});
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == 2, "two samples survive clamping");
+    check(decoded.size() > 0 && decoded[0] > 0.99f,
+          "positive overshoot clamps to full scale");
+    check(decoded.size() > 1 && decoded[1] < -0.99f,
+          "negative overshoot clamps to full scale");
+}
+
+static void test_s16le_decode_range() {
+    // INT16_MIN is the one value that pins the decode scale. Dividing by 32767
+    // instead of 32768 would decode it to -1.00003, outside the [-1, 1] range
+    // the header promises, and every other test would still pass.
+    const std::vector<float> decoded = s16le_to_f32(std::string("\x00\x80", 2));
+    check(decoded.size() == 1, "INT16_MIN decodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == -1.0f,
+          "INT16_MIN decodes to exactly -1.0, not past full scale");
+}
+
+static void test_s16le_nan_input() {
+    // A NaN sample must not reach std::lround, whose result is unspecified for
+    // NaN. Asserting a range is not enough to pin this: the three outcomes the
+    // plausible clamp spellings produce (full scale, negative full scale, zero)
+    // are all finite and all inside [-1, 1], so a range check passes for every
+    // one of them. Only an exact value distinguishes them.
+    // NaN maps to silence, not to full scale: a NaN sample rendered as a
+    // full-scale click is worse audio than a dropped one, and this unit
+    // converts audio that may have originated off the wire.
+    //
+    // volatile so the NaN cannot be constant-folded, which would let the
+    // compiler evaluate the conversion at compile time and raise no
+    // floating-point exception at run time for the check below to observe.
+    volatile float nan_source = std::numeric_limits<float>::quiet_NaN();
+    const std::vector<float> input = {nan_source};
+
+    std::feclearexcept(FE_ALL_EXCEPT);
+    const std::string encoded = f32_to_s16le(input);
+    const bool raised_invalid = std::fetestexcept(FE_INVALID) != 0;
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+
+    check(decoded.size() == 1, "a NaN sample still encodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == 0.0f,
+          "a NaN sample encodes to exactly zero, not to a full-scale click");
+    // Independent of the value: a quiet NaN raises invalid-operation as soon as
+    // it reaches any ordered comparison, which is what std::min and std::max
+    // use, so this fails unless the NaN is diverted before the clamp runs at
+    // all. That is what stops the explicit guard from being dropped in favour
+    // of a clamp spelling that happens to yield zero.
+    check(!raised_invalid,
+          "encoding a NaN sample raises no invalid-operation exception");
+}
+
+static void test_s16le_odd_length() {
+    // A truncated frame must drop the dangling byte rather than read past it.
+    const std::string odd(5, '\0');
+    check(s16le_to_f32(odd).size() == 2, "odd byte count drops the trailing byte");
+    check(s16le_to_f32(std::string()).empty(), "empty input yields no samples");
+}
+
+static void test_interleaved_frame_count() {
+    // Mono is a pass-through, which is the only case the VAD path exercises.
+    check(interleaved_frame_count(16000, 1) == 16000, "mono frames equal samples");
+    // The case that matters: a stereo buffer holds two floats per position, so a
+    // one second 16 kHz stereo clip is 32000 floats and still one second. Handing
+    // the raw float count to samples_to_seconds reports two seconds instead.
+    check(interleaved_frame_count(32000, 2) == 16000,
+          "stereo frames are half the samples");
+    check(samples_to_seconds(interleaved_frame_count(32000, 2), 16000) == 1.0f,
+          "a one second stereo clip measures one second, not two");
+    check(interleaved_frame_count(48000, 3) == 16000,
+          "three channels divide by three");
+    // engine::runtime::AudioBuffer defaults channels to 1, but a reader is free
+    // to report 0, and dividing by that is undefined rather than merely wrong.
+    check(interleaved_frame_count(1000, 0) == 1000,
+          "zero channels is treated as mono");
+    check(interleaved_frame_count(1000, -2) == 1000,
+          "a negative channel count is treated as mono");
+    // A dangling partial frame is not a position every channel reached.
+    check(interleaved_frame_count(3, 2) == 1,
+          "a trailing partial frame is not counted");
+    check(interleaved_frame_count(0, 2) == 0, "an empty buffer has no frames");
+    // Past 2^32 floats, so a size_t narrowed to 32 bits on the way in, or a
+    // signed 32-bit intermediate, shows up here rather than in a multi-hour
+    // recording nobody tests with.
+    check(interleaved_frame_count(static_cast<std::size_t>(9000000000ULL), 2) ==
+              4500000000LL,
+          "a buffer beyond 2^32 floats counts frames without truncating");
+}
+
+int main() {
+    test_interleaved_frame_count();
+    test_nanoseconds();
+    test_seconds();
+    test_s16le_round_trip();
+    test_s16le_endianness();
+    test_s16le_clamping();
+    test_s16le_decode_range();
+    test_s16le_nan_input();
+    test_s16le_odd_length();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_units checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/capability_routing.cpp

@@ -0,0 +1,411 @@
+#include "capability_routing.h"
+
+#include <algorithm>
+
+namespace audiocpp_backend {
+namespace {
+
+struct NamedTask {
+    Task task;
+    const char *name;
+};
+
+// Short names are exactly the strings audio.cpp prints and parses in
+// framework/runtime/session.cpp, so a name pinned here survives conversion at
+// the engine boundary and a name copied out of audio.cpp is accepted here. All
+// thirteen have an upstream name; only "spk" is absent from the --task table in
+// docs/usage.md.
+const NamedTask kTaskNames[] = {
+    {Task::Vad, "vad"},
+    {Task::Asr, "asr"},
+    {Task::Diarization, "diar"},
+    {Task::SourceSeparation, "sep"},
+    {Task::AudioGeneration, "gen"},
+    {Task::Tts, "tts"},
+    {Task::VoiceCloning, "clon"},
+    {Task::VoiceConversion, "vc"},
+    {Task::SpeechToSpeech, "s2s"},
+    {Task::Alignment, "align"},
+    {Task::VoiceDesign, "vdes"},
+    {Task::SpeakerRecognition, "spk"},
+    {Task::Svc, "svc"},
+};
+
+// Accepted on input but never emitted. "spkrec" was this backend's own earlier
+// name for the kind; upstream only ever knew "spk".
+const NamedTask kTaskAliases[] = {
+    {Task::SpeakerRecognition, "spkrec"},
+};
+
+// First match wins, which is safe because a Capabilities value holds at most one
+// entry per task: it mirrors upstream runtime::TaskCapability (model.h), which
+// pairs one kind with a modes vector, and no loader's supported_tasks list
+// repeats a kind.
+bool family_supports(const Capabilities &caps, Task task, Mode mode) {
+    for (const auto &capability : caps.tasks) {
+        if (capability.task != task) {
+            continue;
+        }
+        return std::find(capability.modes.begin(), capability.modes.end(), mode) !=
+               capability.modes.end();
+    }
+    return false;
+}
+
+// Mode preference per RPC. Only AudioTranscriptionStream has a fallback: a
+// server-streaming transcription can be satisfied by an offline run that emits
+// one delta then the final result. Live transcription cannot, because it is
+// bidirectional and must consume audio incrementally.
+std::vector<Mode> mode_candidates(Rpc rpc) {
+    switch (rpc) {
+    case Rpc::TtsStream:
+    case Rpc::AudioTranscriptionLive:
+        return {Mode::Streaming};
+    case Rpc::AudioTranscriptionStream:
+        return {Mode::Streaming, Mode::Offline};
+    default:
+        return {Mode::Offline};
+    }
+}
+
+std::vector<Task> task_candidates(Rpc rpc, const RequestShape &shape) {
+    switch (rpc) {
+    case Rpc::Tts:
+    case Rpc::TtsStream:
+        // A supplied speaker clip is the strongest signal: the caller named the
+        // voice they want. Free-form instructions come next. Both fall back to
+        // plain Tts so a family without the specialised task still answers.
+        if (shape.has_voice_reference) {
+            return {Task::VoiceCloning, Task::Tts, Task::VoiceDesign};
+        }
+        if (shape.has_instructions) {
+            return {Task::VoiceDesign, Task::Tts, Task::VoiceCloning};
+        }
+        return {Task::Tts, Task::VoiceCloning, Task::VoiceDesign};
+    case Rpc::AudioTranscription:
+    case Rpc::AudioTranscriptionStream:
+    case Rpc::AudioTranscriptionLive:
+        // Asr first: `prompt` is also whisper-style decoding context, so its
+        // presence must not hijack a real ASR family into forced alignment.
+        if (shape.has_prompt_text) {
+            return {Task::Asr, Task::Alignment};
+        }
+        return {Task::Asr};
+    case Rpc::Vad:
+        return {Task::Vad};
+    case Rpc::Diarize:
+        return {Task::Diarization};
+    case Rpc::SoundGeneration:
+        return {Task::AudioGeneration};
+    case Rpc::AudioTransform:
+        // Svc is listed for completeness but is unreachable by auto-routing, by
+        // design: the only families advertising it (seed_vc, vevo2) also
+        // advertise VoiceConversion, which always wins, and no request signal
+        // means "this input is singing". Singing voice conversion therefore
+        // requires an explicit task:svc pin.
+        return {Task::SourceSeparation, Task::VoiceConversion, Task::Svc,
+                Task::SpeechToSpeech};
+    }
+    return {};
+}
+
+// The reasons behind unsupported_surfaces(), spelled once because AudioEncode
+// and AudioDecode share theirs. Each is phrased in terms of what upstream does
+// and does not have, so a reader can check it against the pinned checkout
+// rather than take it on trust. Every one of them was checked against
+// audio.cpp e800d435d130dc776baf6f3e6129bb62b1495c89, and one of the four
+// claims this backend was planned against did not survive that check: see
+// kTransformStreamReason.
+//
+// A latent upstream inconsistency worth knowing about but deliberately NOT put
+// on the wire, because it would mislead: model_spec/schema.cpp's task-string
+// whitelist does accept "codec" (and "dialogue"), while
+// model_spec/metadata.cpp's parse_task_kind has no branch for either and
+// throws "unknown model spec task". So a spec declaring "codec" validates and
+// then fails to load. That is a hole in upstream's own validation, not a codec
+// task this backend could reach.
+const char *const kCodecReason =
+    "audio.cpp's VoiceTaskKind has no codec entry, so no family can be asked to "
+    "turn PCM into codec frames or back; miocodec carries a Codec tag in "
+    "upstream's README but its loader advertises only vc and s2s";
+// NOT "streaming exists for tts and asr only", which is what this backend was
+// planned to say and is false: silero_vad advertises vad with RunMode::Streaming
+// (src/models/silero_vad/session.cpp). The claim that actually holds is the
+// narrower one below, about the four tasks AudioTransform routes to.
+// The trailing clause is not padding. The premise is an absence, and an absence
+// does not on its own make the RPC impossible: an offline sep family could be
+// buffered and emitted as a stream, which is what several LocalAI backends do.
+// Stopping at "nothing advertises streaming" would imply an impossibility the
+// evidence does not support. What is true, and what the caller needs, is that
+// this backend declines to dress an offline call up as a streaming one.
+const char *const kTransformStreamReason =
+    "no audio.cpp family advertises streaming for any task AudioTransform routes "
+    "to (sep, vc, svc, s2s); upstream advertises RunMode::Streaming for tts, asr "
+    "and vad only, and no conversion or separation family even implements its "
+    "IStreamingVoiceTaskSession interface, so a streaming transform here would be "
+    "a buffered offline call in disguise, which this backend does not pretend to "
+    "offer";
+// "clip-to-clip processing against a target voice", NOT "voice conversion". The
+// latter is true of miocodec and FALSE of vevo2, whose s2s route is `editing`
+// and only `editing`: src/models/vevo2/session.cpp's default_route_for_task maps
+// SpeechToSpeech to Editing and route_matches_task accepts nothing else, and
+// docs/models/vevo2.md defines that route as "Edit source speech into new target
+// text while using the target voice", requiring --target-text. It rewrites what
+// was said. vevo2's actual voice conversion is its separate vc task, which is
+// why upstream's README tags the family "TTS, Music, VC, Edit". The conclusion
+// is unaffected: neither family converses.
+const char *const kAudioToAudioReason =
+    "LocalAI's contract here is OpenAI-Realtime shaped, an audio conversation "
+    "emitting audio, transcript and tool-call deltas from a system prompt and a "
+    "tool list; audio.cpp's s2s is offline clip-to-clip processing against a "
+    "target voice, declared only by miocodec (voice conversion) and vevo2 "
+    "(speech editing), with no conversation, system prompt or tool loop";
+const char *const kVoiceEmbedReason =
+    "no audio.cpp family advertises the spk (SpeakerRecognition) task, so "
+    "nothing in the engine can produce a speaker embedding; the task kind "
+    "itself exists upstream, and TitaNet and ECAPA-TDNN exist as internal "
+    "conditioning encoders, but neither is registered as a loadable family";
+
+// The tasks an RPC is ever willing to route to, independent of request shape.
+//
+// DERIVED from task_candidates rather than restated, so a task added to an
+// RPC's candidate list cannot become inadmissible as a pin by omission. Setting
+// every shape flag yields each RPC's widest list: the per-flag branches only
+// reorder the same three tasks for Tts, and only ADD Alignment for
+// transcription, so the union is what comes back.
+std::vector<Task> admissible_tasks(Rpc rpc) {
+    RequestShape widest;
+    widest.has_voice_reference = true;
+    widest.has_instructions = true;
+    widest.has_prompt_text = true;
+    return task_candidates(rpc, widest);
+}
+
+std::string join_task_names(const std::vector<Task> &tasks) {
+    std::string out;
+    for (const Task task : tasks) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += task_name(task);
+    }
+    if (out.empty()) {
+        out = "nothing";
+    }
+    return out;
+}
+
+std::string join_attempts(const std::vector<Task> &tasks,
+                          const std::vector<Mode> &modes) {
+    std::string out;
+    for (const Task task : tasks) {
+        for (const Mode mode : modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += task_name(task);
+            out += "/";
+            out += mode_name(mode);
+        }
+    }
+    return out;
+}
+
+} // namespace
+
+const char *task_name(Task task) {
+    for (const auto &entry : kTaskNames) {
+        if (entry.task == task) {
+            return entry.name;
+        }
+    }
+    return "unknown";
+}
+
+const char *mode_name(Mode mode) {
+    return mode == Mode::Streaming ? "streaming" : "offline";
+}
+
+const char *rpc_name(Rpc rpc) {
+    switch (rpc) {
+    case Rpc::Tts:
+        return "TTS";
+    case Rpc::TtsStream:
+        return "TTSStream";
+    case Rpc::AudioTranscription:
+        return "AudioTranscription";
+    case Rpc::AudioTranscriptionStream:
+        return "AudioTranscriptionStream";
+    case Rpc::AudioTranscriptionLive:
+        return "AudioTranscriptionLive";
+    case Rpc::Vad:
+        return "VAD";
+    case Rpc::Diarize:
+        return "Diarize";
+    case Rpc::SoundGeneration:
+        return "SoundGeneration";
+    case Rpc::AudioTransform:
+        return "AudioTransform";
+    }
+    return "unknown";
+}
+
+bool parse_task_name(const std::string &value, Task &out) {
+    for (const auto &entry : kTaskNames) {
+        if (value == entry.name) {
+            out = entry.task;
+            return true;
+        }
+    }
+    for (const auto &entry : kTaskAliases) {
+        if (value == entry.name) {
+            out = entry.task;
+            return true;
+        }
+    }
+    return false;
+}
+
+std::string describe_capabilities(const Capabilities &caps) {
+    std::string out;
+    for (const auto &capability : caps.tasks) {
+        for (const Mode mode : capability.modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += task_name(capability.task);
+            out += "/";
+            out += mode_name(mode);
+        }
+    }
+    if (out.empty()) {
+        out = "nothing";
+    }
+    return out;
+}
+
+const std::vector<UnsupportedSurface> &unsupported_surfaces() {
+    // Ordered as UnsupportedRpc declares them. unsupported_surface() names each
+    // index in a switch rather than casting the enum, so the order is checked at
+    // compile time rather than trusted.
+    static const std::vector<UnsupportedSurface> kSurfaces = {
+        {"AudioEncode", kCodecReason},
+        {"AudioDecode", kCodecReason},
+        {"AudioTransformStream", kTransformStreamReason},
+        {"AudioToAudioStream", kAudioToAudioReason},
+        {"VoiceEmbed", kVoiceEmbedReason},
+    };
+    return kSurfaces;
+}
+
+// A switch with NO default label, deliberately. -Wswitch is on under -Wall, so a
+// sixth UnsupportedRpc added without a case here is a BUILD diagnostic, which is
+// the only place this class of mistake can be caught for free: a positional
+// static_cast<size_t>(rpc) would compile fine and read past the end of the table
+// at run time, on the one code path whose entire job is to be diagnosable. The
+// table stays a table because the tests iterate it.
+//
+// The trailing return is unreachable through the enum and exists only for a
+// caller that hands over a value outside it, which is already undefined
+// behaviour by the time it arrives.
+const UnsupportedSurface &unsupported_surface(UnsupportedRpc rpc) {
+    const std::vector<UnsupportedSurface> &surfaces = unsupported_surfaces();
+    switch (rpc) {
+    case UnsupportedRpc::AudioEncode:
+        return surfaces[0];
+    case UnsupportedRpc::AudioDecode:
+        return surfaces[1];
+    case UnsupportedRpc::AudioTransformStream:
+        return surfaces[2];
+    case UnsupportedRpc::AudioToAudioStream:
+        return surfaces[3];
+    case UnsupportedRpc::VoiceEmbed:
+        return surfaces[4];
+    }
+    return surfaces[0];
+}
+
+std::string unsupported_surface_message(const Capabilities &caps, const char *rpc,
+                                        const char *reason) {
+    return std::string("audio-cpp: the ") + rpc +
+           " RPC is not available through this backend because " + reason +
+           ". Loaded family '" + caps.family +
+           "' supports: " + describe_capabilities(caps);
+}
+
+std::string unsupported_surface_message(const char *rpc, const char *reason) {
+    return std::string("audio-cpp: the ") + rpc +
+           " RPC is not available through this backend because " + reason +
+           ". No model is loaded, so there is no family to list; loading one "
+           "would not change this answer";
+}
+
+Route resolve_route(Rpc rpc, const RequestShape &shape,
+                    const Capabilities &caps) {
+    Route route;
+
+    std::vector<Task> tasks;
+    if (!shape.pinned_task.empty()) {
+        Task pinned = Task::Tts;
+        if (!parse_task_name(shape.pinned_task, pinned)) {
+            route.error = "audio-cpp: unknown task option '" + shape.pinned_task +
+                          "'. Known tasks: gen, tts, clon, vc, svc, s2s, asr, "
+                          "align, vad, diar, sep, vdes, spk";
+            return route;
+        }
+        // A pin is honoured exactly, but ONLY on an RPC that could have routed
+        // to it anyway. It used to replace the candidate list wholesale for
+        // every RPC, and because the model's `task:` option is copied into the
+        // shape by all nine handlers, one pin bled across all nine surfaces and
+        // produced wrong 200s rather than errors: nemotron with task:asr made
+        // Vad return 200 with zero segments after a full ASR decode, so 14
+        // seconds of speech was reported as silence, and silero_vad with
+        // task:vad made AudioTranscription return 200 with empty text and four
+        // segments whose spans were VAD segments, which the srt/vtt/lrc writers
+        // then rendered as a well formed subtitle file of four timed EMPTY
+        // cues. Refusing is what the docs already promise: "if the family
+        // cannot serve it, the request is refused rather than rerouted".
+        //
+        // Every legitimate pin survives, because a pin only ever names the task
+        // its own RPC already routes to: svc is in AudioTransform's candidates,
+        // tts/clon/vdes in TTS's, asr in transcription's, vad and diar in
+        // theirs.
+        const std::vector<Task> admissible = admissible_tasks(rpc);
+        if (std::find(admissible.begin(), admissible.end(), pinned) ==
+            admissible.end()) {
+            route.error = std::string("audio-cpp: this model pins task '") +
+                          task_name(pinned) + "', which the " + rpc_name(rpc) +
+                          " RPC never routes to (it routes to " +
+                          join_task_names(admissible) +
+                          "). Remove the task option to reach this RPC, or call "
+                          "the RPC the pinned task serves";
+            return route;
+        }
+        tasks = {pinned};
+    } else {
+        tasks = task_candidates(rpc, shape);
+    }
+
+    const std::vector<Mode> modes = mode_candidates(rpc);
+
+    // Task-major: prefer the right task in a fallback mode over the wrong task
+    // in the preferred mode.
+    for (const Task task : tasks) {
+        for (const Mode mode : modes) {
+            if (family_supports(caps, task, mode)) {
+                route.ok = true;
+                route.task = task;
+                route.mode = mode;
+                return route;
+            }
+        }
+    }
+
+    route.error = std::string("audio-cpp: family '") + caps.family +
+                  "' cannot serve the " + rpc_name(rpc) + " RPC (tried " +
+                  join_attempts(tasks, modes) + "); it supports: " +
+                  describe_capabilities(caps);
+    return route;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/capability_routing.h

@@ -0,0 +1,131 @@
+#pragma once
+
+// Decides which audio.cpp (task, mode) pair serves a given LocalAI RPC, or
+// produces the capability error when none can. Standard library only, so this
+// unit is tested without an audio.cpp checkout; loaded_model.cpp converts
+// to and from engine::runtime types at the boundary.
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Mirrors engine::runtime::VoiceTaskKind, same members and same order.
+enum class Task {
+    Vad,
+    Asr,
+    Diarization,
+    SourceSeparation,
+    AudioGeneration,
+    Tts,
+    VoiceCloning,
+    VoiceConversion,
+    SpeechToSpeech,
+    Alignment,
+    VoiceDesign,
+    SpeakerRecognition,
+    Svc,
+};
+
+// Mirrors engine::runtime::RunMode.
+enum class Mode { Offline, Streaming };
+
+struct TaskCapability {
+    Task task = Task::Vad;
+    std::vector<Mode> modes;
+};
+
+struct Capabilities {
+    std::string family;
+    std::vector<TaskCapability> tasks;
+};
+
+// The LocalAI RPCs this backend serves. The ones it cannot serve at all are in
+// UnsupportedRpc below rather than here: they never reach routing, because no
+// family could satisfy them.
+enum class Rpc {
+    Tts,
+    TtsStream,
+    AudioTranscription,
+    AudioTranscriptionStream,
+    AudioTranscriptionLive,
+    Vad,
+    Diarize,
+    SoundGeneration,
+    AudioTransform,
+};
+
+struct RequestShape {
+    // A speaker reference clip was supplied (TTSRequest.voice resolved to audio).
+    bool has_voice_reference = false;
+    // TTSRequest.instructions is set.
+    bool has_instructions = false;
+    // TranscriptRequest.prompt is set.
+    bool has_prompt_text = false;
+    // The model's `task:` option, empty when unset. Overrides routing.
+    std::string pinned_task;
+};
+
+struct Route {
+    bool ok = false;
+    Task task = Task::Tts;
+    Mode mode = Mode::Offline;
+    // Set when ok is false. Suitable verbatim as an UNIMPLEMENTED message.
+    std::string error;
+};
+
+Route resolve_route(Rpc rpc, const RequestShape &shape, const Capabilities &caps);
+
+// Canonical audio.cpp short names: gen, tts, clon, vc, svc, s2s, asr, align,
+// vad, diar, sep, vdes, spk. parse_task_name additionally accepts "spkrec" as
+// a legacy alias; task_name only ever emits "spk".
+const char *task_name(Task task);
+const char *mode_name(Mode mode);
+const char *rpc_name(Rpc rpc);
+bool parse_task_name(const std::string &value, Task &out);
+
+// "asr/offline, asr/streaming", for error messages.
+std::string describe_capabilities(const Capabilities &caps);
+
+// The RPCs in LocalAI's backend contract that audio.cpp has no counterpart for,
+// as opposed to the ones in Rpc above, which a particular family may or may not
+// be able to serve. Nothing routes to these: the refusal is a property of the
+// engine, not of the loaded model, so loading a different family cannot change
+// it.
+//
+// This is deliberately NOT deferred work. Each entry names the upstream
+// limitation that keeps it out of Rpc, and each becomes an ordinary routing
+// entry the day upstream lifts that limitation.
+enum class UnsupportedRpc {
+    AudioEncode,
+    AudioDecode,
+    AudioTransformStream,
+    AudioToAudioStream,
+    VoiceEmbed,
+};
+
+struct UnsupportedSurface {
+    // The RPC's name as backend.proto spells it, for the message.
+    const char *rpc;
+    // Why audio.cpp cannot serve it, in terms of what upstream does and does
+    // not have. Stated so a caller can tell "not built yet" from "not possible".
+    const char *reason;
+};
+
+// The table behind the five refusals. Exposed whole so a test can assert every
+// entry rather than the one somebody remembered to cover, and so the reasons
+// are data in one place instead of string literals hand-copied into handlers.
+const std::vector<UnsupportedSurface> &unsupported_surfaces();
+const UnsupportedSurface &unsupported_surface(UnsupportedRpc rpc);
+
+// Message for an RPC this backend cannot serve at all, as opposed to one this
+// particular family cannot serve. `reason` states the upstream limitation.
+std::string unsupported_surface_message(const Capabilities &caps, const char *rpc,
+                                        const char *reason);
+
+// Same, for the no-model-loaded case. Says so explicitly rather than naming an
+// empty family, and says that loading one would not help, because the caller's
+// obvious next move otherwise is to load a model and try again.
+std::string unsupported_surface_message(const char *rpc, const char *reason);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/capability_routing_test.cpp

@@ -0,0 +1,600 @@
+// Unit tests for capability_routing. Standard library only. The harness
+// compiles this as a single translation unit, so the implementation is
+// included directly rather than linked.
+
+#include "capability_routing.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// Mirrors what supertonic advertises: TTS offline and streaming.
+static Capabilities supertonic() {
+    return Capabilities{"supertonic",
+                        {{Task::Tts, {Mode::Offline, Mode::Streaming}}}};
+}
+
+// Mirrors chatterbox: TTS, cloning and voice conversion, offline only.
+static Capabilities chatterbox() {
+    return Capabilities{"chatterbox",
+                        {{Task::Tts, {Mode::Offline}},
+                         {Task::VoiceCloning, {Mode::Offline}},
+                         {Task::VoiceConversion, {Mode::Offline}}}};
+}
+
+// Mirrors nemotron_asr: ASR offline and streaming.
+static Capabilities nemotron() {
+    return Capabilities{"nemotron_asr",
+                        {{Task::Asr, {Mode::Offline, Mode::Streaming}}}};
+}
+
+// Mirrors qwen3_asr: ASR offline only.
+static Capabilities qwen3_asr() {
+    return Capabilities{"qwen3_asr", {{Task::Asr, {Mode::Offline}}}};
+}
+
+// Mirrors qwen3_forced_aligner: alignment only.
+static Capabilities aligner() {
+    return Capabilities{"qwen3_forced_aligner",
+                        {{Task::Alignment, {Mode::Offline}}}};
+}
+
+// Mirrors htdemucs: separation only.
+static Capabilities htdemucs() {
+    return Capabilities{"htdemucs", {{Task::SourceSeparation, {Mode::Offline}}}};
+}
+
+static void test_plain_tts() {
+    const auto r = resolve_route(Rpc::Tts, RequestShape{}, chatterbox());
+    check(r.ok, "plain TTS routes");
+    check(r.task == Task::Tts, "plain TTS picks Tts, not VoiceCloning");
+    check(r.mode == Mode::Offline, "TTS runs offline");
+}
+
+static void test_tts_with_voice_reference_prefers_cloning() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    const auto r = resolve_route(Rpc::Tts, shape, chatterbox());
+    check(r.ok, "TTS with a voice reference routes");
+    check(r.task == Task::VoiceCloning, "voice reference prefers VoiceCloning");
+}
+
+// supertonic has no VoiceCloning: a voice reference must fall back to Tts
+// rather than failing the request.
+static void test_tts_voice_reference_falls_back_to_tts() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    const auto r = resolve_route(Rpc::Tts, shape, supertonic());
+    check(r.ok, "voice reference on a clone-less family still routes");
+    check(r.task == Task::Tts, "falls back to Tts");
+}
+
+static void test_tts_instructions_prefer_voice_design() {
+    RequestShape shape;
+    shape.has_instructions = true;
+    Capabilities caps{"qwen3_tts",
+                      {{Task::Tts, {Mode::Offline}},
+                       {Task::VoiceDesign, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::Tts, shape, caps);
+    check(r.ok, "TTS with instructions routes");
+    check(r.task == Task::VoiceDesign, "instructions prefer VoiceDesign");
+}
+
+// A voice reference is a stronger signal than free-form instructions: cloning
+// a specific voice is what the user asked for.
+static void test_voice_reference_beats_instructions() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    shape.has_instructions = true;
+    Capabilities caps{"omnivoice",
+                      {{Task::Tts, {Mode::Offline}},
+                       {Task::VoiceCloning, {Mode::Offline}},
+                       {Task::VoiceDesign, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::Tts, shape, caps);
+    check(r.ok, "both signals present routes");
+    check(r.task == Task::VoiceCloning, "voice reference outranks instructions");
+}
+
+static void test_tts_stream_requires_streaming() {
+    const auto ok = resolve_route(Rpc::TtsStream, RequestShape{}, supertonic());
+    check(ok.ok, "streaming TTS routes on supertonic");
+    check(ok.mode == Mode::Streaming, "TTSStream runs in streaming mode");
+
+    const auto bad = resolve_route(Rpc::TtsStream, RequestShape{}, chatterbox());
+    check(!bad.ok, "streaming TTS is refused on an offline-only family");
+    check(bad.error.find("chatterbox") != std::string::npos,
+          "error names the family");
+    check(bad.error.find("tts/offline") != std::string::npos,
+          "error lists what the family does support");
+    check(bad.error.find("TTSStream") != std::string::npos,
+          "error names the RPC that was refused");
+    check(bad.error.find("tts/streaming") != std::string::npos,
+          "error lists the (task, mode) pairs that were tried");
+}
+
+static void test_transcription_stream_falls_back_to_offline() {
+    const auto streaming =
+        resolve_route(Rpc::AudioTranscriptionStream, RequestShape{}, nemotron());
+    check(streaming.ok && streaming.mode == Mode::Streaming,
+          "streaming ASR uses streaming mode when offered");
+
+    const auto offline =
+        resolve_route(Rpc::AudioTranscriptionStream, RequestShape{}, qwen3_asr());
+    check(offline.ok, "streaming ASR falls back on an offline-only family");
+    check(offline.mode == Mode::Offline, "fallback mode is offline");
+    check(offline.task == Task::Asr, "fallback task is still Asr");
+}
+
+// Task preference dominates mode preference: it is better to run the right
+// task in a fallback mode than the wrong task in the preferred mode. This is
+// the only RPC where the two orderings can disagree, because it is the only
+// one with more than one acceptable mode.
+static void test_task_preference_beats_mode_preference() {
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    Capabilities mixed{"mixed_asr_aligner",
+                       {{Task::Asr, {Mode::Offline}},
+                        {Task::Alignment, {Mode::Streaming}}}};
+    const auto r = resolve_route(Rpc::AudioTranscriptionStream, shape, mixed);
+    check(r.ok, "mixed family routes");
+    check(r.task == Task::Asr,
+          "the preferred task wins even in its fallback mode");
+    check(r.mode == Mode::Offline,
+          "the fallback mode is accepted to keep the preferred task");
+}
+
+// Live transcription is bidirectional and cannot be faked from an offline run.
+static void test_live_transcription_has_no_offline_fallback() {
+    const auto r =
+        resolve_route(Rpc::AudioTranscriptionLive, RequestShape{}, qwen3_asr());
+    check(!r.ok, "live transcription is refused on an offline-only family");
+}
+
+static void test_alignment_needs_prompt_text() {
+    const auto without =
+        resolve_route(Rpc::AudioTranscription, RequestShape{}, aligner());
+    check(!without.ok, "aligner without a transcript is refused");
+
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    const auto with = resolve_route(Rpc::AudioTranscription, shape, aligner());
+    check(with.ok, "aligner with a transcript routes");
+    check(with.task == Task::Alignment, "routes to Alignment");
+}
+
+// A real ASR family must not be hijacked to Alignment just because the caller
+// passed a prompt: `prompt` is also whisper-style decoding context.
+static void test_prompt_does_not_hijack_asr() {
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    const auto r = resolve_route(Rpc::AudioTranscription, shape, nemotron());
+    check(r.ok, "ASR with a prompt routes");
+
+    // nemotron advertises Asr alone, so the assertion has to be made against a
+    // family that advertises both: otherwise "Asr is preferred" only restates
+    // that Asr is the only option, and reversing the preference order passes.
+    Capabilities both{"asr_with_aligner",
+                      {{Task::Asr, {Mode::Offline}},
+                       {Task::Alignment, {Mode::Offline}}}};
+    const auto pref = resolve_route(Rpc::AudioTranscription, shape, both);
+    check(pref.ok, "a family offering both routes");
+    check(pref.task == Task::Asr, "Asr is preferred over Alignment");
+}
+
+static void test_audio_transform_prefers_separation() {
+    const auto sep =
+        resolve_route(Rpc::AudioTransform, RequestShape{}, htdemucs());
+    check(sep.ok && sep.task == Task::SourceSeparation, "separation routes");
+
+    // htdemucs advertises separation alone, so the check above cannot fail on
+    // ordering. This family advertises both, which is what pins the preference.
+    Capabilities sep_and_vc{"sep_and_vc",
+                            {{Task::SourceSeparation, {Mode::Offline}},
+                             {Task::VoiceConversion, {Mode::Offline}}}};
+    const auto pref =
+        resolve_route(Rpc::AudioTransform, RequestShape{}, sep_and_vc);
+    check(pref.ok && pref.task == Task::SourceSeparation,
+          "separation is preferred over voice conversion");
+
+    Capabilities miocodec{"miocodec",
+                          {{Task::VoiceConversion, {Mode::Offline}},
+                           {Task::SpeechToSpeech, {Mode::Offline}}}};
+    const auto vc = resolve_route(Rpc::AudioTransform, RequestShape{}, miocodec);
+    check(vc.ok && vc.task == Task::VoiceConversion,
+          "voice conversion is preferred over speech-to-speech");
+}
+
+static void test_pinned_task_overrides_routing() {
+    RequestShape shape;
+    shape.pinned_task = "s2s";
+    Capabilities miocodec{"miocodec",
+                          {{Task::VoiceConversion, {Mode::Offline}},
+                           {Task::SpeechToSpeech, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::AudioTransform, shape, miocodec);
+    check(r.ok && r.task == Task::SpeechToSpeech, "pinned task wins");
+
+    RequestShape bad;
+    bad.pinned_task = "not-a-task";
+    const auto e = resolve_route(Rpc::AudioTransform, bad, miocodec);
+    check(!e.ok, "an unknown pinned task is an error");
+    check(e.error.find("not-a-task") != std::string::npos,
+          "error names the bad task");
+
+    // A pinned task the family does not offer must fail, not silently reroute.
+    RequestShape unsupported;
+    unsupported.pinned_task = "sep";
+    const auto u = resolve_route(Rpc::AudioTransform, unsupported, miocodec);
+    check(!u.ok, "a pinned but unsupported task is refused");
+}
+
+// A pin lives on the MODEL, and every one of the nine handlers copies it into
+// the shape, so a pin set for one RPC arrives at all of them. It used to
+// replace the candidate list wholesale, which turned the other eight into wrong
+// 200s rather than errors: nemotron pinned to asr made Vad answer with zero
+// segments after a full ASR decode, and silero_vad pinned to vad made
+// AudioTranscription answer with empty text and four segments whose spans were
+// VAD segments, which the srt/vtt/lrc writers rendered as timed EMPTY cues.
+static void test_pin_must_be_admissible_for_the_rpc() {
+    Capabilities nemotron_asr{"nemotron_asr",
+                              {{Task::Asr, {Mode::Offline, Mode::Streaming}},
+                               {Task::Vad, {Mode::Offline}}}};
+
+    // The pin is legitimate on the RPC it was meant for.
+    RequestShape asr_pin;
+    asr_pin.pinned_task = "asr";
+    const auto transcription =
+        resolve_route(Rpc::AudioTranscription, asr_pin, nemotron_asr);
+    check(transcription.ok && transcription.task == Task::Asr,
+          "an admissible pin is still honoured exactly");
+
+    // ...and refused on one that never routes to it, EVEN THOUGH the family
+    // advertises the pinned task. That is the whole point: family support is
+    // not the question, RPC admissibility is.
+    const auto vad = resolve_route(Rpc::Vad, asr_pin, nemotron_asr);
+    check(!vad.ok, "an inadmissible pin is refused rather than served");
+    check(vad.error.find("asr") != std::string::npos,
+          "the refusal names the pinned task");
+    check(vad.error.find(rpc_name(Rpc::Vad)) != std::string::npos,
+          "the refusal names the RPC that cannot serve it");
+
+    Capabilities silero{"silero_vad", {{Task::Vad, {Mode::Offline}}}};
+    RequestShape vad_pin;
+    vad_pin.pinned_task = "vad";
+    const auto vad_ok = resolve_route(Rpc::Vad, vad_pin, silero);
+    check(vad_ok.ok && vad_ok.task == Task::Vad, "vad is admissible on Vad");
+    const auto transcribe_vad =
+        resolve_route(Rpc::AudioTranscription, vad_pin, silero);
+    check(!transcribe_vad.ok,
+          "a vad pin cannot make a transcription request return empty cues");
+
+    // Every pin a shipped configuration could sensibly set stays reachable on
+    // the RPC that serves it. This is the list the fix was checked against.
+    struct AdmissibleCase {
+        Rpc rpc;
+        const char *task;
+    };
+    const AdmissibleCase kAdmissible[] = {
+        {Rpc::AudioTransform, "svc"},   {Rpc::AudioTransform, "sep"},
+        {Rpc::AudioTransform, "vc"},    {Rpc::AudioTransform, "s2s"},
+        {Rpc::Tts, "tts"},              {Rpc::Tts, "clon"},
+        {Rpc::Tts, "vdes"},             {Rpc::TtsStream, "tts"},
+        {Rpc::AudioTranscription, "asr"},
+        {Rpc::AudioTranscription, "align"},
+        {Rpc::AudioTranscriptionStream, "asr"},
+        {Rpc::AudioTranscriptionLive, "asr"},
+        {Rpc::Vad, "vad"},              {Rpc::Diarize, "diar"},
+        {Rpc::SoundGeneration, "gen"},
+    };
+    for (const auto &entry : kAdmissible) {
+        Task task = Task::Tts;
+        check(parse_task_name(entry.task, task),
+              std::string("known task name: ") + entry.task);
+        // A family that advertises the pinned task offline and nothing else, so
+        // the ONLY thing that can refuse the route is the admissibility check.
+        Capabilities only{"probe",
+                          {{task, {Mode::Offline, Mode::Streaming}}}};
+        RequestShape pin;
+        pin.pinned_task = entry.task;
+        const auto route = resolve_route(entry.rpc, pin, only);
+        check(route.ok && route.task == task,
+              std::string("pin '") + entry.task + "' stays admissible on " +
+                  rpc_name(entry.rpc));
+    }
+
+    // And the pins that must NOT cross over, one per RPC pair that was
+    // observed producing a wrong 200.
+    const AdmissibleCase kInadmissible[] = {
+        {Rpc::Vad, "asr"},          {Rpc::Diarize, "asr"},
+        {Rpc::AudioTranscription, "vad"}, {Rpc::AudioTranscription, "diar"},
+        {Rpc::Tts, "asr"},          {Rpc::Vad, "tts"},
+        {Rpc::SoundGeneration, "tts"},    {Rpc::AudioTransform, "asr"},
+    };
+    for (const auto &entry : kInadmissible) {
+        Task task = Task::Tts;
+        check(parse_task_name(entry.task, task),
+              std::string("known task name: ") + entry.task);
+        Capabilities only{"probe",
+                          {{task, {Mode::Offline, Mode::Streaming}}}};
+        RequestShape pin;
+        pin.pinned_task = entry.task;
+        const auto route = resolve_route(entry.rpc, pin, only);
+        check(!route.ok,
+              std::string("pin '") + entry.task + "' is refused on " +
+                  rpc_name(entry.rpc));
+    }
+}
+
+static void test_vad_and_diarize() {
+    Capabilities silero{"silero_vad", {{Task::Vad, {Mode::Offline, Mode::Streaming}}}};
+    const auto v = resolve_route(Rpc::Vad, RequestShape{}, silero);
+    check(v.ok && v.task == Task::Vad && v.mode == Mode::Offline, "VAD routes offline");
+
+    const auto d = resolve_route(Rpc::Diarize, RequestShape{}, silero);
+    check(!d.ok, "diarization is refused on a VAD-only family");
+
+    Capabilities sortformer{"sortformer_diar", {{Task::Diarization, {Mode::Offline}}}};
+    const auto ok = resolve_route(Rpc::Diarize, RequestShape{}, sortformer);
+    check(ok.ok && ok.task == Task::Diarization, "diarization routes");
+}
+
+static void test_sound_generation() {
+    Capabilities stable{"stable_audio", {{Task::AudioGeneration, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::SoundGeneration, RequestShape{}, stable);
+    check(r.ok && r.task == Task::AudioGeneration, "sound generation routes");
+}
+
+static void test_names_round_trip() {
+    const Task all[] = {Task::Vad, Task::Asr, Task::Diarization,
+                        Task::SourceSeparation, Task::AudioGeneration, Task::Tts,
+                        Task::VoiceCloning, Task::VoiceConversion,
+                        Task::SpeechToSpeech, Task::Alignment, Task::VoiceDesign,
+                        Task::SpeakerRecognition, Task::Svc};
+    for (const Task t : all) {
+        Task parsed = Task::Vad;
+        const bool ok = parse_task_name(task_name(t), parsed);
+        check(ok && parsed == t,
+              std::string("task name round-trips: ") + task_name(t));
+    }
+    check(std::string(mode_name(Mode::Offline)) == "offline", "offline name");
+    check(std::string(mode_name(Mode::Streaming)) == "streaming", "streaming name");
+
+    // The emitted name must be the one audio.cpp itself prints and parses
+    // (framework/runtime/session.cpp), because `task:` is user-facing: a name
+    // copied out of audio.cpp has to be accepted here, and a name pinned here
+    // has to survive conversion at the engine boundary.
+    check(std::string(task_name(Task::SpeakerRecognition)) == "spk",
+          "speaker recognition emits upstream's name 'spk'");
+
+    Task pinned = Task::Vad;
+    check(parse_task_name("spk", pinned) && pinned == Task::SpeakerRecognition,
+          "'spk' parses to SpeakerRecognition");
+
+    // Accepted as a legacy alias so configs written against the earlier name
+    // keep working, but never emitted.
+    Task alias = Task::Vad;
+    check(parse_task_name("spkrec", alias) && alias == Task::SpeakerRecognition,
+          "'spkrec' is still accepted as an alias");
+}
+
+static void test_describe_capabilities() {
+    const std::string described = describe_capabilities(nemotron());
+    check(described.find("asr/offline") != std::string::npos,
+          "description lists asr/offline");
+    check(described.find("asr/streaming") != std::string::npos,
+          "description lists asr/streaming");
+}
+
+static void test_empty_capabilities() {
+    const auto r = resolve_route(Rpc::Tts, RequestShape{}, Capabilities{"mystery", {}});
+    check(!r.ok, "a family advertising nothing is refused");
+    check(r.error.find("mystery") != std::string::npos, "error names the family");
+}
+
+// The table is indexed by UnsupportedRpc's underlying value, so a reordering of
+// either list silently pairs an RPC with another's reason. Nothing else would
+// catch that: both sides still compile and every message still reads plausibly.
+static void test_unsupported_surface_table_matches_the_enum() {
+    check(unsupported_surfaces().size() == 5,
+          "all five unsupported surfaces are tabulated");
+    check(std::string(unsupported_surface(UnsupportedRpc::AudioEncode).rpc) ==
+              "AudioEncode",
+          "UnsupportedRpc::AudioEncode indexes AudioEncode");
+    check(std::string(unsupported_surface(UnsupportedRpc::AudioDecode).rpc) ==
+              "AudioDecode",
+          "UnsupportedRpc::AudioDecode indexes AudioDecode");
+    check(std::string(
+              unsupported_surface(UnsupportedRpc::AudioTransformStream).rpc) ==
+              "AudioTransformStream",
+          "UnsupportedRpc::AudioTransformStream indexes AudioTransformStream");
+    check(std::string(
+              unsupported_surface(UnsupportedRpc::AudioToAudioStream).rpc) ==
+              "AudioToAudioStream",
+          "UnsupportedRpc::AudioToAudioStream indexes AudioToAudioStream");
+    check(std::string(unsupported_surface(UnsupportedRpc::VoiceEmbed).rpc) ==
+              "VoiceEmbed",
+          "UnsupportedRpc::VoiceEmbed indexes VoiceEmbed");
+
+    // Two entries may share a reason (the codec pair does), but two entries
+    // naming the same RPC would mean one of the five is unreachable.
+    for (size_t i = 0; i < unsupported_surfaces().size(); ++i) {
+        for (size_t j = i + 1; j < unsupported_surfaces().size(); ++j) {
+            check(std::string(unsupported_surfaces()[i].rpc) !=
+                      unsupported_surfaces()[j].rpc,
+                  std::string("no duplicate RPC name at ") + std::to_string(i) +
+                      "/" + std::to_string(j));
+        }
+    }
+}
+
+// There is no out-of-range test for unsupported_surface(). It switches over the
+// enumerators with no default label, so a sixth UnsupportedRpc without a case is
+// a -Wswitch diagnostic at build time and cannot reach a run-time check at all.
+
+// Every entry, not just the one somebody remembered to cover. A refusal that
+// drops the family, the RPC or the reason is a refusal the caller cannot act
+// on, which is the entire point of this surface existing.
+static void test_every_unsupported_surface_message_is_diagnosable() {
+    for (const auto &surface : unsupported_surfaces()) {
+        const std::string label = std::string(" [") + surface.rpc + "]";
+        const std::string loaded =
+            unsupported_surface_message(nemotron(), surface.rpc, surface.reason);
+
+        check(loaded.find(surface.rpc) != std::string::npos,
+              "message names the RPC" + label);
+        check(std::string(surface.reason).size() > 20 &&
+                  loaded.find(surface.reason) != std::string::npos,
+              "message gives a substantive upstream reason" + label);
+        check(loaded.find("nemotron_asr") != std::string::npos,
+              "message names the loaded family" + label);
+        check(loaded.find("asr/offline") != std::string::npos &&
+                  loaded.find("asr/streaming") != std::string::npos,
+              "message lists what the family does support" + label);
+
+        // The no-model form keeps the two facts that do not depend on a model
+        // and drops only the one that does, so the caller still learns why.
+        const std::string unloaded =
+            unsupported_surface_message(surface.rpc, surface.reason);
+        check(unloaded.find(surface.rpc) != std::string::npos,
+              "no-model message names the RPC" + label);
+        check(unloaded.find(surface.reason) != std::string::npos,
+              "no-model message gives the upstream reason" + label);
+        check(unloaded.find("nemotron_asr") == std::string::npos,
+              "no-model message names no family" + label);
+        // Without this the caller's obvious next move is to load a model and
+        // retry, which cannot work: the refusal is a property of the engine.
+        check(unloaded.find("would not change this answer") != std::string::npos,
+              "no-model message says loading a model would not help" + label);
+    }
+}
+
+// The reasons are the load-bearing half of this feature and each was checked
+// against the pinned upstream checkout. Pinning the distinguishing phrase here
+// means a later edit that guts one into a generic "not supported" fails rather
+// than passes quietly.
+static void test_unsupported_reasons_name_the_upstream_limitation() {
+    const auto &encode = unsupported_surface(UnsupportedRpc::AudioEncode);
+    const auto &decode = unsupported_surface(UnsupportedRpc::AudioDecode);
+    check(std::string(encode.reason).find("VoiceTaskKind") != std::string::npos &&
+              std::string(encode.reason).find("codec") != std::string::npos,
+          "the AudioEncode reason names the missing VoiceTaskKind entry");
+    // miocodec is the family a reader will reach for first, because upstream's
+    // README tags it Codec. Naming it and its actual advertised tasks is what
+    // stops the next person re-deriving the same dead end.
+    check(std::string(encode.reason).find("miocodec") != std::string::npos,
+          "the AudioEncode reason disposes of miocodec's README Codec tag");
+    check(std::string(encode.reason) == decode.reason,
+          "AudioEncode and AudioDecode refuse for the same reason");
+
+    const auto &transform =
+        unsupported_surface(UnsupportedRpc::AudioTransformStream);
+    // The reason must be scoped to the tasks AudioTransform routes to. The
+    // broader claim, "upstream streams tts and asr only", is FALSE: silero_vad
+    // advertises vad with RunMode::Streaming. A refusal resting on a false
+    // premise is worse than a bare UNIMPLEMENTED, because it will be believed.
+    check(std::string(transform.reason).find("sep, vc, svc, s2s") !=
+              std::string::npos,
+          "the AudioTransformStream reason is scoped to the routed tasks");
+    check(std::string(transform.reason).find("tts, asr and vad") !=
+              std::string::npos,
+          "the AudioTransformStream reason counts vad among the streaming tasks");
+    check(std::string(transform.reason).find("tts and asr only") ==
+              std::string::npos,
+          "the AudioTransformStream reason does not repeat the refuted claim");
+    // An absence is not an impossibility. A sep family could be buffered and
+    // emitted as a stream, so the reason has to say this backend declines to
+    // rather than cannot, or it overreaches on a true premise.
+    check(std::string(transform.reason).find("buffered offline call in disguise") !=
+              std::string::npos,
+          "the AudioTransformStream reason does not overclaim impossibility");
+
+    const auto &s2s = unsupported_surface(UnsupportedRpc::AudioToAudioStream);
+    check(std::string(s2s.reason).find("Realtime") != std::string::npos &&
+              std::string(s2s.reason).find("clip-to-clip") != std::string::npos,
+          "the AudioToAudioStream reason contrasts the two contracts");
+    // Naming both s2s families, and what each of them actually does, makes the
+    // claim checkable. It must NOT say s2s is voice conversion full stop: that
+    // is true of miocodec and false of vevo2, whose s2s route is `editing` and
+    // rewrites the spoken content against a target voice.
+    check(std::string(s2s.reason).find("miocodec (voice conversion)") !=
+                  std::string::npos &&
+              std::string(s2s.reason).find("vevo2 (speech editing)") !=
+                  std::string::npos,
+          "the AudioToAudioStream reason names each s2s family's actual task");
+    check(std::string(s2s.reason).find("s2s is offline voice conversion") ==
+              std::string::npos,
+          "the AudioToAudioStream reason does not miscast vevo2 as conversion");
+
+    const auto &embed = unsupported_surface(UnsupportedRpc::VoiceEmbed);
+    check(std::string(embed.reason).find("spk") != std::string::npos,
+          "the VoiceEmbed reason names the task no family advertises");
+    // spk IS a VoiceTaskKind upstream; what is missing is any family that
+    // advertises it. Saying the kind does not exist would be false, and would
+    // send a reader looking in the wrong place.
+    check(std::string(embed.reason).find("no audio.cpp family") !=
+              std::string::npos,
+          "the VoiceEmbed reason blames the families, not the enum");
+    // The speaker encoders DO exist upstream, as conditioning modules inside
+    // TTS and VC families. Not saying so invites "but audio.cpp ships TitaNet".
+    check(std::string(embed.reason).find("TitaNet") != std::string::npos,
+          "the VoiceEmbed reason disposes of the internal speaker encoders");
+    Task parsed = Task::Vad;
+    check(parse_task_name("spk", parsed) && parsed == Task::SpeakerRecognition,
+          "spk is a real task kind, so the reason must not claim otherwise");
+}
+
+// A family advertising nothing still gets a message that reads, rather than one
+// trailing off after "supports: ".
+static void test_unsupported_surface_message_with_empty_capabilities() {
+    const auto &embed = unsupported_surface(UnsupportedRpc::VoiceEmbed);
+    const std::string message = unsupported_surface_message(
+        Capabilities{"mystery", {}}, embed.rpc, embed.reason);
+    check(message.find("mystery") != std::string::npos,
+          "empty-capability message still names the family");
+    check(message.find("supports: nothing") != std::string::npos,
+          "empty-capability message says the family supports nothing");
+}
+
+int main() {
+    test_plain_tts();
+    test_tts_with_voice_reference_prefers_cloning();
+    test_tts_voice_reference_falls_back_to_tts();
+    test_tts_instructions_prefer_voice_design();
+    test_voice_reference_beats_instructions();
+    test_tts_stream_requires_streaming();
+    test_transcription_stream_falls_back_to_offline();
+    test_task_preference_beats_mode_preference();
+    test_live_transcription_has_no_offline_fallback();
+    test_alignment_needs_prompt_text();
+    test_prompt_does_not_hijack_asr();
+    test_audio_transform_prefers_separation();
+    test_pinned_task_overrides_routing();
+    test_pin_must_be_admissible_for_the_rpc();
+    test_vad_and_diarize();
+    test_sound_generation();
+    test_names_round_trip();
+    test_describe_capabilities();
+    test_empty_capabilities();
+    test_unsupported_surface_table_matches_the_enum();
+    test_every_unsupported_surface_message_is_diagnosable();
+    test_unsupported_reasons_name_the_upstream_limitation();
+    test_unsupported_surface_message_with_empty_capabilities();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all capability_routing checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/family_gate.cpp

@@ -0,0 +1,163 @@
+#include "family_gate.h"
+
+#include <cctype>
+#include <cstddef>
+
+namespace audiocpp_backend {
+namespace {
+
+// PRECONDITION: `suffix` must already be lowercase. Both sides are folded, so
+// this reads as symmetric, but only `value` can carry case in practice and a
+// caller passing ".GGUF" would still work today for that reason alone. Do not
+// rely on it: the fold on the suffix side is the only thing standing between
+// this and a helper that answers false for every input, and it is not covered
+// by any test, because with a lowercase suffix no input can distinguish it.
+bool ends_with_ci(const std::string &value, const std::string &suffix) {
+    if (value.size() <= suffix.size()) {
+        return false; // a bare ".gguf" is an extension, not a model file
+    }
+    const size_t offset = value.size() - suffix.size();
+    for (size_t i = 0; i < suffix.size(); ++i) {
+        const auto lhs = static_cast<unsigned char>(value[offset + i]);
+        const auto rhs = static_cast<unsigned char>(suffix[i]);
+        if (std::tolower(lhs) != std::tolower(rhs)) {
+            return false;
+        }
+    }
+    return true;
+}
+
+} // namespace
+
+bool path_looks_like_gguf(const std::string &path) {
+    return ends_with_ci(path, ".gguf");
+}
+
+FamilyDecision decide_family(bool path_is_gguf, const std::string &embedded_family,
+                             const std::string &configured_family) {
+    FamilyDecision decision;
+
+    if (!configured_family.empty()) {
+        decision.ok = true;
+        decision.family = configured_family;
+        return decision;
+    }
+
+    if (path_is_gguf) {
+        if (!embedded_family.empty()) {
+            decision.ok = true;
+            decision.family = embedded_family;
+            return decision;
+        }
+        decision.error =
+            "audio-cpp: this GGUF carries no 'audiocpp.model_spec.family' "
+            "metadata key, so it is not an audio.cpp model. Convert it with "
+            "audiocpp_gguf, or name the family explicitly with the model option "
+            "'family:<name>'";
+        return decision;
+    }
+
+    decision.error =
+        "audio-cpp: a model path that is not a standalone audio.cpp GGUF needs "
+        "an explicit 'family:<name>' model option, because the audio.cpp family "
+        "cannot be inferred from a safetensors or package directory";
+    return decision;
+}
+
+namespace {
+
+// Families that ABORT THE PROCESS on a weight dtype they cannot handle, and the
+// dtypes they can. See the header for why this is a list of crashes rather than
+// a list of preferences.
+//
+// supertonic: upstream's docs/gguf.md:90 records its 16-bit GGUF column as
+// "---", i.e. NOT TESTED, and its q8_0 as "No (unsupported weight dtype)". Only
+// the `orig` package is marked Pass, and its 698 weight tensors are f32 while
+// its 72 index and shape constants are i64. The f16 abort is a LOCAL
+// OBSERVATION rather than an upstream claim, and it is attributed rather than
+// assumed: it is identical through the unary TTS RPC and through TTSStream, so
+// it is the packaging and not the streaming path. q8_0 was never run here and is
+// refused on upstream's "unsupported weight dtype" alone, which is the weaker of
+// the two claims. See the header for why keeping them apart matters.
+//
+// TO REMOVE AN ENTRY: bump AUDIO_CPP_VERSION past a fix, load a package in the
+// refused dtype, and synthesise. If audio comes out, delete the entry. No test
+// can do that for you, which is exactly why it is written here: the test beside
+// this file pins WHAT the table says, not whether upstream has moved on. Do not
+// widen an entry without running that, because what it prevents is a process
+// death rather than a wrong answer.
+struct DtypeAllowList {
+    // NULL TERMINATED, and the terminator occupies one of these slots: both
+    // loops below stop at the first nullptr and have no other bound, so an entry
+    // that named three dtypes would leave them reading past the end of the
+    // array. That is undefined behaviour rather than a wrong answer, and it is
+    // one keystroke away from any edit that widens an entry, so the terminator
+    // is asserted at compile time below rather than trusted.
+    static constexpr std::size_t kSlots = 3;
+
+    const char *family;
+    const char *allowed[kSlots];
+};
+
+constexpr DtypeAllowList kDtypeAllowLists[] = {
+    {"supertonic", {"f32", "i64", nullptr}},
+};
+
+constexpr bool allow_lists_are_terminated() {
+    for (const auto &entry : kDtypeAllowLists) {
+        if (entry.allowed[DtypeAllowList::kSlots - 1] != nullptr) {
+            return false;
+        }
+    }
+    return true;
+}
+
+static_assert(allow_lists_are_terminated(),
+              "every DtypeAllowList must leave its last slot null: the lookups "
+              "below stop at the first nullptr and would otherwise read past "
+              "the end of the array");
+
+const DtypeAllowList *find_allow_list(const std::string &family) {
+    for (const auto &entry : kDtypeAllowLists) {
+        if (family == entry.family) {
+            return &entry;
+        }
+    }
+    return nullptr;
+}
+
+} // namespace
+
+bool family_has_weight_dtype_allow_list(const std::string &family) {
+    return find_allow_list(family) != nullptr;
+}
+
+bool weight_dtype_is_supported(const std::string &family, const std::string &dtype) {
+    const DtypeAllowList *list = find_allow_list(family);
+    if (list == nullptr) {
+        return true;
+    }
+    for (const char *const *name = list->allowed; *name != nullptr; ++name) {
+        if (dtype == *name) {
+            return true;
+        }
+    }
+    return false;
+}
+
+std::string supported_weight_dtypes(const std::string &family) {
+    const DtypeAllowList *list = find_allow_list(family);
+    if (list == nullptr) {
+        return {};
+    }
+    std::string out;
+    for (const char *const *name = list->allowed; *name != nullptr; ++name) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += *name;
+    }
+    return out;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/family_gate.h

@@ -0,0 +1,86 @@
+#pragma once
+
+// Decides which audio.cpp family a model path belongs to, and refuses paths
+// this backend must not claim. Standard library only.
+//
+// This is the guard against issue #9287. A model config with no explicit
+// backend makes LocalAI probe every installed backend and bind to the first
+// Load that succeeds, so accepting an arbitrary GGUF here would capture
+// unrelated LLMs. audio.cpp GGUFs carry an audiocpp.model_spec.family metadata
+// key; llama.cpp GGUFs do not.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+// True when the path ends in ".gguf", case insensitively, and has a stem.
+bool path_looks_like_gguf(const std::string &path);
+
+struct FamilyDecision {
+    bool ok = false;
+    std::string family;
+    // Set when ok is false. Suitable verbatim as an INVALID_ARGUMENT message.
+    std::string error;
+};
+
+// Precedence:
+//   1. an explicit `family:` option, so a user can override wrong metadata;
+//   2. for a GGUF, the family embedded in audiocpp.model_spec.family;
+//   3. otherwise refuse.
+// A directory path never consults embedded metadata: there is no single GGUF
+// to read it from.
+FamilyDecision decide_family(bool path_is_gguf, const std::string &embedded_family,
+                             const std::string &configured_family);
+
+// True when `family` can run weights stored as `dtype`, where dtype is the
+// string a TensorMetadata carries ("f32", "f16", "q8_0", "i64", ...).
+//
+// This is a LIST OF FAMILIES THAT CRASH THE PROCESS, not a list of families that
+// perform badly. It exists because the failure is not an exception: loading the
+// supertonic f16 GGUF package reaches ggml_concat with one f16 operand and one
+// f32 one, GGML_ASSERT(a->type == b->type) fails (external/ggml/src/ggml.c:2595)
+// and ggml_abort takes the backend down with SIGABRT on the FIRST request.
+// Nothing upstream of the load can catch that, so an operator sees a model that
+// loaded successfully and a backend that dies on every request with no status
+// and no message.
+//
+// EVIDENCE, per dtype, because the two are not equally attested:
+//   - f16 was OBSERVED to abort here, identically through the unary TTS RPC and
+//     through TTSStream, so it is the packaging and not the streaming path.
+//     Upstream's docs/gguf.md:90 has supertonic's 16-bit column as "---", which
+//     its own legend (:53) defines as not tested, so upstream neither confirms
+//     nor contradicts it.
+//   - q8_0 was NOT run here. Upstream records it as "No (unsupported weight
+//     dtype)" in the same row, which is a weaker claim than the f16 abort: it
+//     says the format is unusable, not that it takes the process down.
+// Both are refused, because the allow list is what the family CAN run (f32 for
+// weights, i64 for the shape and index constants) rather than a list of the
+// dtypes that fail, and a format upstream calls unusable has no business being
+// loaded either way.
+//
+// A family with no entry is unrestricted, which is every family but one.
+//
+// Split out of loaded_model.cpp, where the caller lives, so that the policy is
+// stdlib-only and can be held by a test: the caller needs a real GGUF on disk
+// and an engine, and neither is available to a unit test. What the test pins is
+// that the table says what it is meant to say, so widening it is a deliberate
+// act rather than a typo. It CANNOT pin the removal criterion, which is
+// "upstream fixed it": no test can know that without downloading the package and
+// synthesising, so that step stays a documented manual one at the table itself.
+bool weight_dtype_is_supported(const std::string &family, const std::string &dtype);
+
+// True when `family` has an entry in the table at all, which is the question a
+// caller deciding whether to OPEN THE FILE has to ask. Distinct from
+// "supported_weight_dtypes(family) is empty": that string is also empty for an
+// entry with an empty allow list, and such an entry means "this family can run
+// nothing", which weight_dtype_is_supported already answers by refusing every
+// dtype. Deciding from the string would skip the check on precisely the entry
+// that most needs it.
+bool family_has_weight_dtype_allow_list(const std::string &family);
+
+// The dtypes `family` is restricted to, as "f32, i64", or empty when it is not
+// restricted at all. For the refusal message, so the operator is told what to
+// look for rather than only what is wrong.
+std::string supported_weight_dtypes(const std::string &family);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/family_gate_test.cpp

@@ -0,0 +1,179 @@
+// Unit tests for family_gate. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+//
+// This unit is the guard against issue #9287: when a model config has no
+// explicit backend, LocalAI probes every installed backend and binds to the
+// first Load that succeeds. Accepting an arbitrary GGUF here would capture
+// unrelated LLMs.
+
+#include "family_gate.cpp"
+
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+using namespace audiocpp_backend;
+
+static void test_gguf_suffix_detection() {
+    check(path_looks_like_gguf("/models/chatterbox-q8_0.gguf"), "plain .gguf");
+    check(path_looks_like_gguf("/models/CHATTERBOX.GGUF"), "uppercase .GGUF");
+    check(path_looks_like_gguf("/models/x.GgUf"), "mixed case .GgUf");
+    check(path_looks_like_gguf("a.gguf"), "a one character stem is still a stem");
+    check(!path_looks_like_gguf("/models/chatterbox"), "extensionless directory");
+    check(!path_looks_like_gguf("/models/model.safetensors"), "safetensors");
+    check(!path_looks_like_gguf("/models/gguf"), "a name that is merely 'gguf'");
+    check(!path_looks_like_gguf("/models/GGUF"), "an uppercase name that is merely 'GGUF'");
+    check(!path_looks_like_gguf(""), "empty path");
+    check(!path_looks_like_gguf(".gguf"), "a bare extension is not a model file");
+    // The suffix has to be at the end. A prefix or infix match would let
+    // ".gguf.tmp" download artefacts and ".ggufx" siblings through.
+    check(!path_looks_like_gguf("/models/model.gguf.tmp"), ".gguf in the middle");
+    check(!path_looks_like_gguf("/models/model.ggufx"), "a longer extension");
+    // Every character of the suffix has to match, including the last one.
+    check(!path_looks_like_gguf("/models/model.ggug"), "a near miss in the final character");
+    check(!path_looks_like_gguf("/models/model_gguf"), "a near miss in the first character");
+    check(!path_looks_like_gguf("/models/.gguf-notes"), "a leading .gguf");
+}
+
+static void test_explicit_family_always_wins() {
+    // Explicit configuration beats metadata, so a user can force a family when
+    // upstream metadata is wrong or absent.
+    const auto gguf = decide_family(true, "chatterbox", "omnivoice");
+    check(gguf.ok && gguf.family == "omnivoice", "explicit family overrides GGUF metadata");
+    check(gguf.error.empty(), "an accepted decision carries no error text");
+
+    const auto dir = decide_family(false, "", "qwen3_tts");
+    check(dir.ok && dir.family == "qwen3_tts", "explicit family satisfies a directory path");
+
+    // A GGUF with no embedded spec is still loadable when the user names the
+    // family: the option is an override, not a tie-break that needs metadata to
+    // break against.
+    const auto bare = decide_family(true, "", "supertonic");
+    check(bare.ok && bare.family == "supertonic",
+          "explicit family rescues a GGUF that carries no spec");
+}
+
+static void test_gguf_metadata_supplies_the_family() {
+    const auto d = decide_family(true, "nemotron_asr", "");
+    check(d.ok, "an audio.cpp GGUF loads with no family option");
+    check(d.family == "nemotron_asr", "family comes from the embedded spec");
+    check(d.error.empty(), "an accepted GGUF carries no error text");
+}
+
+// THE GATE. A llama.cpp GGUF has no audiocpp.model_spec.family key.
+static void test_foreign_gguf_is_refused() {
+    const auto d = decide_family(true, "", "");
+    check(!d.ok, "a GGUF with no audio.cpp spec is refused");
+    check(d.family.empty(), "no family is guessed");
+    check(!d.error.empty(), "a refusal always says why");
+    check(d.error.find("audiocpp.model_spec.family") != std::string::npos,
+          "error names the missing metadata key so the cause is diagnosable");
+    check(d.error.find("family:") != std::string::npos,
+          "error names the option that would override it");
+}
+
+static void test_directory_without_family_is_refused() {
+    const auto d = decide_family(false, "", "");
+    check(!d.ok, "a non-GGUF path with no family option is refused");
+    check(d.family.empty(), "a refused directory guesses no family");
+    check(d.error.find("family:") != std::string::npos,
+          "error names the required option");
+}
+
+// A directory path never consults embedded metadata, because there is no single
+// GGUF to read it from.
+static void test_directory_ignores_embedded_family() {
+    const auto d = decide_family(false, "chatterbox", "");
+    check(!d.ok, "a directory is refused even when an embedded family is supplied");
+    check(d.family.empty(), "a refused directory does not adopt the embedded family");
+    // If the GGUF branch ever leaked into the directory branch this message
+    // would start blaming a metadata key that a directory has no place to carry.
+    check(d.error.find("audiocpp.model_spec.family") == std::string::npos,
+          "a directory refusal does not blame GGUF metadata it could not have");
+}
+
+// Pins the weight-dtype allow list. Not a style preference: an entry here is a
+// family that ABORTS THE PROCESS on the first request when handed the wrong
+// dtype, so the model loads and then every request kills the backend with no
+// status and no message.
+//
+// What this test can and cannot do, stated so the next reader does not expect
+// more of it: it pins WHAT THE TABLE SAYS, so widening an entry is a deliberate
+// act rather than a typo, and it pins that an unlisted family is unrestricted.
+// It CANNOT pin the removal criterion, which is "upstream fixed it": knowing
+// that needs the package downloaded and a synthesis run, so it stays a manual
+// step documented at the table in family_gate.cpp.
+static void test_weight_dtype_allow_list() {
+    // The entry that exists, and the exact reason it exists.
+    check(!weight_dtype_is_supported("supertonic", "f16"),
+          "supertonic refuses f16, the package that aborts the process");
+    check(!weight_dtype_is_supported("supertonic", "q8_0"),
+          "supertonic refuses q8_0, which upstream records as unsupported");
+    check(!weight_dtype_is_supported("supertonic", "bf16"),
+          "supertonic refuses bf16, which is untested rather than known good");
+    check(weight_dtype_is_supported("supertonic", "f32"),
+          "supertonic accepts f32, which is what the orig package stores");
+    check(weight_dtype_is_supported("supertonic", "i64"),
+          "supertonic accepts i64: the orig package carries 72 such tensors and "
+          "refusing them would refuse the artifact that works");
+
+    // Every other family is unrestricted, and must stay that way: this guard is
+    // for process death, not for quality.
+    check(weight_dtype_is_supported("nemotron_asr", "q8_0"),
+          "an unlisted family is not restricted");
+    check(weight_dtype_is_supported("citrinet_asr", "f16"),
+          "an unlisted family is not restricted by another family's entry");
+    check(weight_dtype_is_supported("", "anything"),
+          "an empty family name is not restricted");
+
+    // The message the operator reads has to name the remedy, so the refusal is
+    // actionable rather than only correct.
+    check_eq(supported_weight_dtypes("supertonic"), "f32, i64",
+             "the refusal can name what to look for");
+    check_eq(supported_weight_dtypes("nemotron_asr"), "",
+             "an unlisted family reports no restriction");
+
+    // What the caller actually decides on, and it is a DIFFERENT question from
+    // "is the description empty": an entry with an empty allow list would
+    // describe itself as "" while refusing every dtype, so a caller that skipped
+    // the file read on the empty string would skip the check on the one entry
+    // that refuses everything.
+    check(family_has_weight_dtype_allow_list("supertonic"),
+          "a listed family has an allow list");
+    check(!family_has_weight_dtype_allow_list("nemotron_asr"),
+          "an unlisted family has none, which is what lets the caller skip "
+          "opening the file at all");
+    check(!family_has_weight_dtype_allow_list(""),
+          "an empty family name has no allow list");
+}
+
+int main() {
+    test_gguf_suffix_detection();
+    test_explicit_family_always_wins();
+    test_gguf_metadata_supplies_the_family();
+    test_foreign_gguf_is_refused();
+    test_directory_without_family_is_refused();
+    test_directory_ignores_embedded_family();
+    test_weight_dtype_allow_list();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all family_gate checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/generation_request.cpp

@@ -0,0 +1,280 @@
+#include "generation_request.h"
+
+#include <filesystem>
+#include <string>
+#include <system_error>
+#include <utility>
+
+namespace audiocpp_backend {
+namespace {
+
+const char *bool_option(bool value) { return value ? "true" : "false"; }
+
+} // namespace
+
+bool voice_is_reference_file(const std::string &voice) {
+    if (voice.empty()) {
+        return false;
+    }
+    std::error_code ec;
+    return std::filesystem::is_regular_file(std::filesystem::path(voice), ec);
+}
+
+RequestShape build_tts_shape(const backend::TTSRequest &request) {
+    RequestShape shape;
+    shape.has_voice_reference = voice_is_reference_file(request.voice());
+    // !empty() as well as has_instructions(), and it must match the guard in
+    // build_tts_request: a request whose instructions are an empty string
+    // carries no style condition, so telling routing to prefer VoiceDesign for
+    // it would route to a task with nothing to design from.
+    shape.has_instructions =
+        request.has_instructions() && !request.instructions().empty();
+    return shape;
+}
+
+engine::runtime::TaskRequest
+build_tts_request(const backend::TTSRequest &request,
+                  std::optional<engine::runtime::AudioBuffer> reference_audio) {
+    engine::runtime::TaskRequest task;
+
+    // The Transcript, not an option, is where every TTS family reads its
+    // language: chatterbox normalises request.text_input->language into its
+    // voice-clone config, qwen3_tts reads it as out.language, ace_step turns it
+    // into vocal_language. Upstream's own HTTP server does the same and sets no
+    // language option at all (app/server/runtime.cpp build_speech_request).
+    engine::runtime::Transcript transcript;
+    transcript.text = request.text();
+    if (request.has_language()) {
+        transcript.language = request.language();
+    }
+    task.text_input = std::move(transcript);
+
+    engine::runtime::VoiceCondition condition;
+    bool condition_used = false;
+
+    if (reference_audio.has_value()) {
+        // A clip: VoiceReference::audio, at the file's own rate and channel
+        // count. See kVoiceReferenceSampleRate in grpc-server.cpp for why it is
+        // not folded first.
+        engine::runtime::VoiceReference reference;
+        reference.audio = std::move(*reference_audio);
+        condition.speaker = std::move(reference);
+        condition_used = true;
+    } else if (!request.voice().empty()) {
+        // A named preset. cached_voice_id is the channel that actually lands:
+        // supertonic (options.voice), pocket_tts (voice_config.preset_name),
+        // voxcpm2, vibevoice, fish_audio (its saved-reference lookup) and
+        // qwen3_tts CustomVoice all read request.voice->speaker->cached_voice_id,
+        // and upstream's own server puts a non-preset `voice` body field in
+        // exactly this slot (app/server/runtime.cpp build_speech_request).
+        engine::runtime::VoiceReference reference;
+        reference.cached_voice_id = request.voice();
+        condition.speaker = std::move(reference);
+        condition_used = true;
+        // Forward-tolerant alias only. A bare "voice" REQUEST OPTION is read by
+        // no family in the pinned upstream: grepping find_option for it returns
+        // nothing. It is sent so a family adopting the name later works with no
+        // change here, not because it does anything today.
+        task.options["voice"] = request.voice();
+    }
+
+    if (request.has_instructions() && !request.instructions().empty()) {
+        // "instruct" is the key upstream itself maps the OpenAI `instructions`
+        // body field onto (app/server/runtime.cpp: request.options["instruct"]
+        // = value->as_string()), and it is read: qwen3_tts VoiceDesign and
+        // CustomVoice both take find_option(options, {"instruct"}) first, and
+        // omnivoice reads it in resolve_instruct.
+        task.options["instruct"] = request.instructions();
+        // "caption" is irodori_tts's name for the same thing, read in its
+        // make_request and documented as the voice-design caption for the 600M
+        // VoiceDesign model (docs/tts.md). Without this, that family's voice
+        // design cannot be driven from this RPC at all.
+        task.options["caption"] = request.instructions();
+        // The proto field's own name, forwarded for the same forward-tolerant
+        // reason as "voice" above and with the same honest accounting: NO family
+        // in the pinned upstream reads a request option called "instructions".
+        task.options["instructions"] = request.instructions();
+
+        engine::runtime::StyleCondition style;
+        // "instruct", not "instructions". This tag IS read, and only under that
+        // spelling: omnivoice and qwen3_tts both fall back to
+        // request.voice->style->tags.find("instruct") when the option is absent.
+        // Spelling it "instructions" here would have made the whole
+        // StyleCondition dead weight.
+        style.tags["instruct"] = request.instructions();
+        // !empty(), matching the option emission below, and load-bearing rather
+        // than tidiness. core/backend/tts.go's newTTSRequest sets
+        // `Language: &language` UNCONDITIONALLY, so has_language() is true on
+        // every request LocalAI sends and carries "" whenever the caller named
+        // no language. An engaged-but-empty style language is WORSE than an
+        // absent one: supertonic reads text_input->language behind its own
+        // !empty() guard and then OVERRIDES it from style->language with no
+        // guard at all (supertonic/session.cpp), so "" would replace its "en"
+        // default and tokenizer_text.cpp would throw
+        // "invalid Supertonic language: " on every request that set
+        // instructions and no language.
+        if (request.has_language() && !request.language().empty()) {
+            style.language = request.language();
+        }
+        condition.style = std::move(style);
+        condition_used = true;
+    }
+
+    if (condition_used) {
+        task.voice = std::move(condition);
+    }
+
+    if (request.has_language() && !request.language().empty()) {
+        // Forward-tolerant alias, exactly as in build_transcription_request. The
+        // families that read a "language" request option are the ASR ones
+        // (nemotron_asr, hviske_asr, vibevoice_asr, higgs_audio_stt), none of
+        // which this RPC can route to; pocket_tts reads one but from its
+        // ModelLoadRequest at load time, not from here. The Transcript above is
+        // what actually carries the language to a TTS family.
+        task.options["language"] = request.language();
+    }
+
+    // LAST, so an explicit params entry wins over anything derived above. That
+    // matters for "caption": a caller who sets params[caption] has named the
+    // exact string they want, and it must not be overwritten by `instructions`.
+    for (const auto &param : request.params()) {
+        task.options[param.first] = param.second;
+    }
+    return task;
+}
+
+engine::runtime::TaskRequest
+build_sound_generation_request(const backend::SoundGenerationRequest &request,
+                               std::optional<engine::runtime::AudioBuffer> source_audio) {
+    engine::runtime::TaskRequest task;
+
+    engine::runtime::Transcript transcript;
+    transcript.text = request.text();
+    if (request.has_language()) {
+        transcript.language = request.language();
+    }
+    task.text_input = std::move(transcript);
+
+    // src is the input clip for the editing routes. ace_step's repaint, cover
+    // and edit routes need it; stable_audio uses it as init_audio or
+    // inpaint_audio; heartmula refuses it outright.
+    if (source_audio.has_value()) {
+        task.audio_input = std::move(*source_audio);
+    }
+
+    // WHAT LANDS AND WHAT DOES NOT. Three families advertise AudioGeneration in
+    // the pinned upstream: ace_step, heartmula and stable_audio. Every key below
+    // was grepped against find_option/parse_*_option in src/ and include/ rather
+    // than assumed, because a key nobody reads is not a feature and shipping one
+    // while implying it works is the mistake this comment exists to prevent.
+    //
+    // Unknown REQUEST options cannot turn a valid request into an error:
+    // families look theirs up by name and ignore the rest, and the unknown-key
+    // refusals upstream does have are on SESSION options, which arrive at load
+    // time. So a forward-tolerant alias is free; it is just not a feature.
+
+    if (request.has_duration()) {
+        // duration_seconds is the key that works, and it works everywhere:
+        // ace_step (request_parser.cpp), heartmula (session.cpp, which also
+        // refuses a non-positive value) and stable_audio (request.cpp) all read
+        // it. This is the SoundGeneration analogue of Task 9's return_timestamps.
+        task.options["duration_seconds"] = std::to_string(request.duration());
+        // The proto field's own name. Read by exactly one family, omnivoice, and
+        // omnivoice advertises Tts rather than AudioGeneration, so this RPC can
+        // never route to it: DEAD here, kept only as a forward-tolerant alias.
+        task.options["duration"] = std::to_string(request.duration());
+    }
+    if (request.has_temperature()) {
+        // Read by heartmula. ace_step's sampling temperature is a different,
+        // narrower knob it calls lm_temperature (it drives the caption/thinking
+        // LM, not the audio diffusion), so this is deliberately NOT mapped onto
+        // it; a caller who wants it sets it through the request options that
+        // reach ace_step by name. stable_audio has no temperature at all.
+        task.options["temperature"] = std::to_string(request.temperature());
+    }
+    if (request.has_sample()) {
+        // do_sample is read widely upstream, but only by TTS and ASR families
+        // (chatterbox, index_tts2, miotts, moss, qwen3_tts, vibevoice,
+        // hviske_asr, voxtral_realtime). NO AudioGeneration family reads it, so
+        // it is dead on this route.
+        task.options["do_sample"] = bool_option(request.sample());
+    }
+    if (request.has_src_divisor()) {
+        // Read by nobody, anywhere in the pinned upstream. Forwarded because the
+        // proto documents it as part of this request and a family adopting it
+        // then works unchanged.
+        task.options["src_divisor"] = std::to_string(request.src_divisor());
+    }
+    if (request.has_think()) {
+        // "thinking" is the key ace_step actually reads (request_parser.cpp),
+        // which is why it is sent alongside the proto's own "think". "think" on
+        // its own is read by nobody.
+        task.options["thinking"] = bool_option(request.think());
+        task.options["think"] = bool_option(request.think());
+    }
+    if (request.has_caption()) {
+        // Read only by irodori_tts, which advertises Tts/VoiceCloning/VoiceDesign
+        // and not AudioGeneration, so it is unreachable from this RPC: DEAD here.
+        task.options["caption"] = request.caption();
+    }
+    if (request.has_lyrics()) {
+        // Read by ace_step and heartmula.
+        task.options["lyrics"] = request.lyrics();
+    }
+    if (request.has_bpm()) {
+        // Read by ace_step.
+        task.options["bpm"] = std::to_string(request.bpm());
+    }
+    if (request.has_keyscale()) {
+        // Read by ace_step.
+        task.options["keyscale"] = request.keyscale();
+    }
+    if (request.has_timesignature()) {
+        // Read by ace_step.
+        task.options["timesignature"] = request.timesignature();
+    }
+    if (request.has_instrumental()) {
+        // Read by nobody: "instrumental" appears in the pinned upstream only as
+        // a roformer STEM NAME, never as a request option. A forward-tolerant
+        // alias and nothing more.
+        task.options["instrumental"] = bool_option(request.instrumental());
+    }
+    if (request.has_language() && !request.language().empty()) {
+        // Alias again: the Transcript above is what ace_step reads as
+        // vocal_language. No AudioGeneration family reads a "language" option.
+        task.options["language"] = request.language();
+    }
+    return task;
+}
+
+bool apply_transform_text_input(engine::runtime::TaskRequest &task) {
+    // Canonical first, alias second, and an empty value falls through to the
+    // next candidate rather than ending the search: a caller who sent
+    // target_text="" and text="the real one" meant the second one.
+    static const char *const kTextKeys[] = {"target_text", "text"};
+
+    std::string text;
+    for (const char *key : kTextKeys) {
+        const auto found = task.options.find(key);
+        if (found != task.options.end() && !found->second.empty()) {
+            text = found->second;
+            break;
+        }
+    }
+    if (text.empty()) {
+        return false;
+    }
+
+    engine::runtime::Transcript transcript;
+    transcript.text = std::move(text);
+    // Inside the has-text branch on purpose. See the header: a language on its
+    // own conditions nothing and must not manufacture a text_input.
+    const auto language = task.options.find("language");
+    if (language != task.options.end()) {
+        transcript.language = language->second;
+    }
+    task.text_input = std::move(transcript);
+    return true;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/generation_request.h

@@ -0,0 +1,108 @@
+#pragma once
+
+// Builds the engine::runtime::TaskRequest for the two audio-PRODUCING offline
+// RPCs, TTS and SoundGeneration, and answers the one filesystem question TTS
+// routing depends on.
+//
+// It is a unit of its own rather than a pair of statics in grpc-server.cpp so
+// that it can be tested: grpc-server.cpp has a main() and cannot be linked into
+// a test binary, and everything here is a pure function of its arguments once
+// the file read has been lifted out (which is why the reference clip arrives as
+// an already-read buffer rather than a path). TTSStream reuses build_tts_request
+// unchanged.
+//
+// Only the plain structs in engine/framework/runtime/session.h are touched, so
+// this compiles against the header without linking engine_runtime, the same way
+// result_map does.
+
+#include "backend.pb.h"
+#include "capability_routing.h"
+
+#include "engine/framework/runtime/session.h"
+
+#include <optional>
+#include <string>
+
+namespace audiocpp_backend {
+
+// True when TTSRequest.voice names an existing regular file, in which case it
+// is a speaker reference clip and routing prefers VoiceCloning; false when it is
+// a named preset (or empty).
+//
+// The overload is LocalAI's, not this backend's: `voice` is the OpenAI speech
+// field and different LocalAI backends have always read it both ways. Deciding
+// it from the filesystem needs no new option and matches how somebody actually
+// configures a cloning family, which is by pointing at a clip.
+//
+// A DIRECTORY is deliberately not a reference: is_regular_file, not exists. A
+// directory named as a voice cannot be read as a WAV, and treating it as a
+// reference would turn a preset typo into "cannot read /x as WAV" instead of
+// letting it travel as the preset name it looks like.
+//
+// The error_code overload is used so an unreadable parent directory answers
+// false rather than throwing. That is the right answer here: the name is then
+// passed on as a preset, and if it really was meant to be a clip the family
+// refuses a request it cannot serve, which is a better message than a
+// filesystem exception thrown while classifying a string.
+bool voice_is_reference_file(const std::string &voice);
+
+// Everything routing needs to know about a TTSRequest, in one place, so that TTS
+// and TTSStream cannot describe the same request differently.
+//
+// `pinned_task` is deliberately NOT filled here: it comes off the LoadedModel,
+// not off the request, and this unit links no engine. The caller must still
+// write `shape.pinned_task = model->pinned_task();` or the model's `task:`
+// option is dead. That is the one field a new handler can forget, so it is the
+// one field left visible at the call site rather than hidden behind this
+// helper.
+RequestShape build_tts_shape(const backend::TTSRequest &request);
+
+// `reference_audio` is the already-read speaker clip, present exactly when
+// voice_is_reference_file(request.voice()) was true. Passing it in rather than a
+// path keeps this function pure and lets the caller do the read where the
+// ordering rules (capability refusal first, then the lane) are enforced.
+//
+// It is taken BY VALUE and moved in: a reference clip is seconds of audio and
+// the caller has no use for it afterwards.
+engine::runtime::TaskRequest
+build_tts_request(const backend::TTSRequest &request,
+                  std::optional<engine::runtime::AudioBuffer> reference_audio);
+
+// `source_audio` is SoundGenerationRequest.src already read, present exactly
+// when the field was set and non-empty. Same reasoning as above.
+engine::runtime::TaskRequest
+build_sound_generation_request(const backend::SoundGenerationRequest &request,
+                               std::optional<engine::runtime::AudioBuffer> source_audio);
+
+// Lifts a text-conditioned transform route's text out of the request params
+// into TaskRequest.text_input, and reports whether it set one.
+//
+// WHY THIS EXISTS. AudioTransform is an audio-in / audio-out RPC and its proto
+// message has no text field, but not every task it routes to is audio-only.
+// vevo2's speech-to-speech and prosody routes read their text from
+// request.text_input (src/models/vevo2/session.cpp fills refs.target_text from
+// exactly there and nowhere else) and refuse the run without one: "Vevo2
+// text/prosody route requires text_input or target_text". The params map is the
+// only channel AudioTransform has that reaches the engine, so the text travels
+// through it and is unpacked here. Without this, s2s is not merely awkward to
+// reach through this RPC, it is unreachable.
+//
+// CALL IT AFTER the params have been copied into task.options, and note that it
+// does NOT erase the keys it reads. vevo2's loader advertises "target_text" in
+// its own documented request-option table, so a family that looks there keeps
+// finding it; the copy in text_input is what the session actually reads today.
+//
+// "target_text" is canonical and "text" is its alias, the same order vevo2's
+// option table declares them in. A request setting both gets target_text, so
+// the canonical spelling wins rather than whichever the map happened to store
+// first. An empty value is not a text: it means the caller sent the key with
+// nothing in it, and a family asked to vocalise "" should say so itself rather
+// than be handed an empty Transcript that looks deliberate.
+//
+// "language" rides along when a text was found, and only then. On its own it
+// conditions nothing, and setting text_input for it alone would turn a plain
+// separation request that happened to carry a language hint into a text-routed
+// one.
+bool apply_transform_text_input(engine::runtime::TaskRequest &task);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/generation_request_ctest.cpp

@@ -0,0 +1,575 @@
+// Tests for the TTS and SoundGeneration request builders, and for the
+// filesystem rule that decides whether TTSRequest.voice is a speaker reference
+// clip or a named preset.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE: see the note at the top of
+// result_map_ctest.cpp. This file needs the generated protobuf messages and the
+// audio.cpp include path, neither of which backend/cpp/run-unit-tests.sh
+// provides, so it is built and run by ctest:
+//
+//   make -C backend/cpp/audio-cpp test-engine
+//
+// The assertions on OPTION KEYS are the point of this file, not decoration.
+// Every one of them names a key that was grepped against the pinned upstream:
+// "instruct" is read, "instructions" is not; "duration_seconds" is read,
+// "duration" is not. A rename that looks harmless is exactly the change that
+// silently stops a family honouring the request, so the spellings are pinned
+// here rather than left to a comment.
+
+#include "generation_request.h"
+
+#include <cstdio>
+#include <filesystem>
+#include <fstream>
+#include <string>
+#include <unordered_map>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+static bool has_key(const std::unordered_map<std::string, std::string> &options,
+                    const std::string &key) {
+    return options.find(key) != options.end();
+}
+
+static std::string option_or(
+    const std::unordered_map<std::string, std::string> &options,
+    const std::string &key, const std::string &fallback) {
+    const auto it = options.find(key);
+    return it == options.end() ? fallback : it->second;
+}
+
+static engine::runtime::AudioBuffer clip(int sample_rate, int channels) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = channels;
+    // Distinguishable content, so a builder that swapped one buffer for another
+    // (or default-constructed one) is visible rather than merely a size change.
+    buffer.samples = {0.25f, -0.5f, 0.75f, -1.0f};
+    return buffer;
+}
+
+static void test_voice_is_reference_file() {
+    const auto dir = std::filesystem::temp_directory_path() /
+                     "audiocpp-generation-request-ctest";
+    std::filesystem::remove_all(dir);
+    std::filesystem::create_directories(dir);
+
+    const auto file = dir / "reference.wav";
+    {
+        std::ofstream out(file, std::ios::binary);
+        out << "not really a wav, but a regular file";
+    }
+    const auto subdir = dir / "a-directory";
+    std::filesystem::create_directories(subdir);
+
+    check(!voice_is_reference_file(""), "voice_is_reference_file: empty");
+    check(!voice_is_reference_file("alloy"),
+          "voice_is_reference_file: bare preset name");
+    check(!voice_is_reference_file((dir / "absent.wav").string()),
+          "voice_is_reference_file: missing path");
+    check(voice_is_reference_file(file.string()),
+          "voice_is_reference_file: existing regular file");
+    // A directory is NOT a reference. exists() would say yes here and the read
+    // would then fail with "cannot read <dir> as WAV", which sends the operator
+    // after a file problem instead of a preset typo.
+    check(!voice_is_reference_file(subdir.string()),
+          "voice_is_reference_file: directory is not a reference");
+
+    std::filesystem::remove_all(dir);
+}
+
+// build_tts_shape is what TTS and TTSStream both hand to routing, so a wrong
+// answer here silently changes which task a request runs as, with a 200 and no
+// diagnostic. Every field is asserted in both directions.
+static void test_tts_shape() {
+    const auto dir = std::filesystem::temp_directory_path() /
+                     "audiocpp-generation-request-ctest-shape";
+    std::filesystem::remove_all(dir);
+    std::filesystem::create_directories(dir);
+    const auto file = dir / "reference.wav";
+    {
+        std::ofstream out(file, std::ios::binary);
+        out << "a regular file";
+    }
+
+    {
+        backend::TTSRequest request;
+        request.set_text("hello");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference && !shape.has_instructions,
+              "shape: bare request has neither signal");
+        // Never filled here: it comes off the LoadedModel, and leaving it empty
+        // is what makes the caller's assignment visible at the call site.
+        check(shape.pinned_task.empty(), "shape: pinned_task is left to the caller");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(file.string());
+        const auto shape = build_tts_shape(request);
+        check(shape.has_voice_reference,
+              "shape: an existing file is a voice reference");
+        check(!shape.has_instructions, "shape: a clip is not an instruction");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice("alloy");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference,
+              "shape: a preset name is not a voice reference");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(dir.string());
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference,
+              "shape: a directory is not a voice reference");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_instructions("a calm older man");
+        const auto shape = build_tts_shape(request);
+        check(shape.has_instructions, "shape: instructions are seen");
+        check(!shape.has_voice_reference,
+              "shape: instructions do not imply a reference");
+    }
+    {
+        // The guard that has to match build_tts_request's. An empty
+        // instructions string builds no style condition, so telling routing to
+        // prefer VoiceDesign for it would route to a task with nothing to
+        // design from.
+        backend::TTSRequest request;
+        request.set_instructions("");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_instructions,
+              "shape: an empty instructions string is not an instruction");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(file.string());
+        request.set_instructions("a calm older man");
+        const auto shape = build_tts_shape(request);
+        check(shape.has_voice_reference && shape.has_instructions,
+              "shape: both signals are reported when both are set");
+    }
+
+    std::filesystem::remove_all(dir);
+}
+
+static void test_tts_plain() {
+    backend::TTSRequest request;
+    request.set_text("hello there");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.text_input.has_value() && task.text_input->text == "hello there",
+          "tts: text reaches the transcript");
+    // No voice and no instructions means NO voice condition at all. A builder
+    // that always emitted one would make every family think a speaker was
+    // named, and chatterbox in particular refuses a prepare whose voice
+    // condition carries neither audio nor anything else it can use.
+    check(!task.voice.has_value(), "tts: no voice condition when nothing is set");
+    check(task.options.empty(), "tts: no options when nothing is set");
+    check(!task.audio_input.has_value(), "tts: no audio input");
+}
+
+static void test_tts_named_preset() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("alloy");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.voice.has_value() && task.voice->speaker.has_value(),
+          "tts preset: speaker condition present");
+    check(task.voice->speaker->cached_voice_id.has_value() &&
+              *task.voice->speaker->cached_voice_id == "alloy",
+          "tts preset: lands in cached_voice_id");
+    // The clip slot must stay empty, or a cloning family would try to prepare
+    // conditionals from a default-constructed buffer.
+    check(!task.voice->speaker->audio.has_value(),
+          "tts preset: no reference audio");
+    check(!task.voice->style.has_value(), "tts preset: no style condition");
+    check(option_or(task.options, "voice", "") == "alloy",
+          "tts preset: forwarded as the voice option too");
+}
+
+static void test_tts_reference_clip() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("/tmp/reference.wav");
+
+    const auto task = build_tts_request(request, clip(44100, 2));
+
+    check(task.voice.has_value() && task.voice->speaker.has_value(),
+          "tts clip: speaker condition present");
+    check(task.voice->speaker->audio.has_value(),
+          "tts clip: reference audio present");
+    // Rate and channels survive untouched. This is the assertion that fails if
+    // anybody decides to fold the clip to 16 kHz mono on the way in.
+    check(task.voice->speaker->audio->sample_rate == 44100 &&
+              task.voice->speaker->audio->channels == 2 &&
+              task.voice->speaker->audio->samples.size() == 4,
+          "tts clip: rate, channels and samples pass through unchanged");
+    // A clip is NOT also a cached voice id, and the path must not travel as a
+    // preset name: a family reading cached_voice_id would then look up a voice
+    // called "/tmp/reference.wav".
+    check(!task.voice->speaker->cached_voice_id.has_value(),
+          "tts clip: no cached_voice_id");
+    check(!has_key(task.options, "voice"), "tts clip: no voice option");
+}
+
+static void test_tts_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("a calm older man, speaking slowly");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(option_or(task.options, "instruct", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: instruct option is the one qwen3_tts reads");
+    check(option_or(task.options, "caption", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: caption option is the one irodori_tts reads");
+    check(option_or(task.options, "instructions", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: proto field name forwarded as an alias");
+    check(task.voice.has_value() && task.voice->style.has_value(),
+          "tts instructions: style condition present");
+    // "instruct", not "instructions". omnivoice and qwen3_tts both look this tag
+    // up by that exact name and by no other.
+    check(option_or(task.voice->style->tags, "instruct", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: style tag is spelled instruct");
+    check(!has_key(task.voice->style->tags, "instructions"),
+          "tts instructions: style tag is NOT spelled instructions");
+    // Instructions alone must not invent a speaker: has_voice_reference is what
+    // routing keys VoiceCloning off, and a speaker here would make a
+    // clone-capable family expect a clip it never received.
+    check(!task.voice->speaker.has_value(),
+          "tts instructions: no speaker without a voice");
+}
+
+static void test_tts_empty_instructions_are_not_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    // has_instructions() is true here, because the field was set. An empty
+    // string is still no instruction, and forwarding it would set an empty
+    // instruct option that qwen3_tts would prefer over its style tag fallback.
+    check(!has_key(task.options, "instruct"),
+          "tts: an empty instructions string sets no instruct option");
+    check(!task.voice.has_value(),
+          "tts: an empty instructions string sets no voice condition");
+}
+
+// THE EXACT SHAPE LocalAI PUTS ON THE WIRE. core/backend/tts.go's newTTSRequest
+// sets Language: &language UNCONDITIONALLY, so has_language() is true on every
+// request that ever reaches this backend, carrying an empty string whenever the
+// caller named no language.
+//
+// An empty StyleCondition::language is not a harmless default. supertonic reads
+// text_input->language behind a !empty() guard and then OVERRIDES it from
+// style->language whenever that optional is engaged, with no guard at all
+// (supertonic/session.cpp generation_options_from_request), so an empty style
+// language replaces its "en" default (session.h) with "" and
+// tokenizer_text.cpp's preprocess throws "invalid Supertonic language: ".
+// Every /v1/audio/speech request carrying instructions and no language would be
+// an INTERNAL. A plain request never sees it, because the style condition only
+// exists when instructions are non-empty.
+static void test_tts_empty_language_is_not_a_language() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("a calm older man");
+    request.set_language("");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.voice.has_value() && task.voice->style.has_value(),
+          "tts empty language: the style condition still exists");
+    check(!task.voice->style->language.has_value(),
+          "tts empty language: style language is left unset, not set to empty");
+    // The option emission has always guarded on !empty(); this pins the two to
+    // the same rule so they cannot drift apart again.
+    check(!has_key(task.options, "language"),
+          "tts empty language: no language option");
+    check(task.text_input.has_value() && task.text_input->language.empty(),
+          "tts empty language: transcript language stays empty");
+}
+
+// The one shape routing treats specially: a clip outranks instructions, and the
+// VoiceCondition then has to carry BOTH, because the family that wins is chosen
+// on the clip but may still read the style tag.
+static void test_tts_clip_and_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("/tmp/reference.wav");
+    request.set_instructions("bright and fast");
+    request.set_language("en");
+
+    const auto task = build_tts_request(request, clip(22050, 1));
+
+    check(task.voice.has_value(), "tts clip+instructions: voice condition present");
+    check(task.voice->speaker.has_value() &&
+              task.voice->speaker->audio.has_value() &&
+              task.voice->speaker->audio->sample_rate == 22050,
+          "tts clip+instructions: speaker carries the clip");
+    check(!task.voice->speaker->cached_voice_id.has_value(),
+          "tts clip+instructions: the clip path is not also a preset id");
+    check(task.voice->style.has_value() &&
+              option_or(task.voice->style->tags, "instruct", "") == "bright and fast",
+          "tts clip+instructions: style carries the instruct tag");
+    check(task.voice->style->language.has_value() &&
+              *task.voice->style->language == "en",
+          "tts clip+instructions: a real language does reach the style condition");
+    check(option_or(task.options, "instruct", "") == "bright and fast",
+          "tts clip+instructions: instruct option still emitted");
+    check(!has_key(task.options, "voice"),
+          "tts clip+instructions: still no voice option for a clip");
+}
+
+static void test_tts_language_and_params() {
+    backend::TTSRequest request;
+    request.set_text("ciao");
+    request.set_language("it");
+    request.set_instructions("warm");
+    (*request.mutable_params())["exaggeration"] = "0.7";
+    // An explicit param must win over the value derived from instructions.
+    (*request.mutable_params())["caption"] = "explicitly chosen caption";
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.text_input.has_value() && task.text_input->language == "it",
+          "tts language: reaches the transcript");
+    check(option_or(task.options, "language", "") == "it",
+          "tts language: forwarded as an option alias");
+    check(task.voice.has_value() && task.voice->style.has_value() &&
+              task.voice->style->language.has_value() &&
+              *task.voice->style->language == "it",
+          "tts language: reaches the style condition");
+    check(option_or(task.options, "exaggeration", "") == "0.7",
+          "tts params: passed through verbatim");
+    check(option_or(task.options, "caption", "") == "explicitly chosen caption",
+          "tts params: an explicit param overrides the derived caption");
+}
+
+static void test_sound_generation_minimal() {
+    backend::SoundGenerationRequest request;
+    request.set_text("a distant thunderstorm");
+
+    const auto task = build_sound_generation_request(request, std::nullopt);
+
+    check(task.text_input.has_value() &&
+              task.text_input->text == "a distant thunderstorm",
+          "sound: text reaches the transcript");
+    // Unset optionals must emit NOTHING. Emitting a zero for an unset duration
+    // would make heartmula refuse the request ("duration_seconds must be
+    // positive") on a request that never mentioned a duration.
+    check(task.options.empty(), "sound: unset optionals emit no options");
+    check(!task.audio_input.has_value(), "sound: no audio input without src");
+    check(!task.voice.has_value(), "sound: no voice condition");
+}
+
+static void test_sound_generation_full() {
+    backend::SoundGenerationRequest request;
+    request.set_text("a slow blues in E");
+    request.set_duration(30.0f);
+    request.set_temperature(0.8f);
+    request.set_sample(false);
+    request.set_src_divisor(4);
+    request.set_think(true);
+    request.set_caption("smoky bar recording");
+    request.set_lyrics("first line\nsecond line");
+    request.set_bpm(72);
+    request.set_keyscale("E minor");
+    request.set_language("en");
+    request.set_timesignature("4/4");
+    request.set_instrumental(true);
+
+    const auto task = build_sound_generation_request(request, clip(48000, 2));
+
+    // duration_seconds is the key every AudioGeneration family actually reads;
+    // "duration" rides along as an alias. Both spellings are pinned so a
+    // "cleanup" that keeps only the proto's own name is a test failure and not
+    // a silent loss of the duration.
+    check(option_or(task.options, "duration_seconds", "").rfind("30.", 0) == 0,
+          "sound: duration lands as duration_seconds");
+    check(option_or(task.options, "duration", "").rfind("30.", 0) == 0,
+          "sound: duration also forwarded under its own name");
+    check(option_or(task.options, "temperature", "").rfind("0.8", 0) == 0,
+          "sound: temperature forwarded");
+    // Set to FALSE, so this also proves the key is written whenever the field is
+    // present rather than only when the value is truthy.
+    check(option_or(task.options, "do_sample", "") == "false",
+          "sound: sample=false is forwarded as do_sample=false");
+    check(option_or(task.options, "src_divisor", "") == "4",
+          "sound: src_divisor forwarded");
+    check(option_or(task.options, "thinking", "") == "true",
+          "sound: think lands as thinking, the key ace_step reads");
+    check(option_or(task.options, "think", "") == "true",
+          "sound: think also forwarded under its own name");
+    check(option_or(task.options, "caption", "") == "smoky bar recording",
+          "sound: caption forwarded");
+    check(option_or(task.options, "lyrics", "") == "first line\nsecond line",
+          "sound: lyrics forwarded");
+    check(option_or(task.options, "bpm", "") == "72", "sound: bpm forwarded");
+    check(option_or(task.options, "keyscale", "") == "E minor",
+          "sound: keyscale forwarded");
+    check(option_or(task.options, "timesignature", "") == "4/4",
+          "sound: timesignature forwarded");
+    check(option_or(task.options, "instrumental", "") == "true",
+          "sound: instrumental forwarded");
+    check(option_or(task.options, "language", "") == "en",
+          "sound: language forwarded as an option alias");
+    check(task.text_input->language == "en",
+          "sound: language reaches the transcript, which is what ace_step reads");
+    check(task.audio_input.has_value() &&
+              task.audio_input->sample_rate == 48000 &&
+              task.audio_input->channels == 2 &&
+              task.audio_input->samples.size() == 4,
+          "sound: src passes through at its own rate and channel count");
+}
+
+
+// ---------------------------------------------------------------------------
+// apply_transform_text_input
+//
+// AudioTransform has no text field on the wire, so a text-conditioned route
+// (vevo2's speech-to-speech) can only be reached if the text travels as a
+// param and is unpacked into text_input. Every assertion below pins a spelling
+// or a precedence that a family actually depends on, not a shape that merely
+// looks tidy.
+
+static void test_transform_text_absent() {
+    engine::runtime::TaskRequest task;
+    task.options["stem"] = "vocals";
+    check(!apply_transform_text_input(task),
+          "transform text: reports false when no text key is present");
+    check(!task.text_input.has_value(),
+          "transform text: a request with no text keeps text_input unset");
+}
+
+static void test_transform_text_canonical_key() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "sing this line";
+    check(apply_transform_text_input(task), "transform text: target_text reports true");
+    check(task.text_input.has_value() && task.text_input->text == "sing this line",
+          "transform text: target_text becomes text_input.text");
+    check(has_key(task.options, "target_text"),
+          "transform text: target_text survives in options for families that read it there");
+}
+
+static void test_transform_text_alias_key() {
+    engine::runtime::TaskRequest task;
+    task.options["text"] = "say this instead";
+    check(apply_transform_text_input(task), "transform text: text alias reports true");
+    check(task.text_input.has_value() && task.text_input->text == "say this instead",
+          "transform text: the text alias becomes text_input.text");
+}
+
+static void test_transform_text_canonical_wins() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "canonical";
+    task.options["text"] = "alias";
+    check(apply_transform_text_input(task), "transform text: both keys reports true");
+    check(task.text_input.has_value() && task.text_input->text == "canonical",
+          "transform text: target_text wins over text, not whichever hashed first");
+}
+
+static void test_transform_text_empty_is_not_a_text() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "";
+    check(!apply_transform_text_input(task),
+          "transform text: an empty target_text reports false");
+    check(!task.text_input.has_value(),
+          "transform text: an empty target_text leaves text_input unset");
+}
+
+static void test_transform_text_empty_canonical_falls_through_to_alias() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "";
+    task.options["text"] = "the real one";
+    check(apply_transform_text_input(task),
+          "transform text: an empty canonical key does not mask a usable alias");
+    check(task.text_input.has_value() && task.text_input->text == "the real one",
+          "transform text: the alias is used when the canonical key is empty");
+}
+
+static void test_transform_text_language_rides_along() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "vocalise me";
+    task.options["language"] = "ja";
+    check(apply_transform_text_input(task), "transform text: text plus language reports true");
+    check(task.text_input.has_value() && task.text_input->language == "ja",
+          "transform text: language lands on the Transcript alongside the text");
+    check(has_key(task.options, "language"),
+          "transform text: language survives in options too");
+}
+
+static void test_transform_language_alone_is_not_a_text() {
+    engine::runtime::TaskRequest task;
+    task.options["language"] = "ja";
+    check(!apply_transform_text_input(task),
+          "transform text: a language with no text reports false");
+    check(!task.text_input.has_value(),
+          "transform text: a language alone must not route a separation request through text");
+}
+
+static void test_transform_text_preserves_other_inputs() {
+    engine::runtime::TaskRequest task;
+    engine::runtime::AudioBuffer audio;
+    audio.sample_rate = 44100;
+    audio.channels = 2;
+    audio.samples = {0.1f, 0.2f, 0.3f, 0.4f};
+    task.audio_input = audio;
+    task.options["target_text"] = "keep the audio";
+    check(apply_transform_text_input(task), "transform text: with audio present reports true");
+    check(task.audio_input.has_value() && task.audio_input->samples.size() == 4 &&
+              task.audio_input->sample_rate == 44100,
+          "transform text: the source audio is untouched");
+}
+
+int main() {
+    test_voice_is_reference_file();
+    test_tts_shape();
+    test_tts_plain();
+    test_tts_named_preset();
+    test_tts_reference_clip();
+    test_tts_instructions();
+    test_tts_empty_instructions_are_not_instructions();
+    test_tts_empty_language_is_not_a_language();
+    test_tts_clip_and_instructions();
+    test_tts_language_and_params();
+    test_sound_generation_minimal();
+    test_sound_generation_full();
+    test_transform_text_absent();
+    test_transform_text_canonical_key();
+    test_transform_text_alias_key();
+    test_transform_text_canonical_wins();
+    test_transform_text_empty_is_not_a_text();
+    test_transform_text_empty_canonical_falls_through_to_alias();
+    test_transform_text_language_rides_along();
+    test_transform_language_alone_is_not_a_text();
+    test_transform_text_preserves_other_inputs();
+
+    if (failures != 0) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/inference_lane.cpp

@@ -0,0 +1,114 @@
+#include "inference_lane.h"
+
+#include <algorithm>
+#include <chrono>
+
+namespace audiocpp_backend {
+
+std::int64_t monotonic_millis() {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+               std::chrono::steady_clock::now().time_since_epoch())
+        .count();
+}
+
+int resolve_wait_budget_ms(int policy_ceiling_ms, int request_hint_ms) {
+    // Both sides normalise to 0 for "not specified", which is also the value
+    // that means unbounded on the way out, so the unspecified cases fall out of
+    // the arithmetic instead of needing their own branches.
+    const int ceiling = policy_ceiling_ms > 0 ? policy_ceiling_ms : 0;
+    const int hint = request_hint_ms > 0 ? request_hint_ms : 0;
+
+    if (ceiling == 0) {
+        return hint;
+    }
+    if (hint == 0) {
+        return ceiling;
+    }
+    // The hint can tighten the ceiling but never loosen it.
+    return std::min(ceiling, hint);
+}
+
+bool run_exceeds_budget(bool lane_occupied, std::int64_t run_started_ms,
+                        std::int64_t now_ms, int budget_ms) {
+    if (!lane_occupied || budget_ms <= 0) {
+        return false;
+    }
+    return now_ms - run_started_ms > static_cast<std::int64_t>(budget_ms);
+}
+
+namespace {
+
+std::string busy_prefix(const std::string &model_label) {
+    return "inference lane for model '" + model_label + "' is busy: ";
+}
+
+// States the measurement and nothing else. A short-budget caller meeting a
+// legitimately long run lands here too, so this text must not declare the run
+// broken; the numbers let a reader decide that for themselves.
+std::string overrun_message(const std::string &model_label,
+                            std::int64_t run_age_ms, int budget_ms) {
+    return busy_prefix(model_label) + "the in-flight run has been running for " +
+           std::to_string(run_age_ms) + " ms, longer than this request's " +
+           std::to_string(budget_ms) + " ms wait budget";
+}
+
+std::string wait_exhausted_message(const std::string &model_label,
+                                   int budget_ms) {
+    return busy_prefix(model_label) + "timed out after " +
+           std::to_string(budget_ms) +
+           " ms waiting for the in-flight run to finish";
+}
+
+} // namespace
+
+void InferenceLane::occupy(int budget_ms) {
+    std::unique_lock<std::mutex> lock(state_mutex_);
+
+    // Checked once, on arrival: if the run already in the lane has outlived what
+    // this caller brought, no amount of waiting can help it, and queueing here
+    // is exactly how a wedged run swallows every handler thread.
+    const std::int64_t arrived_ms = monotonic_millis();
+    if (run_exceeds_budget(occupied_, run_started_ms_, arrived_ms, budget_ms)) {
+        throw LaneUnavailable(overrun_message(
+            model_label_, arrived_ms - run_started_ms_, budget_ms));
+    }
+
+    if (budget_ms > 0) {
+        if (!vacated_.wait_for(lock, std::chrono::milliseconds(budget_ms),
+                               [this] { return !occupied_; })) {
+            throw LaneUnavailable(
+                wait_exhausted_message(model_label_, budget_ms));
+        }
+    } else {
+        vacated_.wait(lock, [this] { return !occupied_; });
+    }
+
+    // Only now, holding both the mutex and the lane. Stamping any earlier would
+    // restart the age of the run for every caller behind this one and make a
+    // genuinely wedged holder look freshly started forever.
+    occupied_ = true;
+    run_started_ms_ = monotonic_millis();
+}
+
+void InferenceLane::vacate() {
+    {
+        std::lock_guard<std::mutex> lock(state_mutex_);
+        occupied_ = false;
+    }
+    // notify_all, not notify_one: a waiter that times out concurrently with a
+    // notification can consume it, and losing the only wakeup would park the
+    // remaining waiters for the rest of the run's lifetime. Waking all of them
+    // still admits exactly one, since the rest re-test occupancy under the mutex
+    // and go back to waiting, and ordering among waiters is not a requirement.
+    vacated_.notify_all();
+}
+
+LaneEntry::LaneEntry(InferenceLane &lane, int budget_ms) : lane_(lane) {
+    // If this throws, the object never existed, so ~LaneEntry does not run and
+    // cannot hand back a lane this caller never held.
+    lane_.occupy(budget_ms);
+}
+
+LaneEntry::~LaneEntry() { lane_.vacate(); }
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/inference_lane.h

@@ -0,0 +1,138 @@
+#pragma once
+
+// Serializes inference against the single audio.cpp model a backend process
+// owns. Standard library only.
+//
+// Why a plain mutex is not enough: an audio.cpp session is not reentrant, so
+// concurrent gRPC handlers have to take turns. But once a GPU call stops making
+// progress there is nothing a host thread can do to take it back, and an
+// unbounded queue behind such a run would absorb the handler threads one by one
+// until nothing is left to answer with. A caller therefore needs to be able to
+// walk away, and needs to be able to tell "the lane is busy with normal work and
+// I ran out of patience" apart from "the run in the lane has already outlived
+// the patience I brought".
+
+#include <condition_variable>
+#include <cstdint>
+#include <mutex>
+#include <stdexcept>
+#include <string>
+
+namespace audiocpp_backend {
+
+// Reading off a monotonic clock, in milliseconds. Monotonic on purpose: a wall
+// clock adjustment must never make an in-flight run look younger or older than
+// it is, because that reading decides whether callers give up.
+std::int64_t monotonic_millis();
+
+// Collapses the per-model configured ceiling and the optional per-request hint
+// into the wait budget a caller actually gets. Returns 0 for "wait
+// indefinitely".
+//
+// Either input may be non-positive, which means "not specified":
+//   - an unspecified hint yields the ceiling,
+//   - an unspecified ceiling means no policy limit, so the hint stands,
+//   - unspecified on both sides is unbounded.
+// A specified hint may only tighten the ceiling. A client asking for a longer
+// wait than the model's policy allows does not get it, because that would let a
+// request weaken an operator's choice.
+int resolve_wait_budget_ms(int policy_ceiling_ms, int request_hint_ms);
+
+// True when a caller carrying budget_ms should give up on arrival rather than
+// queue up. Deliberately a pure function of the lane's observable state so the
+// decision can be tested without threads or sleeping.
+//
+// Only occupancy makes a start timestamp meaningful: a lane nobody holds is
+// never overrunning, whatever timestamp the last holder left behind. An
+// unbounded caller (non-positive budget) has no budget to exceed. And the
+// comparison is strict, so a run whose age exactly equals the budget still has
+// its last millisecond.
+bool run_exceeds_budget(bool lane_occupied, std::int64_t run_started_ms,
+                        std::int64_t now_ms, int budget_ms);
+
+// Thrown when a caller cannot take the lane, in either of the two situations
+// resolve_wait_budget_ms allows for. The message distinguishes them; callers
+// that need to report a status code can treat them alike.
+class LaneUnavailable : public std::runtime_error {
+  public:
+    explicit LaneUnavailable(const std::string &reason)
+        : std::runtime_error(reason) {}
+};
+
+class LaneEntry;
+
+// One lane per loaded model. Shared by every handler thread; not copyable.
+class InferenceLane {
+  public:
+    explicit InferenceLane(std::string model_label)
+        : model_label_(std::move(model_label)) {}
+
+    InferenceLane(const InferenceLane &) = delete;
+    InferenceLane &operator=(const InferenceLane &) = delete;
+
+    const std::string &model_label() const { return model_label_; }
+
+  private:
+    // Occupancy is only reachable through LaneEntry, so there is no way to take
+    // the lane without also having something that gives it back.
+    friend class LaneEntry;
+
+    void occupy(int budget_ms);
+    void vacate();
+
+    const std::string model_label_;
+
+    std::mutex state_mutex_;
+    std::condition_variable vacated_;
+    bool occupied_ = false;
+    // Only meaningful while occupied_ is true.
+    std::int64_t run_started_ms_ = 0;
+};
+
+// Scoped occupancy of a lane. Construct it where the inference happens and it
+// is given back on every exit from that scope, including an exception and
+// including a caller that returns from the middle of a long stream. Throws
+// LaneUnavailable if the lane could not be taken, in which case there is no
+// object and nothing to release.
+//
+// Not reentrant, and it does not detect reentrancy: a second entry constructed
+// while the calling thread already holds the same lane waits for a lane only
+// that thread can release. With a positive budget that surfaces as
+// LaneUnavailable, but in unbounded mode the thread parks with no diagnostic at
+// all. Keep entries one per call: a handler that holds one across a stream must
+// not let a helper it calls construct another.
+class LaneEntry {
+  public:
+    // budget_ms <= 0 waits indefinitely. Pass the output of
+    // resolve_wait_budget_ms.
+    LaneEntry(InferenceLane &lane, int budget_ms);
+    ~LaneEntry();
+
+    LaneEntry(const LaneEntry &) = delete;
+    LaneEntry &operator=(const LaneEntry &) = delete;
+
+    // Deliberately immovable rather than carefully movable. A moved-from entry
+    // would have to stop releasing the lane while the lane still records it as
+    // occupied, and that hazard is not worth the convenience: the lane can only
+    // be recovered by whoever took it.
+    //
+    // To hold a lane for longer than one scope, construct the entry in place
+    // instead of moving one in. Two shapes work:
+    //   std::optional<LaneEntry> held;   // member or local
+    //   held.emplace(lane, budget_ms);   // takes the lane, held.reset() gives it back
+    //   auto held = std::make_unique<LaneEntry>(lane, budget_ms);  // also returnable
+    // Both outlive the acquiring scope and still release exactly once, when they
+    // are reset or destroyed. Prefer the optional for a member whose lifetime is
+    // the handler's; use the unique_ptr when the entry has to be returned, since
+    // an optional of an immovable type is itself immovable and cannot be. A
+    // factory may instead write `return LaneEntry(lane, budget_ms);`, which C++17
+    // guarantees to elide, whereas `LaneEntry entry(...); return entry;` does not
+    // compile, because that form is a move.
+    LaneEntry(LaneEntry &&) = delete;
+    LaneEntry &operator=(LaneEntry &&) = delete;
+
+  private:
+    InferenceLane &lane_;
+};
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/inference_lane_test.cpp

@@ -0,0 +1,618 @@
+// Unit tests for inference_lane. Standard library only. The harness compiles
+// this file as a single translation unit, so the implementation is included
+// directly rather than linked.
+//
+// Two kinds of test live here:
+//
+//   * The pure ones (budget negotiation, the overrun predicate) run with no
+//     threads and no sleeping. They carry the arithmetic, so they are the tests
+//     that must be exhaustive.
+//   * The threaded ones exercise the lane itself. Every one of them is bounded:
+//     contenders use a generous wait budget instead of the unbounded mode
+//     wherever the point of the test does not require unbounded, and a watchdog
+//     in main() puts a ceiling on the whole file. A broken implementation must
+//     go red, not hang, because a hung job costs CI far more than a red one.
+//
+// Wall-clock margins are called out individually. The rule applied throughout:
+// a margin is only allowed if a slow or loaded machine pushes the measurement
+// deeper into the passing region.
+
+#include "inference_lane.cpp"
+
+#include <atomic>
+#include <chrono>
+#include <condition_variable>
+#include <cstdio>
+#include <cstdlib>
+#include <mutex>
+#include <string>
+#include <thread>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using audiocpp_backend::InferenceLane;
+using audiocpp_backend::LaneEntry;
+using audiocpp_backend::LaneUnavailable;
+using audiocpp_backend::resolve_wait_budget_ms;
+using audiocpp_backend::run_exceeds_budget;
+
+// A one-shot, level-triggered signal with a bounded wait. Preferred over sleeps
+// for "the other thread got there" so the tests do not encode a guess about
+// scheduling.
+class Signal {
+  public:
+    void raise() {
+        {
+            std::lock_guard<std::mutex> lock(mutex_);
+            raised_ = true;
+        }
+        cv_.notify_all();
+    }
+
+    bool await(int timeout_ms) {
+        std::unique_lock<std::mutex> lock(mutex_);
+        return cv_.wait_for(lock, std::chrono::milliseconds(timeout_ms),
+                            [this] { return raised_; });
+    }
+
+  private:
+    std::mutex mutex_;
+    std::condition_variable cv_;
+    bool raised_ = false;
+};
+
+static std::int64_t elapsed_ms_since(
+    const std::chrono::steady_clock::time_point &start) {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+               std::chrono::steady_clock::now() - start)
+        .count();
+}
+
+static void nap(int ms) {
+    std::this_thread::sleep_for(std::chrono::milliseconds(ms));
+}
+
+static bool mentions(const std::string &haystack, const std::string &needle) {
+    return haystack.find(needle) != std::string::npos;
+}
+
+// Wording that separates the two failure modes. Kept here so a message reword
+// that erases the distinction breaks these tests loudly.
+static const char *const kTimedOutPhrase = "timed out after";
+static const char *const kStillRunningPhrase = "has been running for";
+
+// Attempts an entry and reports what happened, so the threaded tests can assert
+// on the message rather than only on the exception type.
+struct EntryOutcome {
+    bool acquired = false;
+    std::string message;
+    std::int64_t took_ms = 0;
+};
+
+static EntryOutcome try_entry(InferenceLane &lane, int budget_ms) {
+    EntryOutcome out;
+    const auto start = std::chrono::steady_clock::now();
+    try {
+        LaneEntry entry(lane, budget_ms);
+        out.acquired = true;
+    } catch (const LaneUnavailable &refused) {
+        out.message = refused.what();
+    }
+    out.took_ms = elapsed_ms_since(start);
+    return out;
+}
+
+// ---------------------------------------------------------------------------
+// B9: budget negotiation. Pure, no threads.
+// ---------------------------------------------------------------------------
+static void test_budget_negotiation() {
+    check(resolve_wait_budget_ms(0, 0) == 0,
+          "B9 no ceiling and no request hint is unbounded");
+    check(resolve_wait_budget_ms(-1, -1) == 0,
+          "B9 negative ceiling and negative hint is unbounded");
+
+    check(resolve_wait_budget_ms(5000, 0) == 5000,
+          "B9 absent hint yields the ceiling");
+    check(resolve_wait_budget_ms(5000, -250) == 5000,
+          "B9 negative hint yields the ceiling");
+
+    check(resolve_wait_budget_ms(5000, 1200) == 1200,
+          "B9 a shorter request hint is granted");
+    check(resolve_wait_budget_ms(5000, 1) == 1,
+          "B9 a much shorter request hint is granted");
+
+    check(resolve_wait_budget_ms(5000, 9000) == 5000,
+          "B9 a longer request hint cannot weaken the ceiling");
+    check(resolve_wait_budget_ms(5000, 5001) == 5000,
+          "B9 a hint one ms over the ceiling is clamped");
+    check(resolve_wait_budget_ms(5000, 5000) == 5000,
+          "B9 a hint equal to the ceiling is the ceiling");
+
+    check(resolve_wait_budget_ms(0, 1200) == 1200,
+          "B9 without a policy limit the request hint applies");
+    check(resolve_wait_budget_ms(-5, 1200) == 1200,
+          "B9 a negative ceiling is no policy limit");
+
+    check(resolve_wait_budget_ms(1, 0) == 1,
+          "B9 a one ms ceiling survives negotiation");
+}
+
+// ---------------------------------------------------------------------------
+// B5 and B6: the overrun predicate. Pure, no threads.
+// ---------------------------------------------------------------------------
+static void test_overrun_predicate() {
+    // B5: strictly longer.
+    check(run_exceeds_budget(true, 1000, 1100, 100) == false,
+          "B5 elapsed exactly equal to the budget is not an overrun");
+    check(run_exceeds_budget(true, 1000, 1101, 100) == true,
+          "B5 one ms past the budget is an overrun");
+    check(run_exceeds_budget(true, 1000, 1099, 100) == false,
+          "B5 one ms short of the budget is not an overrun");
+    check(run_exceeds_budget(true, 0, 1, 1) == false,
+          "B5 a one ms budget at one ms elapsed is not an overrun");
+    check(run_exceeds_budget(true, 0, 2, 1) == true,
+          "B5 a one ms budget at two ms elapsed is an overrun");
+
+    // B6: an unoccupied lane is never stuck, whatever the leftover timestamp
+    // says. This is the pure half of B6; the wiring half is threaded below.
+    check(run_exceeds_budget(false, 0, 10000000, 1) == false,
+          "B6 an idle lane with an ancient start timestamp is not an overrun");
+    check(run_exceeds_budget(false, 5, 5, 5) == false,
+          "B6 an idle lane is not an overrun at any elapsed value");
+
+    // An unbounded caller has no budget to exceed, so it never fails fast.
+    check(run_exceeds_budget(true, 0, 10000000, 0) == false,
+          "B2 an unbounded caller never sees an overrun");
+    check(run_exceeds_budget(true, 0, 10000000, -1) == false,
+          "B2 a negative budget never sees an overrun");
+}
+
+// ---------------------------------------------------------------------------
+// B1: mutual exclusion under real contention, and a release admits a waiter.
+// ---------------------------------------------------------------------------
+static void test_mutual_exclusion() {
+    InferenceLane lane("exclusion-model");
+
+    constexpr int kContenders = 4; // "at least three simultaneous contenders"
+    constexpr int kHoldMs = 15;
+    // Generous on purpose: the point of this test is exclusion, not timeouts.
+    // A larger budget only makes a healthy run more likely to pass, while still
+    // bounding a broken one at roughly five seconds instead of forever.
+    constexpr int kBudgetMs = 5000;
+
+    std::atomic<int> in_flight{0};
+    std::atomic<int> peak_in_flight{0};
+    std::atomic<int> completed{0};
+    std::atomic<int> refused{0};
+
+    Signal go;
+    std::vector<std::thread> contenders;
+    for (int i = 0; i < kContenders; i++) {
+        contenders.emplace_back([&] {
+            go.await(5000);
+            try {
+                LaneEntry entry(lane, kBudgetMs);
+                const int now_inside = in_flight.fetch_add(1) + 1;
+                int seen = peak_in_flight.load();
+                while (now_inside > seen &&
+                       !peak_in_flight.compare_exchange_weak(seen, now_inside)) {
+                    // retry with the refreshed value
+                }
+                nap(kHoldMs);
+                in_flight.fetch_sub(1);
+                completed.fetch_add(1);
+            } catch (const LaneUnavailable &) {
+                refused.fetch_add(1);
+            }
+        });
+    }
+    go.raise();
+    for (auto &t : contenders) {
+        t.join();
+    }
+
+    check(refused.load() == 0, "B1 no contender was refused within its budget");
+    check(completed.load() == kContenders,
+          "B1 every contender eventually got the lane");
+    check(peak_in_flight.load() == 1,
+          "B1 never more than one holder inside the lane at once");
+}
+
+// ---------------------------------------------------------------------------
+// B2: unbounded mode waits out a run longer than any bound would allow.
+// ---------------------------------------------------------------------------
+static void test_unbounded_waits_out_the_holder() {
+    InferenceLane lane("patient-model");
+    constexpr int kHoldMs = 300;
+
+    Signal held;
+    Signal patient_done;
+    std::int64_t patient_wait_ms = -1;
+    bool patient_acquired = false;
+
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        nap(kHoldMs);
+    });
+    check(held.await(5000), "B2 holder took the lane");
+
+    std::thread patient([&] {
+        const auto start = std::chrono::steady_clock::now();
+        try {
+            LaneEntry entry(lane, 0);
+            patient_acquired = true;
+        } catch (const LaneUnavailable &) {
+            patient_acquired = false;
+        }
+        patient_wait_ms = elapsed_ms_since(start);
+        patient_done.raise();
+    });
+
+    // Bound on the unbounded mode: a lane that never wakes its waiters goes red
+    // here instead of hanging in join(). The named failure is printed before the
+    // join, so even a hard hang leaves a diagnosis behind for the watchdog.
+    check(patient_done.await(10000), "B2 unbounded caller returned at all");
+    patient.join();
+    holder.join();
+
+    check(patient_acquired, "B2 unbounded caller acquired instead of failing");
+    // Margin: the holder holds for 300 ms, so the waiter must block for about
+    // that long. Asserting only half of it means a loaded machine, which makes
+    // the wait longer, drifts further into passing.
+    check(patient_wait_ms >= kHoldMs / 2,
+          "B2 unbounded caller actually waited for the in-flight run");
+}
+
+// ---------------------------------------------------------------------------
+// B3: a bounded caller that cannot get in gives up with the timeout wording.
+// ---------------------------------------------------------------------------
+static void test_bounded_wait_times_out() {
+    InferenceLane lane("impatient-model");
+    constexpr int kBudgetMs = 120;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B3 holder took the lane");
+
+    // The waiter arrives immediately, so the holder's elapsed time is far below
+    // the budget and the fail-fast path must not trigger here.
+    //
+    // Load-sensitive margin, and the tightest one in this file: what makes this
+    // the timeout path rather than the fail-fast path is the holder's age
+    // staying under 120 ms at the arrival check. All that sits between the
+    // holder's stamp and this call is one signal handover, microseconds against
+    // a 120 ms allowance, but unlike the other margins here load pushes this one
+    // toward failing rather than away from it. If it ever does flip, the symptom
+    // is the wording assertions below going red, not a hang, and the fix is a
+    // larger budget rather than a weaker assertion.
+    const EntryOutcome outcome = try_entry(lane, kBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!outcome.acquired, "B3 bounded caller did not acquire a held lane");
+    check(mentions(outcome.message, kTimedOutPhrase),
+          "B3 failure names the exhausted wait, not an overrunning run");
+    check(!mentions(outcome.message, kStillRunningPhrase),
+          "B3 failure is not worded as an overrun");
+    check(mentions(outcome.message, "impatient-model"),
+          "B3 failure names the model");
+    check(mentions(outcome.message, "120"),
+          "B3 failure reports the budget it waited out");
+    // Margin: wait_for cannot return before its deadline, so the true value is
+    // at least 120 ms and load only raises it. Asserting 100 leaves room for
+    // clock granularity while still catching an implementation that returns
+    // early without waiting.
+    check(outcome.took_ms >= 100, "B3 bounded caller waited out its budget");
+}
+
+// ---------------------------------------------------------------------------
+// B4: a caller whose budget is already exceeded fails at once.
+// ---------------------------------------------------------------------------
+static void test_fail_fast_against_a_long_run() {
+    InferenceLane lane("wedged-model");
+    constexpr int kBudgetMs = 200;
+    constexpr int kRunAgeMs = 400;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B4 holder took the lane");
+
+    // Margin: the arriving caller needs the holder's elapsed time to exceed
+    // 200 ms. Sleeping 400 ms means a loaded machine oversleeps and pushes the
+    // elapsed time further past the budget, never below it.
+    nap(kRunAgeMs);
+    const EntryOutcome outcome = try_entry(lane, kBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!outcome.acquired, "B4 caller did not acquire a long-running lane");
+    check(mentions(outcome.message, kStillRunningPhrase),
+          "B4 failure states the measured age of the in-flight run");
+    check(!mentions(outcome.message, kTimedOutPhrase),
+          "B4 failure is not worded as an exhausted wait");
+    check(mentions(outcome.message, "wedged-model"),
+          "B4 failure names the model");
+    // Margin: a fail-fast return takes microseconds. 150 ms of headroom under a
+    // 200 ms budget separates "returned at once" from "waited out the budget"
+    // by a wide enough gap that scheduler noise cannot close it. The message
+    // assertions above are the load-independent proof; this one pins the timing.
+    check(outcome.took_ms < 150, "B4 caller failed without waiting out its budget");
+}
+
+// ---------------------------------------------------------------------------
+// B6 wiring: an idle lane never looks stuck, however old the last run is.
+// ---------------------------------------------------------------------------
+static void test_idle_lane_is_never_stuck() {
+    InferenceLane lane("idle-model");
+
+    {
+        LaneEntry entry(lane, 0);
+    }
+    // Ages the leftover start timestamp well past the tiny budget used below.
+    // A longer sleep only makes a stale-timestamp bug more visible, so load
+    // helps this test rather than hurting it.
+    nap(80);
+
+    const EntryOutcome first = try_entry(lane, 20);
+    check(first.acquired, "B6 tiny budget still acquires an idle lane");
+
+    nap(80);
+    const EntryOutcome second = try_entry(lane, 1);
+    check(second.acquired, "B6 a one ms budget still acquires an idle lane");
+}
+
+// ---------------------------------------------------------------------------
+// B7: every ownership exit clears the busy state, including an exception
+// thrown from inside the guarded region.
+// ---------------------------------------------------------------------------
+static void test_release_on_exception() {
+    InferenceLane lane("throwing-model");
+
+    struct GuardedRegionFailure {};
+    bool propagated = false;
+    try {
+        LaneEntry entry(lane, 0);
+        throw GuardedRegionFailure{};
+    } catch (const GuardedRegionFailure &) {
+        propagated = true;
+    }
+    check(propagated, "B7 the guarded region's own exception propagated");
+
+    // If the throw had leaked the busy state, this tiny budget would fail.
+    const EntryOutcome after_throw = try_entry(lane, 20);
+    check(after_throw.acquired, "B7 lane is free after an exception unwound it");
+
+    // Same check for a holder that unwinds on another thread, which is the shape
+    // a gRPC handler failing mid-inference actually has.
+    Signal thrown;
+    std::thread unlucky([&] {
+        try {
+            LaneEntry entry(lane, 0);
+            throw GuardedRegionFailure{};
+        } catch (const GuardedRegionFailure &) {
+            thrown.raise();
+        }
+    });
+    check(thrown.await(5000), "B7 worker thread unwound its guarded region");
+    unlucky.join();
+
+    const EntryOutcome after_worker = try_entry(lane, 20);
+    check(after_worker.acquired,
+          "B7 lane is free after a worker thread unwound it");
+}
+
+// ---------------------------------------------------------------------------
+// B8: a waiter must not publish itself as the holder. If it did, its arrival
+// would restart the elapsed-time measurement and hide the real holder.
+//
+// Timeline, with the holder taking the lane at t0 and never letting go:
+//
+//   t0        holder acquires, elapsed measurement starts here and only here
+//   t0+100    waiter arrives with a 400 ms budget, blocks, and times out
+//   t0+500    late caller arrives with a 450 ms budget
+//
+// A correct lane measures 500 ms of holding at the late arrival, which is more
+// than 450, so the late caller fails fast. An implementation that let the
+// waiter stamp itself as holder measures only the 400 ms since the waiter
+// arrived, which is under 450, so the late caller would queue behind a stuck
+// run instead. The two paths are told apart by their wording.
+// ---------------------------------------------------------------------------
+static void test_waiter_does_not_become_the_holder() {
+    InferenceLane lane("stamp-model");
+    constexpr int kWaiterArrivesAfterMs = 100;
+    constexpr int kWaiterBudgetMs = 400;
+    constexpr int kLateBudgetMs = 450;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B8 holder took the lane");
+
+    // Margin: the waiter must not fail fast on arrival, which needs the
+    // holder's elapsed time to stay under 400 ms. Arriving at 100 ms leaves
+    // 300 ms of slack, so oversleeping under load does not flip the path.
+    nap(kWaiterArrivesAfterMs);
+
+    EntryOutcome waiter_outcome;
+    std::thread waiter([&] { waiter_outcome = try_entry(lane, kWaiterBudgetMs); });
+    waiter.join();
+    check(!waiter_outcome.acquired, "B8 mid-queue waiter did not acquire");
+    check(mentions(waiter_outcome.message, kTimedOutPhrase),
+          "B8 mid-queue waiter waited out its budget and timed out");
+
+    // Margin: the holder has now been in the lane for at least 500 ms against a
+    // 450 ms budget. Load lengthens both sleeps, so the measured age only grows
+    // and the fail-fast path only becomes more certain.
+    const EntryOutcome late = try_entry(lane, kLateBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!late.acquired, "B8 late caller did not acquire");
+    check(mentions(late.message, kStillRunningPhrase),
+          "B8 elapsed time is still measured from the real holder's acquisition");
+    check(late.took_ms < 200,
+          "B8 late caller failed fast rather than queueing behind the holder");
+}
+
+// ---------------------------------------------------------------------------
+// B8, other direction: the age of a run is measured from the moment its holder
+// acquired, not from the moment that holder arrived. A caller that queued for a
+// while and then got in is starting a fresh run, and its time in the queue must
+// not be billed to it: if it were, every handover would hand the new holder a
+// head start towards looking overrun, and short-budget callers would be turned
+// away from a run that has barely begun.
+//
+//   t0        first holder acquires and holds for 300 ms
+//   t0        second caller arrives and queues
+//   t0+300    second caller acquires, so its own run age is ~0 here
+//   t0+300    a third caller arrives with a 300 ms budget
+//
+// Correct: the third caller sees a run that just started, so it queues and then
+// times out. Billing the queue time to the second caller would show a 300 ms old
+// run instead, and the third caller would be turned away as an overrun.
+// ---------------------------------------------------------------------------
+static void test_run_age_starts_at_acquisition() {
+    InferenceLane lane("handover-model");
+    constexpr int kFirstHoldMs = 300;
+    constexpr int kThirdBudgetMs = 200;
+
+    Signal first_held;
+    Signal handed_over;
+    Signal release_second;
+
+    std::thread first([&] {
+        LaneEntry entry(lane, 0);
+        first_held.raise();
+        nap(kFirstHoldMs);
+    });
+    // Ordering matters only for the diagnosis, not for the assertion: waiting
+    // for the first holder guarantees the second caller really does queue, which
+    // is what gives it queue time to be wrongly billed for.
+    check(first_held.await(5000), "B8 first holder took the lane");
+
+    std::thread second([&] {
+        LaneEntry entry(lane, 0);
+        handed_over.raise();
+        release_second.await(10000);
+    });
+    check(handed_over.await(10000), "B8 queued caller was handed the lane");
+
+    // Margin: the new holder's run is a few ms old against a 200 ms budget, so
+    // this caller must queue. Load can only add a few ms of handover latency,
+    // well inside that slack, while it lengthens the queue time that the buggy
+    // version would bill, making the bug more visible rather than less.
+    const EntryOutcome third = try_entry(lane, kThirdBudgetMs);
+    release_second.raise();
+    second.join();
+    first.join();
+
+    check(!third.acquired, "B8 lane was still held by the queued caller");
+    check(mentions(third.message, kTimedOutPhrase),
+          "B8 a fresh holder's run age excludes the time it spent queueing");
+}
+
+// ---------------------------------------------------------------------------
+// B10: both failure modes carry a usable message, and the fail-fast one reports
+// a measurement rather than diagnosing a cause.
+// ---------------------------------------------------------------------------
+static void test_failure_messages_are_diagnosable() {
+    InferenceLane lane("diagnosable-model");
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B10 holder took the lane");
+
+    // The two budgets have to straddle the run's age, or both callers take the
+    // same path and the comparisons below are between two fail-fast messages
+    // that differ only in the budget they print.
+    //
+    // Margin: 30 ms is far under the ~120 ms age, and load only ages the run
+    // further, so `fast` fails fast. 400 ms is far over it, with the same
+    // slack and the same safe direction as the B3 test, so `slow` queues and
+    // then times out.
+    nap(120);
+    const EntryOutcome fast = try_entry(lane, 30);
+    const EntryOutcome slow = try_entry(lane, 400);
+    release.raise();
+    holder.join();
+
+    check(mentions(fast.message, kStillRunningPhrase) &&
+              mentions(slow.message, kTimedOutPhrase),
+          "B10 one caller took the fail-fast path and the other timed out");
+    check(fast.message != slow.message,
+          "B10 the two failure modes do not share one message");
+    check(!fast.message.empty() && !slow.message.empty(),
+          "B10 both failures carry text");
+    check(mentions(fast.message, "diagnosable-model") &&
+              mentions(slow.message, "diagnosable-model"),
+          "B10 both failures name the model");
+    // The fail-fast wording must not accuse the run of being stuck: a short
+    // budget meeting a legitimately long run reaches this path too.
+    for (const char *verdict : {"stuck", "wedged", "hung", "deadlock"}) {
+        check(!mentions(fast.message, verdict),
+              std::string("B10 fail-fast message avoids diagnosing '") +
+                  verdict + "'");
+    }
+}
+
+int main() {
+    // Last resort only. Every threaded test above is individually bounded, so
+    // this should never fire; it exists so that an implementation which parks a
+    // thread forever still ends the job instead of occupying a CI runner.
+    std::thread watchdog([] {
+        std::this_thread::sleep_for(std::chrono::seconds(30));
+        fprintf(stderr, "FAIL: watchdog fired, an inference_lane test hung\n");
+        fflush(stderr);
+        std::_Exit(1);
+    });
+    watchdog.detach();
+
+    test_budget_negotiation();
+    test_overrun_predicate();
+    test_mutual_exclusion();
+    test_unbounded_waits_out_the_holder();
+    test_bounded_wait_times_out();
+    test_fail_fast_against_a_long_run();
+    test_idle_lane_is_never_stuck();
+    test_release_on_exception();
+    test_waiter_does_not_become_the_holder();
+    test_run_age_starts_at_acquisition();
+    test_failure_messages_are_diagnosable();
+
+    if (failures == 0) {
+        fprintf(stderr, "\nAll inference_lane tests passed.\n");
+        return 0;
+    }
+    fprintf(stderr, "\n%d inference_lane test(s) failed.\n", failures);
+    return 1;
+}

backend/cpp/audio-cpp/live_watchdog.cpp

@@ -0,0 +1,75 @@
+#include "live_watchdog.h"
+
+#include <utility>
+
+namespace audiocpp_backend {
+
+IdleWatchdog::IdleWatchdog(std::chrono::milliseconds window,
+                           std::function<void()> on_idle)
+    : window_(window), on_idle_(std::move(on_idle)),
+      last_(std::chrono::steady_clock::now()) {
+    if (window_.count() <= 0) {
+        // Disabled: no thread at all, rather than a thread with an infinite
+        // deadline. A thread that exists is a thread that has to be joined on
+        // every exit path, and there is nothing for this one to do.
+        return;
+    }
+    thread_ = std::thread([this] { run(); });
+}
+
+IdleWatchdog::~IdleWatchdog() { disarm(); }
+
+void IdleWatchdog::touch() {
+    std::lock_guard<std::mutex> lock(mu_);
+    last_ = std::chrono::steady_clock::now();
+    // Deliberately does NOT notify. The waiter recomputes its deadline from
+    // last_ every time it wakes, so a touch that lands mid-window is picked up
+    // when the old deadline expires, and a touch is the hot path: it runs once
+    // per frame on the wire.
+}
+
+void IdleWatchdog::disarm() {
+    {
+        std::lock_guard<std::mutex> lock(mu_);
+        stop_ = true;
+    }
+    cv_.notify_all();
+    if (thread_.joinable()) {
+        thread_.join();
+    }
+}
+
+bool IdleWatchdog::fired() const {
+    std::lock_guard<std::mutex> lock(mu_);
+    return fired_;
+}
+
+void IdleWatchdog::run() {
+    std::unique_lock<std::mutex> lock(mu_);
+    while (!stop_) {
+        const auto deadline = last_ + window_;
+        if (cv_.wait_until(lock, deadline, [this] { return stop_; })) {
+            return; // disarmed
+        }
+        // The deadline passed, but last_ may have moved while this thread was
+        // waiting, and a condition variable may also wake spuriously. Re-read
+        // it: without this check a touch that landed mid-window would still be
+        // followed by a cancellation, i.e. a live client cut off mid-sentence.
+        if (std::chrono::steady_clock::now() < last_ + window_) {
+            continue;
+        }
+        fired_ = true;
+        auto callback = on_idle_;
+        lock.unlock();
+        if (callback) {
+            callback();
+        }
+        return; // one shot
+    }
+}
+
+bool live_frame_carries_audio(bool has_audio, bool pcm_empty) {
+    return has_audio && !pcm_empty;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/live_watchdog.h

@@ -0,0 +1,99 @@
+#pragma once
+
+// A one-shot idle timer for a bidirectional stream. Standard library only, so
+// it is tested without an audio.cpp checkout or a gRPC server.
+//
+// WHY IT EXISTS. AudioTranscriptionLive holds the model's inference lane for the
+// whole stream, because the streaming session is stateful and a concurrent run
+// would interleave two callers' audio. Every other RPC in this backend holds the
+// lane across COMPUTE, or across a write to a slow reader, and both of those
+// terminate on their own. A live stream instead blocks in a client-driven read,
+// and a peer that goes silent WITHOUT closing the stream never terminates
+// anything: the lane stays taken and every other RPC against that model queues
+// behind a client that has stopped speaking. A websocket death does cancel the
+// RPC and free it, but "the peer's TCP connection eventually dies" is not a
+// bound anyone can state, so this supplies one.
+//
+// HOW IT ENDS THE STREAM, and the part that is not obvious: gRPC's synchronous
+// ServerReaderWriter::Read has no timeout and cannot be given one. The only way
+// to unblock it from another thread is ServerContext::TryCancel, which is what
+// the callback is for. That means the client sees CANCELLED rather than whatever
+// status the handler goes on to return: the returned status is for the server's
+// own record. Releasing the lane is the point.
+//
+// ONE SHOT on purpose. Once the callback has run the stream is being torn down,
+// so there is nothing left to watch, and a repeating timer would call TryCancel
+// on a context the handler may already have returned from.
+
+#include <chrono>
+#include <condition_variable>
+#include <functional>
+#include <mutex>
+#include <thread>
+
+namespace audiocpp_backend {
+
+class IdleWatchdog {
+public:
+    // A window that is not positive DISABLES the watchdog entirely: no thread is
+    // started and fired() never becomes true. That is the operator's escape
+    // hatch for a client that legitimately holds a stream open through long
+    // pauses, and it is why the option carrying it documents 0 as "no limit"
+    // rather than as "expire immediately".
+    //
+    // `on_idle` runs on the watchdog's own thread with no lock held. It must be
+    // safe to call while the watched thread is blocked in a read, which is the
+    // only reason this class exists; ServerContext::TryCancel is documented as
+    // exactly that.
+    IdleWatchdog(std::chrono::milliseconds window, std::function<void()> on_idle);
+
+    // Joins the thread, so the callback can safely capture anything that
+    // outlives this object's scope and nothing else has to be reasoned about.
+    ~IdleWatchdog();
+
+    IdleWatchdog(const IdleWatchdog &) = delete;
+    IdleWatchdog &operator=(const IdleWatchdog &) = delete;
+
+    // Restarts the window. Call it whenever the peer proves it is still there.
+    void touch();
+
+    // Stops watching and joins. Idempotent, and REQUIRED before any long
+    // non-read work the window must not cover: the caller's own decode is not
+    // the peer going quiet, and cancelling in the middle of it would throw away
+    // a transcript the client is waiting for.
+    void disarm();
+
+    // True once the window elapsed and the callback ran. Stays true after
+    // disarm, so the caller can tell "the peer closed" from "we cancelled it".
+    bool fired() const;
+
+private:
+    void run();
+
+    const std::chrono::milliseconds window_;
+    std::function<void()> on_idle_;
+
+    mutable std::mutex mu_;
+    std::condition_variable cv_;
+    std::chrono::steady_clock::time_point last_;
+    bool stop_ = false;
+    bool fired_ = false;
+    std::thread thread_;
+};
+
+// Whether one message read off a live stream is a frame the decoder can
+// actually consume, which is the ONLY thing that counts as the peer proving it
+// is still there.
+//
+// Split out of the read loop so the distinction is testable, and because
+// getting it wrong is silent. The loop used to touch the watchdog on ANY
+// message, before it filtered on has_audio and on an empty pcm field, so a peer
+// writing unset-oneof or zero-length frames faster than the window held the
+// lane forever: no audio was ever fed, no work was ever done, and the timer
+// that exists to break exactly that grip was reset by the frames doing it.
+// There is one lane per model and one model per process, so that is a single
+// client denying the whole backend. The thrown message already said "no audio
+// frame arrived"; this is the code agreeing with it.
+bool live_frame_carries_audio(bool has_audio, bool pcm_empty);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/live_watchdog_test.cpp

@@ -0,0 +1,225 @@
+// Unit tests for the live stream idle watchdog. Standard library only; the
+// harness compiles this as a single translation unit, so the implementation is
+// included directly.
+//
+// These are TIMING tests, which is unavoidable: what is under test is a
+// deadline. Every window here is short and every assertion waits several
+// multiples of it, so a loaded machine slows the test down rather than
+// flipping its answer. The one thing never asserted is how SOON something
+// happens, only that it eventually does or never does.
+
+#include "live_watchdog.cpp"
+
+#include <atomic>
+#include <cstdio>
+#include <string>
+#include <thread>
+
+using namespace std::chrono_literals;
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+// A peer that goes quiet without closing. This is the whole point: the lane it
+// holds has to come back.
+static void test_it_fires_when_nothing_touches_it() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(100ms, [&calls] { ++calls; });
+    std::this_thread::sleep_for(600ms);
+    check(watchdog.fired(), "a window that elapses untouched fires");
+    check(calls.load() == 1, "the callback runs exactly once, not once per window");
+}
+
+// A peer that is still speaking must never be cut off. Touches land at a third
+// of the window, for six windows' worth of wall clock.
+static void test_touching_defers_it_indefinitely() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(300ms, [&calls] { ++calls; });
+    for (int i = 0; i < 20; ++i) {
+        std::this_thread::sleep_for(100ms);
+        watchdog.touch();
+    }
+    check(!watchdog.fired(),
+          "a stream touched inside every window is never cancelled");
+    check(calls.load() == 0, "no callback runs while the peer is still there");
+}
+
+// Disarm is what the handler calls when the read side closes, before a decode
+// that can take longer than the window. Firing after that would throw away the
+// transcript the client is waiting for.
+static void test_disarm_stops_it_before_the_window() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(200ms, [&calls] { ++calls; });
+    std::this_thread::sleep_for(20ms);
+    watchdog.disarm();
+    std::this_thread::sleep_for(600ms);
+    check(!watchdog.fired(), "a disarmed watchdog does not fire");
+    check(calls.load() == 0, "a disarmed watchdog runs no callback");
+}
+
+static void test_disarm_is_idempotent() {
+    audiocpp_backend::IdleWatchdog watchdog(50ms, [] {});
+    watchdog.disarm();
+    watchdog.disarm();
+    watchdog.disarm();
+    check(true, "disarming three times joins once and does not abort");
+}
+
+// The operator's escape hatch, for a client that legitimately holds a stream
+// open through long pauses. Not "expire immediately", which is what a naive
+// reading of a zero timeout would give.
+static void test_a_non_positive_window_disables_it() {
+    std::atomic<int> calls{0};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(0ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(300ms);
+        check(!watchdog.fired(), "a zero window never fires");
+    }
+    {
+        audiocpp_backend::IdleWatchdog watchdog(-5ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(300ms);
+        check(!watchdog.fired(), "a negative window never fires");
+    }
+    check(calls.load() == 0, "a disabled watchdog runs no callback");
+}
+
+// fired() has to survive the disarm, because the handler reads it AFTER the
+// read loop ends to tell "the peer closed" from "we cancelled the peer", and
+// those two get different statuses.
+static void test_fired_survives_a_later_disarm() {
+    audiocpp_backend::IdleWatchdog watchdog(80ms, [] {});
+    std::this_thread::sleep_for(500ms);
+    watchdog.disarm();
+    check(watchdog.fired(), "a watchdog that fired still says so after disarm");
+}
+
+// The destructor joins, so a callback capturing the handler's frame cannot run
+// after that frame is gone. Without the join this is a use after free that only
+// shows up under load.
+//
+// The window is LONGER than the scope on purpose. An earlier version of this
+// test slept past the window inside the scope, so the callback had already run
+// by the time the object was destroyed and a destructor that DETACHED the thread
+// instead of joining it passed unnoticed. Mutation testing is what found that;
+// the shape below kills it, because a detached thread wakes after the object is
+// gone and calls a callback that must never run.
+static void test_the_destructor_joins() {
+    std::atomic<int> calls{0};
+    std::atomic<bool> alive{true};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(200ms, [&calls, &alive] {
+            check(alive.load(),
+                  "the callback never runs after the watched scope ended");
+            ++calls;
+        });
+        std::this_thread::sleep_for(20ms);
+    }
+    alive.store(false);
+    std::this_thread::sleep_for(600ms);
+    check(calls.load() == 0,
+          "destruction stops the timer rather than leaving it running against a "
+          "dead frame");
+}
+
+// The other half of that pair: a callback that DOES fire inside the scope runs
+// exactly once, so the test above is not passing merely because nothing ever
+// fires.
+static void test_a_firing_watchdog_still_joins_cleanly() {
+    std::atomic<int> calls{0};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(50ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(400ms);
+    }
+    check(calls.load() == 1, "the callback ran once, inside the scope");
+}
+
+// The predicate the live read loop filters on.
+static void test_only_a_frame_with_audio_counts() {
+    using audiocpp_backend::live_frame_carries_audio;
+    check(live_frame_carries_audio(true, false),
+          "a frame with a non-empty pcm field carries audio");
+    check(!live_frame_carries_audio(true, true),
+          "an empty pcm field does not");
+    check(!live_frame_carries_audio(false, false),
+          "an unset audio oneof does not, whatever the pcm field looks like");
+    check(!live_frame_carries_audio(false, true),
+          "and neither does an unset oneof with an empty pcm field");
+}
+
+// The defect this closes, expressed as behaviour rather than as a call order:
+// a peer writing frames the decoder cannot consume, faster than the window,
+// used to hold the model's only inference lane forever, because the read loop
+// touched the watchdog before it filtered them out. One lane per model and one
+// model per process, so that is a single client denying the whole backend,
+// which is exactly what the watchdog exists to prevent.
+static void test_empty_frames_do_not_hold_the_lane() {
+    std::atomic<int> cancels{0};
+    std::atomic<bool> stop{false};
+    audiocpp_backend::IdleWatchdog watchdog(80ms, [&cancels] { ++cancels; });
+
+    // The read loop with the real filter in it: frames arrive continuously,
+    // none of them carries audio, and only a frame that does may touch.
+    std::thread peer([&] {
+        while (!stop.load()) {
+            if (audiocpp_backend::live_frame_carries_audio(false, true)) {
+                watchdog.touch();
+            }
+            std::this_thread::sleep_for(5ms);
+        }
+    });
+
+    std::this_thread::sleep_for(600ms);
+    stop.store(true);
+    peer.join();
+    watchdog.disarm();
+
+    check(cancels.load() == 1,
+          "a flood of frames with no audio in them still releases the lane");
+
+    // The mirror image, so this cannot pass merely because the watchdog always
+    // fires: a peer that keeps sending audio is left alone, exactly as before.
+    std::atomic<int> live_cancels{0};
+    std::atomic<bool> live_stop{false};
+    audiocpp_backend::IdleWatchdog live(80ms, [&live_cancels] { ++live_cancels; });
+    std::thread speaker([&] {
+        while (!live_stop.load()) {
+            if (audiocpp_backend::live_frame_carries_audio(true, false)) {
+                live.touch();
+            }
+            std::this_thread::sleep_for(5ms);
+        }
+    });
+    std::this_thread::sleep_for(600ms);
+    live_stop.store(true);
+    speaker.join();
+    live.disarm();
+    check(live_cancels.load() == 0,
+          "a peer that keeps sending audio is never cancelled");
+}
+
+int main() {
+    test_it_fires_when_nothing_touches_it();
+    test_touching_defers_it_indefinitely();
+    test_disarm_stops_it_before_the_window();
+    test_disarm_is_idempotent();
+    test_a_non_positive_window_disables_it();
+    test_fired_survives_a_later_disarm();
+    test_the_destructor_joins();
+    test_a_firing_watchdog_still_joins_cleanly();
+    test_only_a_frame_with_audio_counts();
+    test_empty_frames_do_not_hold_the_lane();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all live_watchdog checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/loaded_model.cpp

@@ -0,0 +1,802 @@
+#include "loaded_model.h"
+
+#include "family_gate.h"
+
+#include "engine/framework/assets/tensor_source.h"
+
+#include <algorithm>
+#include <cstddef>
+#include <cstdint>
+#include <filesystem>
+#include <utility>
+
+#if defined(__APPLE__)
+#include <mach-o/dyld.h>
+#include <cstdint>
+#include <vector>
+#elif defined(__linux__)
+#include <unistd.h>
+#endif
+
+namespace audiocpp_backend {
+
+// --------------------------------------------------------------------------
+// Enum coupling
+//
+// audiocpp_backend::Task mirrors engine::runtime::VoiceTaskKind positionally so
+// capability_routing can stay stdlib-only and testable without an audio.cpp
+// checkout. Nothing about that mirroring is enforced by the type system, and a
+// drift is silent in the worst possible way: every unit still compiles, every
+// test still passes, and the backend runs a different task than the one the
+// caller asked for.
+//
+// Two mechanisms pin it, and both are needed because they catch different edits:
+//
+//   1. The assertions below pin every enumerator's value on both sides. An
+//      insertion or a reorder anywhere before the last member shifts the values
+//      after it and fails the build here.
+//   2. An enumerator APPENDED after the last one shifts nothing, so no value
+//      assertion can see it. What sees it is the switch in from_engine_task,
+//      which covers the engine enum with no `default:` label. CMakeLists.txt
+//      compiles this file with -Werror=switch so that omission is an error
+//      rather than a warning nobody reads.
+//
+// Neither mechanism catches a pure RENAME of an upstream enumerator, but that
+// does not need catching: the switch stops naming an enumerator that exists and
+// the build fails on its own.
+// --------------------------------------------------------------------------
+
+namespace {
+
+constexpr int kEngine(engine::runtime::VoiceTaskKind kind) {
+    return static_cast<int>(kind);
+}
+constexpr int kMirror(Task task) { return static_cast<int>(task); }
+
+} // namespace
+
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Vad) == 0, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Asr) == 1, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Diarization) == 2, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SourceSeparation) == 3, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::AudioGeneration) == 4, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Tts) == 5, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceCloning) == 6, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceConversion) == 7, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SpeechToSpeech) == 8, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Alignment) == 9, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceDesign) == 10, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SpeakerRecognition) == 11, "VoiceTaskKind drifted");
+// The last member. Pinning it pins the member count too, as long as the
+// enumerators stay contiguous and unassigned, which upstream's declaration is.
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Svc) == 12,
+              "engine::runtime::VoiceTaskKind gained, lost or reordered a member. "
+              "audiocpp_backend::Task mirrors it positionally: update capability_routing.h, "
+              "to_engine_task and from_engine_task together, then move this pin.");
+
+static_assert(kMirror(Task::Vad) == 0, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Asr) == 1, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Diarization) == 2, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SourceSeparation) == 3, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::AudioGeneration) == 4, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Tts) == 5, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceCloning) == 6, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceConversion) == 7, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SpeechToSpeech) == 8, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Alignment) == 9, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceDesign) == 10, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SpeakerRecognition) == 11, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Svc) == 12, "Task drifted from VoiceTaskKind");
+
+static_assert(static_cast<int>(engine::runtime::RunMode::Offline) == 0, "RunMode drifted");
+static_assert(static_cast<int>(engine::runtime::RunMode::Streaming) == 1,
+              "engine::runtime::RunMode gained, lost or reordered a member. "
+              "audiocpp_backend::Mode mirrors it positionally.");
+static_assert(static_cast<int>(Mode::Offline) == 0, "Mode drifted from RunMode");
+static_assert(static_cast<int>(Mode::Streaming) == 1, "Mode drifted from RunMode");
+
+namespace {
+
+engine::core::BackendType parse_backend_type(const std::string &value) {
+    if (value == "cuda") {
+        return engine::core::BackendType::Cuda;
+    }
+    if (value == "vulkan") {
+        return engine::core::BackendType::Vulkan;
+    }
+    if (value == "metal") {
+        return engine::core::BackendType::Metal;
+    }
+    if (value == "best") {
+        return engine::core::BackendType::BestAvailable;
+    }
+    if (value == "cpu" || value.empty()) {
+        return engine::core::BackendType::Cpu;
+    }
+    throw ConfigError("audio-cpp: unknown backend option '" + value +
+                      "'. Known backends: cpu, cuda, vulkan, metal, best");
+}
+
+std::filesystem::path executable_directory() {
+#if defined(__APPLE__)
+    std::uint32_t size = 0;
+    _NSGetExecutablePath(nullptr, &size);
+    std::vector<char> buffer(size + 1, '\0');
+    if (_NSGetExecutablePath(buffer.data(), &size) != 0) {
+        return std::filesystem::current_path();
+    }
+    return std::filesystem::path(buffer.data()).parent_path();
+#elif defined(__linux__)
+    std::error_code ec;
+    const auto self = std::filesystem::read_symlink("/proc/self/exe", ec);
+    if (ec) {
+        return std::filesystem::current_path();
+    }
+    return self.parent_path();
+#else
+    return std::filesystem::current_path();
+#endif
+}
+
+// Runs the load gate. Throws ConfigError rather than returning a decision,
+// because the only caller is a delegating constructor whose member initializer
+// list has nowhere to put a failure.
+std::string require_family(const std::string &resolved_path,
+                           const ModelOptions &options) {
+    const std::filesystem::path path(resolved_path);
+    std::error_code ec;
+    if (!std::filesystem::exists(path, ec)) {
+        throw ConfigError("audio-cpp: model path does not exist: " + resolved_path);
+    }
+
+    const bool is_gguf = path_looks_like_gguf(resolved_path);
+    // Only a GGUF is asked for embedded metadata. A directory has no single
+    // file to read it from, and probing one would make the gate's refusal
+    // depend on which file happened to be inside.
+    const std::string embedded =
+        is_gguf ? read_gguf_family(resolved_path) : std::string();
+
+    const FamilyDecision decision =
+        decide_family(is_gguf, embedded, options.family);
+    if (!decision.ok) {
+        throw ConfigError(decision.error);
+    }
+    return decision.family;
+}
+
+// Refuses a GGUF whose weights are stored in a dtype the family cannot survive.
+//
+// The POLICY lives in family_gate's weight_dtype_is_supported, which is
+// stdlib-only and therefore testable; this is only the part that needs a file
+// and an engine to read one. See the table there for why an entry exists and
+// what has to be run before deleting it.
+//
+// Only GGUF paths are inspected. A directory of safetensors carries its dtypes
+// per file and has not been tested against this failure, so it is passed
+// through rather than guessed at.
+void require_supported_weight_dtypes(const std::string &family,
+                                     const std::string &resolved_path) {
+    // Asked as "is there an entry", not as "is the description non-empty": an
+    // entry with an empty allow list describes a family that can run nothing,
+    // and reading the description would skip the check on exactly that entry
+    // while weight_dtype_is_supported refused every dtype. No such entry exists
+    // today; the two questions are different ones and only one of them is this
+    // guard's.
+    if (!family_has_weight_dtype_allow_list(family) ||
+        !path_looks_like_gguf(resolved_path)) {
+        return;
+    }
+
+    std::string offending_dtype;
+    std::string offending_tensor;
+    try {
+        const auto source =
+            engine::assets::open_tensor_source(std::filesystem::path(resolved_path));
+        if (source == nullptr) {
+            return;
+        }
+        for (const auto &tensor : source->tensors()) {
+            if (!weight_dtype_is_supported(family, tensor.dtype)) {
+                offending_dtype = tensor.dtype;
+                offending_tensor = tensor.name;
+                break;
+            }
+        }
+    } catch (const std::exception &) {
+        // Unreadable as a tensor source. Not this guard's problem to report:
+        // the registry load below produces a message naming the real fault, and
+        // refusing here would turn every unusual packaging into this error.
+        return;
+    }
+
+    if (offending_dtype.empty()) {
+        return;
+    }
+    throw ConfigError(
+        "audio-cpp: family '" + family + "' cannot run weights stored as '" +
+        offending_dtype + "' (tensor '" + offending_tensor + "' in " +
+        resolved_path +
+        "); it aborts the backend process on the first request rather than "
+        "failing the request. Use the 'orig' GGUF package, whose weights are " +
+        supported_weight_dtypes(family) + ".");
+}
+
+} // namespace
+
+engine::runtime::VoiceTaskKind to_engine_task(Task task) {
+    using K = engine::runtime::VoiceTaskKind;
+    // An explicit switch, never a cast: a cast would keep compiling through
+    // exactly the drift the assertions above exist to catch.
+    switch (task) {
+    case Task::Vad:                return K::Vad;
+    case Task::Asr:                return K::Asr;
+    case Task::Diarization:        return K::Diarization;
+    case Task::SourceSeparation:   return K::SourceSeparation;
+    case Task::AudioGeneration:    return K::AudioGeneration;
+    case Task::Tts:                return K::Tts;
+    case Task::VoiceCloning:       return K::VoiceCloning;
+    case Task::VoiceConversion:    return K::VoiceConversion;
+    case Task::SpeechToSpeech:     return K::SpeechToSpeech;
+    case Task::Alignment:          return K::Alignment;
+    case Task::VoiceDesign:        return K::VoiceDesign;
+    case Task::SpeakerRecognition: return K::SpeakerRecognition;
+    case Task::Svc:                return K::Svc;
+    }
+    // Unreachable for any valid enumerator. No `default:` label, so -Wswitch
+    // still reports a member this switch stops covering.
+    return K::Vad;
+}
+
+Task from_engine_task(engine::runtime::VoiceTaskKind kind) {
+    using K = engine::runtime::VoiceTaskKind;
+    switch (kind) {
+    case K::Vad:                return Task::Vad;
+    case K::Asr:                return Task::Asr;
+    case K::Diarization:        return Task::Diarization;
+    case K::SourceSeparation:   return Task::SourceSeparation;
+    case K::AudioGeneration:    return Task::AudioGeneration;
+    case K::Tts:                return Task::Tts;
+    case K::VoiceCloning:       return Task::VoiceCloning;
+    case K::VoiceConversion:    return Task::VoiceConversion;
+    case K::SpeechToSpeech:     return Task::SpeechToSpeech;
+    case K::Alignment:          return Task::Alignment;
+    case K::VoiceDesign:        return Task::VoiceDesign;
+    case K::SpeakerRecognition: return Task::SpeakerRecognition;
+    case K::Svc:                return Task::Svc;
+    }
+    return Task::Vad;
+}
+
+engine::runtime::RunMode to_engine_mode(Mode mode) {
+    using M = engine::runtime::RunMode;
+    switch (mode) {
+    case Mode::Offline:   return M::Offline;
+    case Mode::Streaming: return M::Streaming;
+    }
+    return M::Offline;
+}
+
+Mode from_engine_mode(engine::runtime::RunMode mode) {
+    using M = engine::runtime::RunMode;
+    switch (mode) {
+    case M::Offline:   return Mode::Offline;
+    case M::Streaming: return Mode::Streaming;
+    }
+    return Mode::Offline;
+}
+
+Capabilities to_capabilities(const std::string &family,
+                             const engine::runtime::CapabilitySet &set) {
+    Capabilities caps;
+    caps.family = family;
+    caps.tasks.reserve(set.supported_tasks.size());
+    for (const auto &supported : set.supported_tasks) {
+        TaskCapability capability;
+        capability.task = from_engine_task(supported.task);
+        capability.modes.reserve(supported.modes.size());
+        for (const auto mode : supported.modes) {
+            capability.modes.push_back(from_engine_mode(mode));
+        }
+        caps.tasks.push_back(std::move(capability));
+    }
+    return caps;
+}
+
+std::string read_gguf_family(const std::string &path) {
+    try {
+        const auto spec =
+            engine::assets::read_gguf_embedded_model_spec(std::filesystem::path(path));
+        if (spec.has_value()) {
+            return spec->family;
+        }
+    } catch (...) {
+        // A file that is not a readable GGUF simply has no family. The load
+        // gate turns that into a clear refusal; a throw here would surface as
+        // an opaque internal error during backend probing.
+    }
+    return {};
+}
+
+std::string resolve_model_path(const std::string &model_path_dir,
+                               const std::string &model_file,
+                               const std::string &model_name) {
+    const std::string candidate = !model_file.empty() ? model_file : model_name;
+
+    // The bundled: form is looked for in BOTH fields, and in ModelOptions.Model
+    // FIRST, because that is the only field it survives in. LocalAI fills
+    // ModelFile by joining ModelPath onto the configured model string
+    // (pkg/model/loader.go, LoadModelWithFile), so a model YAML saying
+    // `model: bundled:silero_vad` arrives here as ModelFile
+    // "/models/bundled:silero_vad" and Model "bundled:silero_vad". Testing
+    // `candidate` alone therefore made the zero-download VAD path reachable only
+    // from a hand-written LoadModel call that left ModelFile empty, and every
+    // model YAML using it failed with "model path does not exist".
+    const std::string bundled_prefix = "bundled:";
+    for (const std::string *field : {&model_name, &model_file}) {
+        if (field->rfind(bundled_prefix, 0) == 0) {
+            const std::string name = field->substr(bundled_prefix.size());
+            return (executable_directory() / "assets" / name).string();
+        }
+    }
+
+    std::filesystem::path path(candidate);
+    if (path.is_absolute() || model_path_dir.empty()) {
+        return path.string();
+    }
+    return (std::filesystem::path(model_path_dir) / path).string();
+}
+
+LoadedModel::LoadedModel(const std::string &resolved_path,
+                         const ModelOptions &options,
+                         std::string model_identity)
+    : LoadedModel(resolved_path, options, require_family(resolved_path, options),
+                  std::move(model_identity)) {}
+
+LoadedModel::LoadedModel(const std::string &resolved_path,
+                         const ModelOptions &options, std::string family,
+                         std::string model_identity)
+    : lane_(family), registry_(engine::runtime::make_default_registry()),
+      identity_(std::move(model_identity)) {
+    if (!registry_.supports_family(family)) {
+        throw ConfigError("audio-cpp: unknown audio.cpp family '" + family + "'");
+    }
+
+    // Before the load, for the same reason parse_backend_type runs before it:
+    // a refusal a metadata read can produce should not cost a full model load.
+    // More importantly it must precede the FIRST REQUEST, since that is where
+    // an unsupported dtype aborts the process rather than failing.
+    require_supported_weight_dtypes(family, resolved_path);
+
+    // Session options are built BEFORE the load, because parse_backend_type
+    // rejects an unknown backend name. Validating after the load would make
+    // `backend:cudaa` cost a full model load, on a fault a string comparison
+    // could have caught.
+    session_options_.backend.type = parse_backend_type(options.backend);
+    session_options_.backend.device = options.device;
+    if (options.threads > 0) {
+        session_options_.backend.threads = options.threads;
+    }
+    for (const auto &entry : options.session_options) {
+        session_options_.options[entry.first] = entry.second;
+    }
+
+    pinned_task_ = options.task;
+    wait_budget_ceiling_ms_ = options.busy_timeout_ms;
+    live_idle_timeout_ms_ = options.live_idle_timeout_ms;
+
+    engine::runtime::ModelLoadRequest request;
+    request.model_path = std::filesystem::path(resolved_path);
+    request.family_hint = family;
+    if (!options.model_spec_override.empty()) {
+        request.model_spec_override =
+            std::filesystem::path(options.model_spec_override);
+    }
+    for (const auto &entry : options.load_options) {
+        request.options[entry.first] = entry.second;
+    }
+
+    try {
+        model_ = registry_.load(request);
+    } catch (const std::exception &err) {
+        throw ConfigError("audio-cpp: failed to load family '" + family +
+                          "' from " + resolved_path + ": " + err.what());
+    }
+    if (model_ == nullptr) {
+        throw ConfigError("audio-cpp: the registry returned no model for " +
+                          resolved_path);
+    }
+
+    const auto &metadata = model_->metadata();
+    const auto &engine_caps = model_->capabilities();
+    variant_ = metadata.variant;
+    description_ = metadata.description;
+    languages_ = engine_caps.languages;
+    supports_timestamps_ = engine_caps.supports_timestamps;
+    capabilities_ = to_capabilities(family, engine_caps);
+}
+
+Route LoadedModel::check_can_serve(Rpc rpc, const RequestShape &shape) const {
+    const Route route = resolve_route(rpc, shape, capabilities_);
+    if (!route.ok) {
+        throw CapabilityError(route.error);
+    }
+    // Returned so a handler can act on the task before running it. It is the
+    // same route session_for will resolve, since both read the immutable
+    // capabilities_ from the same shape.
+    return route;
+}
+
+LoadedModel::Session LoadedModel::session_for(Rpc rpc, const RequestShape &shape,
+                                              LaneEntry &lane) {
+    // Proof of holding only. Nothing here reads it, and nothing should: its
+    // whole job is to make a caller that has not taken the lane fail to
+    // compile. Non-const so it cannot bind to an inline acquire(), whose
+    // temporary would be released at the end of this call.
+    (void)lane;
+    const Route route = resolve_route(rpc, shape, capabilities_);
+    if (!route.ok) {
+        throw CapabilityError(route.error);
+    }
+
+    const SessionKey key{static_cast<int>(route.task), static_cast<int>(route.mode)};
+    auto found = sessions_.find(key);
+    const bool cache_hit = found != sessions_.end();
+    if (!cache_hit) {
+        engine::runtime::TaskSpec spec;
+        spec.task = to_engine_task(route.task);
+        spec.mode = to_engine_mode(route.mode);
+        std::unique_ptr<engine::runtime::IVoiceTaskSession> created;
+        try {
+            created = model_->create_task_session(spec, session_options_);
+        } catch (const std::exception &err) {
+            // NOT a CapabilityError. The family said it supports this pair, and
+            // a throw from here is overwhelmingly an environment fault: a ggml
+            // backend .so that package.sh did not ship, an out of memory, a CUDA
+            // device that is not there. UNIMPLEMENTED would tell LocalAI and
+            // every client "this model cannot do this, never retry", and send an
+            // operator hunting a capability bug instead of a packaging one. A
+            // plain runtime_error maps to INTERNAL, which is what a fixable
+            // deployment fault should look like.
+            throw std::runtime_error(
+                std::string("audio-cpp: family '") + capabilities_.family +
+                "' advertises " + task_name(route.task) + "/" +
+                mode_name(route.mode) + " but refused to create the session: " +
+                err.what());
+        }
+        if (created == nullptr) {
+            // A null return with no throw is the family declining, which is a
+            // genuine capability answer and stays UNIMPLEMENTED.
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' returned no session for " +
+                                  task_name(route.task) + "/" +
+                                  mode_name(route.mode));
+        }
+        found = sessions_.emplace(key, std::move(created)).first;
+    }
+
+    Session session;
+    session.task = route.task;
+    session.mode = route.mode;
+    engine::runtime::IVoiceTaskSession *raw = found->second.get();
+    if (route.mode == Mode::Streaming) {
+        session.streaming =
+            dynamic_cast<engine::runtime::IStreamingVoiceTaskSession *>(raw);
+        if (session.streaming == nullptr) {
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' advertises " + task_name(route.task) +
+                                  "/streaming but its session is not streaming");
+        }
+        // Deliberately NOT reset here, though a cached streaming session does
+        // carry state across chunks. reset() is not callable at this point:
+        // silero_vad's implementation throws "session prepare() must be called
+        // before Silero VAD reset()", so resetting on a cache hit would turn an
+        // ordinary second fetch into a hard error, which is worse than the leak
+        // it would prevent.
+        //
+        // The state is instead cleared by the sequence every streaming caller
+        // owes anyway. IStreamingVoiceTaskSession::start_stream's base
+        // implementation IS a call to reset(), so a caller that runs
+        // prepare(...) then start_stream(...) at the top of each stream gets a
+        // clean session for free. See the STATE CONTRACT in loaded_model.h.
+    } else {
+        session.offline =
+            dynamic_cast<engine::runtime::IOfflineVoiceTaskSession *>(raw);
+        if (session.offline == nullptr) {
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' advertises " + task_name(route.task) +
+                                  "/offline but its session is not offline");
+        }
+    }
+    return session;
+}
+
+LaneEntry LoadedModel::acquire(int requested_timeout_ms) {
+    // Constructed straight into the return value. C++17 requires that, which is
+    // what lets an immovable type be returned at all; a named local here would
+    // not compile.
+    return LaneEntry(lane_,
+                     resolve_wait_budget_ms(wait_budget_ceiling_ms_,
+                                            requested_timeout_ms));
+}
+
+std::unique_ptr<LaneEntry> LoadedModel::acquire_owned(int requested_timeout_ms) {
+    return std::make_unique<LaneEntry>(
+        lane_,
+        resolve_wait_budget_ms(wait_budget_ceiling_ms_, requested_timeout_ms));
+}
+
+engine::runtime::TaskResult run_offline(const LoadedModel::Session &session,
+                                        const engine::runtime::TaskRequest &request,
+                                        LaneEntry &lane) {
+    // Proof of holding only, as in session_for.
+    (void)lane;
+    if (session.offline == nullptr) {
+        throw CapabilityError("audio-cpp: no offline session for this request");
+    }
+    session.offline->prepare(engine::runtime::build_preparation_request(request));
+    return session.offline->run(request);
+}
+
+namespace {
+
+engine::runtime::IStreamingVoiceTaskSession &
+require_streaming(const LoadedModel::Session &session) {
+    if (session.streaming == nullptr) {
+        throw CapabilityError("audio-cpp: no streaming session for this request");
+    }
+    return *session.streaming;
+}
+
+// Clears the stream event sink on every exit from the driver, including the
+// exception path. The session is CACHED and outlives the call that installed
+// the sink, so a std::function left behind holding references into that call's
+// frame is called with dangling captures by whoever streams next.
+class ScopedStreamSink {
+public:
+    ScopedStreamSink(engine::runtime::IStreamingVoiceTaskSession &session,
+                     engine::runtime::StreamEventCallback sink)
+        : session_(session) {
+        session_.set_stream_event_sink(std::move(sink));
+    }
+    ~ScopedStreamSink() { session_.set_stream_event_sink(nullptr); }
+
+    ScopedStreamSink(const ScopedStreamSink &) = delete;
+    ScopedStreamSink &operator=(const ScopedStreamSink &) = delete;
+
+private:
+    engine::runtime::IStreamingVoiceTaskSession &session_;
+};
+
+// Frames per chunk to feed a streaming session, from its own policy.
+//
+// FRAMES, not floats. preferred_audio_chunk_samples is a per-channel count
+// everywhere upstream sets it (nemotron_asr uses its frontend sample rate,
+// i.e. one second), and vibevoice_asr refuses a chunk whose float count is not
+// divisible by its channel count, so slicing on floats would both mis-size the
+// window and hand a family a half frame.
+std::int64_t chunk_frames_for(const engine::runtime::StreamingPolicy &policy,
+                              int sample_rate) {
+    if (policy.preferred_audio_chunk_samples > 0) {
+        return policy.preferred_audio_chunk_samples;
+    }
+    // higgs_audio_stt states its window in seconds (4.0) and leaves the sample
+    // count at zero, so this branch is real rather than defensive.
+    if (policy.preferred_audio_chunk_seconds > 0.0 && sample_rate > 0) {
+        const auto frames = static_cast<std::int64_t>(
+            policy.preferred_audio_chunk_seconds * static_cast<double>(sample_rate));
+        if (frames > 0) {
+            return frames;
+        }
+    }
+    // The interface's own default, from IStreamingVoiceTaskSession::streaming_policy.
+    return 512;
+}
+
+} // namespace
+
+void begin_stream(const LoadedModel::Session &session,
+                  const engine::runtime::TaskRequest &request, LaneEntry &lane) {
+    // Proof of holding only, as in session_for.
+    (void)lane;
+    auto &streaming = require_streaming(session);
+    // Order is load-bearing: start_stream's reset() is illegal before prepare().
+    streaming.prepare(engine::runtime::build_preparation_request(request));
+    streaming.start_stream(request);
+}
+
+engine::runtime::TaskResult run_streaming_pull(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+    begin_stream(session, request, lane);
+    while (const auto event = streaming.next_stream_event()) {
+        if (on_event) {
+            on_event(*event);
+        }
+        // No pinned family sets is_final on a pulled event, so this is not what
+        // ends the loop today; the nullopt above is. Honoured anyway, because a
+        // family that does set it is saying the stream is over and pulling once
+        // more would be asking a finished session for another chunk.
+        if (event->is_final) {
+            break;
+        }
+    }
+    return streaming.finish_stream();
+}
+
+engine::runtime::TaskResult run_streaming_audio(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const engine::runtime::AudioBuffer &audio,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+
+    const int channels = audio.channels > 0 ? audio.channels : 1;
+    // REFUSED, not rounded away, and checked before anything is touched so a
+    // refusal leaves no half-started stream on a cached session.
+    //
+    // An interleaved buffer whose float count is not a whole number of frames
+    // is a truncated input, and the integer division below would silently drop
+    // the tail floats: they would never be fed, never reach the transcript, and
+    // nothing would say so. Upstream refuses the same condition rather than
+    // tolerating it, in two places: vibevoice_asr's audio_frame_count throws
+    // "VibeVoice-ASR audio samples must be divisible by channel count"
+    // (session.cpp:70-76), and its process_audio_chunk throws the same with
+    // "streamed" in the text about the chunks this driver hands it
+    // (session.cpp:742-747).
+    //
+    // ConfigError, i.e. INVALID_ARGUMENT, because the buffer came from the
+    // caller's file. read_audio_file's positive-rate path always answers mono
+    // and so cannot reach this, but its native-rate path passes the reader's
+    // sample count through unchanged, and a driver does not get to assume which
+    // path its caller took.
+    if (audio.samples.size() % static_cast<std::size_t>(channels) != 0) {
+        throw ConfigError(
+            "audio-cpp: streaming input is not a whole number of frames: " +
+            std::to_string(audio.samples.size()) + " samples across " +
+            std::to_string(channels) + " channels");
+    }
+
+    // Installed BEFORE the stream begins, so a family that reports during
+    // start_stream is not silently dropped, and destroyed after finish_stream,
+    // because nemotron_asr emits every one of its partials from inside
+    // finalize().
+    ScopedStreamSink sink(streaming,
+                          [&on_event](const engine::runtime::StreamEvent &event) {
+                              if (on_event) {
+                                  on_event(event);
+                              }
+                          });
+
+    begin_stream(session, request, lane);
+
+    const auto total_frames =
+        static_cast<std::int64_t>(audio.samples.size() / static_cast<size_t>(channels));
+    const std::int64_t chunk_frames =
+        chunk_frames_for(streaming.streaming_policy(), audio.sample_rate);
+
+    for (std::int64_t offset = 0; offset < total_frames; offset += chunk_frames) {
+        const std::int64_t end = std::min(offset + chunk_frames, total_frames);
+        engine::runtime::AudioChunk chunk;
+        chunk.sample_rate = audio.sample_rate;
+        chunk.channels = channels;
+        // A FRAME index, which is what every span in a returned event is
+        // expressed in. vibevoice_asr adds the chunk's own frame count to it to
+        // offset the spans it reports, so a float index here would place every
+        // span of a stereo stream at twice its real time.
+        chunk.start_sample = offset;
+        chunk.samples.assign(
+            audio.samples.begin() + static_cast<std::ptrdiff_t>(offset * channels),
+            audio.samples.begin() + static_cast<std::ptrdiff_t>(end * channels));
+        const auto event = streaming.process_audio_chunk(chunk);
+        if (on_event) {
+            on_event(event);
+        }
+    }
+    return streaming.finish_stream();
+}
+
+engine::runtime::TaskResult run_streaming_live(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<bool(std::vector<float> &)> &next_frames,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+
+    // The contract is the only thing that says what rate and layout the frames
+    // about to arrive are in, and prepare() needs it: see the header.
+    if (!request.audio_input.has_value()) {
+        throw ConfigError(
+            "audio-cpp: a live streaming request carries no audio contract");
+    }
+    const int sample_rate = request.audio_input->sample_rate;
+    const int channels =
+        request.audio_input->channels > 0 ? request.audio_input->channels : 1;
+
+    // Installed BEFORE the stream begins and cleared on every exit, including
+    // the exception path, for the reasons spelled out in run_streaming_audio.
+    ScopedStreamSink sink(streaming,
+                          [&on_event](const engine::runtime::StreamEvent &event) {
+                              if (on_event) {
+                                  on_event(event);
+                              }
+                          });
+
+    begin_stream(session, request, lane);
+
+    const std::int64_t chunk_frames =
+        chunk_frames_for(streaming.streaming_policy(), sample_rate);
+    // chunk_frames_for never returns a non-positive count, so this is never
+    // zero and the accumulation loop below always terminates.
+    const std::size_t chunk_floats = static_cast<std::size_t>(chunk_frames) *
+                                     static_cast<std::size_t>(channels);
+
+    std::int64_t fed_frames = 0;
+    const auto feed = [&](std::vector<float> samples) {
+        engine::runtime::AudioChunk chunk;
+        chunk.sample_rate = sample_rate;
+        chunk.channels = channels;
+        // A FRAME index, counted across the whole stream: vibevoice_asr offsets
+        // every span it reports by it, so restarting it per chunk would put
+        // every word at the top of the recording.
+        chunk.start_sample = fed_frames;
+        chunk.samples = std::move(samples);
+        fed_frames +=
+            static_cast<std::int64_t>(chunk.samples.size()) / channels;
+        const auto event = streaming.process_audio_chunk(chunk);
+        if (on_event) {
+            on_event(event);
+        }
+    };
+
+    std::vector<float> pending;
+    std::vector<float> incoming;
+    while (true) {
+        incoming.clear();
+        if (!next_frames(incoming)) {
+            break;
+        }
+        pending.insert(pending.end(), incoming.begin(), incoming.end());
+        while (pending.size() >= chunk_floats) {
+            std::vector<float> window(pending.begin(),
+                                      pending.begin() +
+                                          static_cast<std::ptrdiff_t>(chunk_floats));
+            pending.erase(pending.begin(),
+                          pending.begin() +
+                              static_cast<std::ptrdiff_t>(chunk_floats));
+            feed(std::move(window));
+        }
+    }
+
+    if (!pending.empty()) {
+        // The tail is whatever did not fill a window. Refused rather than
+        // truncated when it is not a whole number of frames, exactly as in
+        // run_streaming_audio: the division above would drop the stray floats
+        // from the transcript with no diagnostic. Unreachable for a mono live
+        // stream, which is every live stream today.
+        if (pending.size() % static_cast<std::size_t>(channels) != 0) {
+            throw ConfigError(
+                "audio-cpp: live stream ended mid-frame: " +
+                std::to_string(pending.size()) + " trailing samples across " +
+                std::to_string(channels) + " channels");
+        }
+        feed(std::move(pending));
+    }
+
+    if (fed_frames == 0) {
+        // Nothing was spoken. See the header: finalizing an empty stream is not
+        // legal for every family, and an empty transcript is the truthful
+        // answer rather than an engine-internal INTERNAL.
+        return engine::runtime::TaskResult{};
+    }
+    return streaming.finish_stream();
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/loaded_model.h

@@ -0,0 +1,378 @@
+#pragma once
+
+// Owns one audio.cpp model for the life of the process, plus a lazily created
+// session per (task, mode) so the same model answers both TTS and TTSStream.
+// This is the only unit that converts between the stdlib-only mirror types and
+// engine::runtime types.
+
+#include "capability_routing.h"
+#include "inference_lane.h"
+#include "model_options.h"
+
+#include "engine/framework/runtime/model.h"
+#include "engine/framework/runtime/registry.h"
+#include "engine/framework/runtime/session.h"
+
+#include <functional>
+#include <map>
+#include <memory>
+#include <stdexcept>
+#include <string>
+#include <utility>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// User-fixable configuration problem. grpc-server maps this to INVALID_ARGUMENT.
+class ConfigError : public std::runtime_error {
+public:
+    using std::runtime_error::runtime_error;
+};
+
+// The family cannot serve the requested RPC. Maps to UNIMPLEMENTED.
+class CapabilityError : public std::runtime_error {
+public:
+    using std::runtime_error::runtime_error;
+};
+
+engine::runtime::VoiceTaskKind to_engine_task(Task task);
+engine::runtime::RunMode to_engine_mode(Mode mode);
+Task from_engine_task(engine::runtime::VoiceTaskKind kind);
+Mode from_engine_mode(engine::runtime::RunMode mode);
+Capabilities to_capabilities(const std::string &family,
+                             const engine::runtime::CapabilitySet &set);
+
+// Reads audiocpp.model_spec.family from a GGUF. Returns an empty string when
+// the file is not a GGUF, carries no audio.cpp spec, or cannot be read. Never
+// throws: an unreadable file is the load gate's problem, not a crash.
+std::string read_gguf_family(const std::string &path);
+
+// Builds the absolute model path from LocalAI's (ModelPath, ModelFile, Model)
+// triple. Either the Model or the ModelFile field may carry the form
+// "bundled:<name>", which resolves to <executable dir>/assets/<name>, where
+// package.sh puts upstream's bundled silero_vad and marblenet_vad assets. BOTH
+// are checked because LocalAI fills ModelFile by joining ModelPath onto the
+// configured model string, so a model YAML using the form has it intact only in
+// Model.
+std::string resolve_model_path(const std::string &model_path_dir,
+                               const std::string &model_file,
+                               const std::string &model_name);
+
+class LoadedModel {
+public:
+    struct Session {
+        Task task = Task::Tts;
+        Mode mode = Mode::Offline;
+        // Exactly one of these is non-null, matching the resolved mode.
+        engine::runtime::IOfflineVoiceTaskSession *offline = nullptr;
+        engine::runtime::IStreamingVoiceTaskSession *streaming = nullptr;
+    };
+
+    // Throws ConfigError when the path does not exist, the family cannot be
+    // determined, or the registry rejects the family.
+    //
+    // `model_identity` is ModelOptions.Model verbatim: the UNTRANSLATED
+    // controller-side name. It is a constructor argument rather than a setter
+    // so identity and model are inseparable. llama-cpp keeps its equivalent in
+    // a separate global from the model, which leaves a window where a handler
+    // can read one without the other; here a handler that holds the model
+    // through snapshot() necessarily holds the identity it was loaded with.
+    LoadedModel(const std::string &resolved_path, const ModelOptions &options,
+                std::string model_identity);
+
+    LoadedModel(const LoadedModel &) = delete;
+    LoadedModel &operator=(const LoadedModel &) = delete;
+
+    const std::string &family() const noexcept { return capabilities_.family; }
+    // Empty when the controller predates ModelOptions.ModelIdentity, which the
+    // identity check reads as "skip". See check_model_identity in grpc-server.
+    const std::string &identity() const noexcept { return identity_; }
+    const std::string &variant() const noexcept { return variant_; }
+    const std::string &description() const noexcept { return description_; }
+    const std::vector<std::string> &languages() const noexcept { return languages_; }
+    const Capabilities &capabilities() const noexcept { return capabilities_; }
+    bool supports_timestamps() const noexcept { return supports_timestamps_; }
+    const engine::runtime::SessionOptions &session_options() const noexcept {
+        return session_options_;
+    }
+
+    // The model's `task:` option, empty when unset. Every handler must copy it
+    // into RequestShape::pinned_task before calling session_for: routing is
+    // otherwise derived from the RPC alone, and this is the option's only route
+    // from the load to the request that honours it.
+    const std::string &pinned_task() const noexcept { return pinned_task_; }
+
+    // The `live_idle_timeout_ms` option: how long AudioTranscriptionLive waits
+    // for the next audio frame before cancelling the stream to give this
+    // model's lane back. 0 means no limit. See the option in model_options.h
+    // for why it exists and how the default was chosen.
+    int live_idle_timeout_ms() const noexcept { return live_idle_timeout_ms_; }
+
+    // Throws the same CapabilityError session_for would throw when this family
+    // cannot serve the RPC, and RETURNS THE RESOLVED ROUTE otherwise.
+    //
+    // The route is returned rather than computed and dropped because a handler
+    // often has to know which task it is about to run BEFORE running it.
+    // AudioTransform refuses params[stem] on any route but source separation,
+    // and reading that off the route costs microseconds where reading it off
+    // the result costs a whole inference first. A caller with no such need
+    // ignores the value, which is what the three transcription-shaped handlers
+    // do.
+    //
+    // It exists so a refusal does not have to buy a place in the queue first.
+    // resolve_route is a pure function of capabilities_, which is fixed at
+    // construction and never written again, so unlike the session cache it
+    // needs no lane and no lock: a model that cannot transcribe can say so
+    // while another request is halfway through a thirty second run. Without
+    // this the refusal waits for that run to finish only to be told no.
+    //
+    // It does NOT replace the routing inside session_for, and must not be made
+    // to: session_for still needs the route to key the session cache. The two
+    // calls agree because both read the same immutable capabilities. What this
+    // one adds is only the ordering, so call it before acquire().
+    //
+    // Const and lane-free on purpose. If a future edit makes routing depend on
+    // mutable state, this must grow the lane parameter its siblings carry.
+    Route check_can_serve(Rpc rpc, const RequestShape &shape) const;
+
+    // Routes the RPC and returns the cached session, creating it on first use.
+    // Throws CapabilityError when this family cannot serve the RPC, and a plain
+    // runtime_error when it can but the session could not be built, which is an
+    // environment fault rather than a capability answer.
+    //
+    // The `lane` parameter is a PROOF OF HOLDING and is otherwise unused: it
+    // exists so the rule below is a compile error rather than prose. The
+    // session cache is an unsynchronised std::map and the sessions themselves
+    // are not reentrant, so this must only be called with the lane held; the
+    // lane admits one caller at a time, which is exactly the constraint the
+    // sessions impose. Pass the LaneEntry from acquire().
+    //
+    // NON-CONST reference on purpose, and do not "tidy" it to const. A const
+    // reference binds to a temporary, which makes this compile:
+    //
+    //     auto session = model->session_for(rpc, shape, model->acquire(0));
+    //     auto result  = run_offline(session, task, model->acquire(0));
+    //
+    // and each temporary dies at the end of its own full-expression, so the
+    // lane is released between the two calls. That is precisely the split this
+    // parameter exists to prevent, and it is the form a future caller is most
+    // likely to reach for because it reads as tidy. Requiring an lvalue forces
+    // a named entry whose scope spans both calls.
+    //
+    // What it proves is bounded, so do not over-trust it: it proves A lane was
+    // taken, not THIS model's lane. A caller determined to defeat it can
+    // construct an entry on an unrelated InferenceLane and pass that. It
+    // therefore catches the two mistakes that actually happen, forgetting the
+    // lane entirely and taking it after routing, and does not catch lane
+    // identity.
+    //
+    // STATE CONTRACT, and it is the CALLER'S to honour. Sessions are cached per
+    // (task, mode), so a streaming session is normally the same warm object the
+    // previous stream used, carrying that stream's state. session_for hands it
+    // back as it is.
+    //
+    // Every streaming caller must therefore begin a stream through
+    // begin_stream() below, which is prepare() then start_stream() in that
+    // order and is the ONLY implementation of that sequence. start_stream's
+    // base implementation is a call to reset(), which is what clears the
+    // previous stream, and reset() is only legal after prepare(): silero_vad
+    // throws "session prepare() must be called before Silero VAD reset()"
+    // otherwise. That ordering constraint is also why session_for cannot do
+    // this for you. Skipping it does not raise an error, it silently continues
+    // the previous stream.
+    //
+    // Offline sessions need no such care: their interface has no reset and
+    // run() takes a whole request.
+    Session session_for(Rpc rpc, const RequestShape &shape, LaneEntry &lane);
+
+    // Takes the inference lane, or throws LaneUnavailable. Serializes runs
+    // against this model. `requested_timeout_ms` is a per-request wait hint
+    // where a value <= 0 means "use the model's configured ceiling"; a hint may
+    // only tighten that ceiling, never loosen it.
+    //
+    // LaneEntry is deliberately immovable, so bind the result to a named local
+    // in the scope the inference happens in:
+    //
+    //     LaneEntry entry = model.acquire(request_hint_ms);
+    //
+    // which C++17 initializes in place. A handler that has to keep the lane
+    // beyond one scope, for instance in a member that outlives the call that
+    // took it, wants acquire_owned instead.
+    LaneEntry acquire(int requested_timeout_ms);
+
+    // Same lane, heap-allocated so it can be stored or handed on. Prefer
+    // acquire: this one adds a null state that the scoped form does not have.
+    std::unique_ptr<LaneEntry> acquire_owned(int requested_timeout_ms);
+
+private:
+    // Keyed by the enum values so the map needs no custom comparator.
+    using SessionKey = std::pair<int, int>;
+
+    // The public constructor runs the load gate, then delegates here. The
+    // detour exists because lane_ has to be built from the family in the member
+    // initializer list, and the family is only known after the gate has run.
+    // Four parameters rather than three so it cannot be confused with the
+    // public constructor, whose third argument is also a std::string.
+    LoadedModel(const std::string &resolved_path, const ModelOptions &options,
+                std::string family, std::string model_identity);
+
+    // MEMBER ORDER IS LOAD-BEARING BELOW THIS LINE. Members are destroyed in
+    // reverse declaration order.
+    //
+    // lane_ is first so it is destroyed last: nothing that runs during teardown
+    // can then find a lane that has already gone.
+    InferenceLane lane_;
+    // registry_ before model_: the registry owns the loader that produced the
+    // model, and the model may hold loader-owned state.
+    engine::runtime::ModelRegistry registry_;
+    // model_ before sessions_, so sessions_ is destroyed FIRST and the model
+    // second. A session is created from the model and must not outlive it. Do
+    // not reorder these two.
+    std::unique_ptr<engine::runtime::ILoadedVoiceModel> model_;
+    std::map<SessionKey, std::unique_ptr<engine::runtime::IVoiceTaskSession>> sessions_;
+
+    engine::runtime::SessionOptions session_options_;
+    Capabilities capabilities_;
+    std::string variant_;
+    std::string description_;
+    std::vector<std::string> languages_;
+    std::string pinned_task_;
+    std::string identity_;
+    bool supports_timestamps_ = false;
+    int wait_budget_ceiling_ms_ = 0;
+    int live_idle_timeout_ms_ = 0;
+};
+
+// Prepares and runs an offline session. prepare() is called for every run
+// rather than once per session, because SessionPreparationRequest is derived
+// from the request itself (audio contract, text, voice condition) and not from
+// the model: a second request with a different sample rate or length would
+// otherwise run against the first request's contract.
+//
+// `lane` is a PROOF OF HOLDING, unused at runtime, for the same reason
+// session_for takes one: the session is not reentrant and prepare() mutates it,
+// so running without the lane is a data race. Making it a parameter turns that
+// into a compile error instead of a comment. Non-const for the same reason as
+// session_for's: a const reference would bind to `model.acquire(0)` written
+// inline, and that temporary dies at the end of this call, releasing the lane
+// before the caller's next one.
+//
+// Throws CapabilityError when the session is not an offline one. That should be
+// unreachable through session_for, which already refuses a non-offline session
+// for an offline route, and is checked anyway because the alternative is a null
+// dereference.
+engine::runtime::TaskResult run_offline(const LoadedModel::Session &session,
+                                        const engine::runtime::TaskRequest &request,
+                                        LaneEntry &lane);
+
+// THE ONE IMPLEMENTATION of the streaming state obligation described in
+// session_for's STATE CONTRACT: prepare(), then start_stream(), in that order.
+//
+// It is a function rather than a comment because the obligation is invisible
+// when it is broken. Streaming sessions are CACHED per (task, mode), so the
+// object a second stream gets is the warm one the first stream left behind,
+// still holding its audio, its tokens and its started flag. What clears it is
+// start_stream, whose base implementation IS a reset() and whose seven family
+// overrides (nemotron_asr, vibevoice_asr, higgs_audio_stt, voxtral_realtime,
+// supertonic, omnivoice, voxcpm2) every one call reset() as their first
+// statement, verified in the pinned checkout. Nothing in the type system pins
+// that. A future override that dropped the reset would break every call site
+// at once with no compile error and no exception, only a second transcript
+// that begins with the first one's audio, so the fewer call sites there are to
+// break, the better: this is the only one.
+//
+// prepare() must come first and cannot be folded into session_for, because
+// reset() is illegal before prepare() (silero_vad throws "session prepare()
+// must be called before Silero VAD reset()"), and because the preparation
+// request is derived from the REQUEST, not the model: build_preparation_request
+// reads the audio contract, the text and the voice condition off it, so a
+// second stream with a different sample rate or length would otherwise run
+// against the first stream's contract.
+//
+// `lane` is a PROOF OF HOLDING, unused at runtime, exactly as in run_offline.
+//
+// Throws CapabilityError when the session is not a streaming one.
+void begin_stream(const LoadedModel::Session &session,
+                  const engine::runtime::TaskRequest &request, LaneEntry &lane);
+
+// Drives a streaming session that takes NO incremental input, which is the TTS
+// shape (StreamingInputKind::None, StreamingOutputKind::PullEvents): begin the
+// stream, pull events until the session says there are no more, then finish.
+//
+// NO STREAM EVENT SINK IS INSTALLED HERE, and that is deliberate rather than an
+// omission. voxcpm2's start_stream runs the whole synthesis and pushes every
+// chunk to the sink, then its next_stream_event replays those same chunks out
+// of the stored result, so a sink on this path would put every chunk of audio
+// on the wire twice. supertonic and omnivoice ignore set_stream_event_sink
+// outright. The pull loop is therefore the single delivery channel.
+//
+// The returned TaskResult is the session's own merged whole for all three
+// families, NOT a tail the pull loop missed. A caller that already emitted the
+// pulled events must not also emit its audio; see the TTSStream handler.
+engine::runtime::TaskResult run_streaming_pull(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+// Drives a streaming session that CONSUMES audio chunks, which is the ASR shape
+// (StreamingInputKind::AudioChunks): begin the stream, feed the buffer in
+// policy-sized chunks, then finalize.
+//
+// A STREAM EVENT SINK IS INSTALLED HERE, and it is not optional: nemotron_asr
+// reports its partial text ONLY through the sink, and only from inside
+// finalize(), because its decode does not start until the audio is complete.
+// Without the sink that family streams a transcript with no partials at all.
+// The sink is cleared again before returning, including on the exception path:
+// the session is cached and outlives this call, so a sink left holding a
+// reference to the caller's frame is a use after free waiting for the next
+// stream.
+//
+// Both delivery channels are consumed, the sink and the value process_audio_chunk
+// returns, because the families do not agree on which they use, and
+// voxtral_realtime uses BOTH for the same event. The duplicate that produces is
+// absorbed by TranscriptDeltaTracker in stream_delta.h rather than here.
+engine::runtime::TaskResult run_streaming_audio(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const engine::runtime::AudioBuffer &audio,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+// Drives the same ASR shape as run_streaming_audio when the audio DOES NOT
+// EXIST YET, which is the live-microphone case: instead of slicing a buffer it
+// pulls frames from the caller until the input side closes.
+//
+// `next_frames` fills `out` with interleaved float PCM and returns true, or
+// returns false when there is no more input. It is expected to BLOCK, since the
+// only real implementation is a gRPC stream Read, and it may throw: a request
+// the handler has to refuse mid-stream unwinds through here, and the sink is
+// cleared on that path like every other.
+//
+// The audio contract comes from `request.audio_input`, which for a live stream
+// is an EMPTY buffer carrying only the sample rate and channel count. It is not
+// optional: nemotron_asr's streaming prepare() throws "Nemotron ASR streaming
+// prepare() requires an audio contract" without one, and there is no buffer to
+// derive it from here.
+//
+// Frames are BUFFERED to the family's own preferred window rather than fed in
+// whatever sizes the wire delivered them in, because that window is a family's
+// statement about what it can decode (nemotron_asr asks for one second, higgs
+// for four), and a 512-sample gRPC frame is a property of the client's audio
+// callback rather than of the model. The tail shorter than a window is fed at
+// the end.
+//
+// A stream that carried NO AUDIO returns an empty TaskResult and never calls
+// finish_stream. Finalizing an empty stream is not universally legal:
+// nemotron_asr throws "Nemotron ASR finalize requires streamed audio", so a
+// client that opens a session and closes it without speaking would receive an
+// INTERNAL naming an engine internal instead of an empty transcript, which is
+// the truthful answer to "transcribe nothing".
+engine::runtime::TaskResult run_streaming_live(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<bool(std::vector<float> &)> &next_frames,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options.cpp

@@ -0,0 +1,153 @@
+#include "model_options.h"
+
+#include <cctype>
+#include <cerrno>
+#include <climits>
+#include <cstdlib>
+
+namespace audiocpp_backend {
+namespace {
+
+std::string trim(const std::string &value) {
+    size_t begin = 0;
+    while (begin < value.size() &&
+           std::isspace(static_cast<unsigned char>(value[begin])) != 0) {
+        ++begin;
+    }
+    size_t end = value.size();
+    while (end > begin &&
+           std::isspace(static_cast<unsigned char>(value[end - 1])) != 0) {
+        --end;
+    }
+    return value.substr(begin, end - begin);
+}
+
+// Parses a non-negative integer. Returns false on anything else, including
+// empty strings, signs, trailing garbage, and values too large for int.
+//
+// strtol rather than atoi: atoi is undefined behaviour once the digits exceed
+// long, and in practice it hands back a wrapped value. That would let
+// "device:2147483648" through as -2147483648 and send a negative index to the
+// ggml backend selector, from a function whose error text promises the caller a
+// non-negative integer.
+bool parse_non_negative_int(const std::string &value, int &out) {
+    if (value.empty()) {
+        return false;
+    }
+    for (const char ch : value) {
+        if (std::isdigit(static_cast<unsigned char>(ch)) == 0) {
+            return false;
+        }
+    }
+
+    errno = 0;
+    char *end = nullptr;
+    const long parsed = std::strtol(value.c_str(), &end, 10);
+    if (errno == ERANGE || end == nullptr || *end != '\0') {
+        return false;
+    }
+    if (parsed < 0 || parsed > INT_MAX) {
+        return false;
+    }
+
+    out = static_cast<int>(parsed);
+    return true;
+}
+
+bool starts_with(const std::string &value, const std::string &prefix) {
+    return value.size() >= prefix.size() &&
+           value.compare(0, prefix.size(), prefix) == 0;
+}
+
+} // namespace
+
+ParsedOptions parse_model_options(const std::vector<std::string> &entries) {
+    ParsedOptions parsed;
+
+    for (const auto &raw : entries) {
+        const std::string entry = trim(raw);
+        if (entry.empty()) {
+            continue;
+        }
+
+        // Split on the FIRST colon: values are often paths that contain more.
+        const size_t sep = entry.find(':');
+        if (sep == std::string::npos) {
+            parsed.error = "audio-cpp: option '" + entry +
+                           "' is not in key:value form";
+            return parsed;
+        }
+
+        const std::string key = trim(entry.substr(0, sep));
+        const std::string value = trim(entry.substr(sep + 1));
+
+        if (starts_with(key, "load.")) {
+            const std::string inner = key.substr(5);
+            if (inner.empty()) {
+                parsed.error = "audio-cpp: option '" + entry +
+                               "' has an empty load option name";
+                return parsed;
+            }
+            parsed.options.load_options[inner] = value;
+            continue;
+        }
+        if (starts_with(key, "session.")) {
+            const std::string inner = key.substr(8);
+            if (inner.empty()) {
+                parsed.error = "audio-cpp: option '" + entry +
+                               "' has an empty session option name";
+                return parsed;
+            }
+            parsed.options.session_options[inner] = value;
+            continue;
+        }
+
+        if (key == "family") {
+            parsed.options.family = value;
+        } else if (key == "task") {
+            parsed.options.task = value;
+        } else if (key == "backend") {
+            parsed.options.backend = value;
+        } else if (key == "model_spec_override") {
+            parsed.options.model_spec_override = value;
+        } else if (key == "device") {
+            if (!parse_non_negative_int(value, parsed.options.device)) {
+                parsed.error = "audio-cpp: option 'device' needs a non-negative "
+                               "integer, got '" + value + "'";
+                return parsed;
+            }
+            parsed.options.device_set = true;
+        } else if (key == "threads") {
+            if (!parse_non_negative_int(value, parsed.options.threads)) {
+                parsed.error = "audio-cpp: option 'threads' needs a non-negative "
+                               "integer, got '" + value + "'";
+                return parsed;
+            }
+        } else if (key == "busy_timeout_ms") {
+            if (!parse_non_negative_int(value, parsed.options.busy_timeout_ms)) {
+                parsed.error = "audio-cpp: option 'busy_timeout_ms' needs a "
+                               "non-negative integer, got '" + value + "'";
+                return parsed;
+            }
+        } else if (key == "live_idle_timeout_ms") {
+            if (!parse_non_negative_int(value,
+                                        parsed.options.live_idle_timeout_ms)) {
+                parsed.error = "audio-cpp: option 'live_idle_timeout_ms' needs a "
+                               "non-negative integer, got '" + value + "'";
+                return parsed;
+            }
+        } else {
+            // Quotes the whole entry, not just the key: an entry like ":value"
+            // has an empty key and would otherwise leave nothing to grep for.
+            parsed.error = "audio-cpp: unknown option key '" + entry +
+                           "'. Known keys: family, task, backend, device, "
+                           "threads, model_spec_override, busy_timeout_ms, "
+                           "live_idle_timeout_ms, load.<key>, session.<key>";
+            return parsed;
+        }
+    }
+
+    return parsed;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options.h

@@ -0,0 +1,66 @@
+#pragma once
+
+// Parses the model YAML's `options:` list (ModelOptions.Options in
+// backend.proto) into a struct. Standard library only: this unit is compiled
+// and tested by backend/cpp/run-unit-tests.sh without an audio.cpp checkout.
+
+#include <map>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+struct ModelOptions {
+    // audio.cpp model family. Empty means "derive from the GGUF's embedded
+    // audiocpp.model_spec.family key"; a non-GGUF path with an empty family is
+    // rejected at load time, not here.
+    std::string family;
+    // Pins the audio.cpp task, overriding RPC-based routing. Empty means route.
+    std::string task;
+    // ggml backend: cpu, cuda, vulkan, metal, best.
+    std::string backend = "cpu";
+    int device = 0;
+    // True once a `device:` entry has been seen. 0 is both the default and a
+    // legitimate device index, so the value alone cannot tell an explicit
+    // `device:0` from an unset option, and a caller merging in its own fallback
+    // would silently override the explicit choice.
+    bool device_set = false;
+    // 0 means "let the runtime decide".
+    int threads = 0;
+    std::string model_spec_override;
+    // 0 disables the run guard's fail-fast, restoring an unbounded wait.
+    int busy_timeout_ms = 0;
+    // How long AudioTranscriptionLive waits for the next audio frame before it
+    // cancels the stream and gives the model's lane back. 0 means NO LIMIT.
+    //
+    // It exists because that RPC holds the lane for the whole stream, so a peer
+    // that stops sending WITHOUT closing blocks every other request against this
+    // model for as long as its socket stays up. No other RPC can do that: they
+    // hold the lane across compute, which ends on its own.
+    //
+    // 30 seconds, and the number is picked from what the only in-tree client
+    // does. core/http/endpoints/openai/realtime.go drives a 300 ms ticker and
+    // feeds every tick that produced new audio while a turn is open, so 30 s of
+    // silence is a hundred ticks that delivered nothing: the peer is gone, or
+    // its socket is wedged. It is also comfortably longer than any pause a
+    // speaker takes mid-utterance, which is the case that must never be cut off,
+    // and backend.proto allows one stream to span many utterances, so a client
+    // that pauses for longer than this between them should raise it rather than
+    // discover it. Lowering it below a few seconds risks cancelling a live
+    // speaker; 0 turns the limit off for a client that legitimately idles.
+    int live_idle_timeout_ms = 30000;
+    // `load.<key>:<value>` entries, prefix stripped.
+    std::map<std::string, std::string> load_options;
+    // `session.<key>:<value>` entries, prefix stripped.
+    std::map<std::string, std::string> session_options;
+};
+
+struct ParsedOptions {
+    ModelOptions options;
+    // Non-empty means the caller must fail the load with INVALID_ARGUMENT.
+    std::string error;
+};
+
+ParsedOptions parse_model_options(const std::vector<std::string> &entries);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options_test.cpp

@@ -0,0 +1,165 @@
+// Unit tests for model_options. Standard library only, so
+// backend/cpp/run-unit-tests.sh picks this up with no engine checkout.
+//
+// The harness compiles this file as a single translation unit with no other
+// sources, so the implementation is included directly rather than linked.
+//
+// Build and run standalone:
+//   g++ -std=c++17 -I. model_options_test.cpp -o t && ./t
+
+#include "model_options.cpp"
+
+#include <cstdio>
+#include <map>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+// Returns the mapped value, or an empty string when the key is absent. map::at
+// would throw on a miss and abort the whole binary, so one prefix off-by-one
+// would hide every check that follows it instead of failing a single one.
+static std::string lookup(const std::map<std::string, std::string> &values,
+                          const std::string &key) {
+    const auto found = values.find(key);
+    return found == values.end() ? std::string() : found->second;
+}
+
+using audiocpp_backend::parse_model_options;
+
+static void test_defaults() {
+    auto r = parse_model_options({});
+    check(r.error.empty(), "empty option list is not an error");
+    check(r.options.family.empty(), "family defaults to empty");
+    check(r.options.task.empty(), "task defaults to empty");
+    check(r.options.backend == "cpu", "backend defaults to cpu");
+    check(r.options.device == 0, "device defaults to 0");
+    check(r.options.threads == 0, "threads defaults to 0");
+    check(r.options.busy_timeout_ms == 0, "busy_timeout_ms defaults to 0");
+    // NOT zero, unlike every other numeric option here. A live stream holds the
+    // model's lane while it waits on the client, so the default has to bound
+    // that wait; 0 is the explicit "no limit" the operator opts into.
+    check(r.options.live_idle_timeout_ms == 30000,
+          "live_idle_timeout_ms defaults to 30000");
+    check(r.options.load_options.empty(), "load_options defaults empty");
+    check(r.options.session_options.empty(), "session_options defaults empty");
+}
+
+static void test_scalar_options() {
+    auto r = parse_model_options({
+        "family:qwen3_tts",
+        "task:tts",
+        "backend:cuda",
+        "device:1",
+        "threads:8",
+        "busy_timeout_ms:30000",
+        "live_idle_timeout_ms:5000",
+    });
+    check(r.error.empty(), "scalar options parse without error");
+    check(r.options.family == "qwen3_tts", "family parsed");
+    check(r.options.task == "tts", "task parsed");
+    check(r.options.backend == "cuda", "backend parsed");
+    check(r.options.device == 1, "device parsed");
+    check(r.options.threads == 8, "threads parsed");
+    check(r.options.busy_timeout_ms == 30000, "busy_timeout_ms parsed");
+    check(r.options.live_idle_timeout_ms == 5000, "live_idle_timeout_ms parsed");
+    check(parse_model_options({"live_idle_timeout_ms:0"}).options.live_idle_timeout_ms == 0,
+          "an explicit 0 turns the live idle limit off rather than reverting to "
+          "the default");
+}
+
+// Values containing colons must survive: split on the FIRST colon only.
+static void test_value_containing_colon() {
+    auto r = parse_model_options({"model_spec_override:/models/a:b/spec.json"});
+    check(r.error.empty(), "colon-bearing value is not an error");
+    check(r.options.model_spec_override == "/models/a:b/spec.json",
+          "value keeps every colon after the first separator");
+}
+
+static void test_namespaced_options() {
+    auto r = parse_model_options({
+        "load.weight_type:q8_0",
+        "session.miocodec.weight_type:f16",
+        "session.graph_capacity:tiered",
+    });
+    check(r.error.empty(), "namespaced options parse without error");
+    check(r.options.load_options.size() == 1, "one load option");
+    check(lookup(r.options.load_options, "weight_type") == "q8_0", "load prefix stripped");
+    check(r.options.session_options.size() == 2, "two session options");
+    check(lookup(r.options.session_options, "miocodec.weight_type") == "f16",
+          "session prefix stripped, inner dots kept");
+    check(lookup(r.options.session_options, "graph_capacity") == "tiered",
+          "second session option parsed");
+}
+
+static void test_errors() {
+    check(!parse_model_options({"family"}).error.empty(),
+          "entry without a colon is rejected");
+    check(!parse_model_options({"nonsense:1"}).error.empty(),
+          "unknown key is rejected");
+    check(!parse_model_options({"device:abc"}).error.empty(),
+          "non-numeric device is rejected");
+    check(!parse_model_options({"threads:-1"}).error.empty(),
+          "negative threads is rejected");
+    check(!parse_model_options({"load.:x"}).error.empty(),
+          "empty load key is rejected");
+    check(!parse_model_options({"session.:x"}).error.empty(),
+          "empty session key is rejected");
+    check(!parse_model_options({"busy_timeout_ms:abc"}).error.empty(),
+          "non-numeric busy_timeout_ms is rejected");
+    check(!parse_model_options({"live_idle_timeout_ms:abc"}).error.empty(),
+          "non-numeric live_idle_timeout_ms is rejected");
+    check(!parse_model_options({"live_idle_timeout_ms:-1"}).error.empty(),
+          "negative live_idle_timeout_ms is rejected");
+    check(!parse_model_options({"device:-1"}).error.empty(),
+          "negative device is rejected");
+    check(!parse_model_options({"threads:x"}).error.empty(),
+          "non-numeric threads is rejected");
+
+    // Values too large for int must be rejected, not silently wrapped into a
+    // negative device index that then reaches the ggml backend selector.
+    check(!parse_model_options({"device:2147483648"}).error.empty(),
+          "device above INT_MAX is rejected");
+    check(!parse_model_options({"threads:99999999999999"}).error.empty(),
+          "threads above INT_MAX is rejected");
+
+    // The error text must name the offending entry so a user can fix their YAML.
+    const auto r = parse_model_options({"nonsense:1"});
+    check(r.error.find("nonsense") != std::string::npos,
+          "error names the offending key");
+
+    // An empty key still has to give the user something to grep for.
+    const auto empty_key = parse_model_options({":value"});
+    check(empty_key.error.find(":value") != std::string::npos,
+          "unknown-key error names the entry even when the key is empty");
+}
+
+static void test_blank_entries_ignored() {
+    auto r = parse_model_options({"", "   ", "family:supertonic"});
+    check(r.error.empty(), "blank entries are skipped, not rejected");
+    check(r.options.family == "supertonic", "real entry still parsed");
+}
+
+int main() {
+    test_defaults();
+    test_scalar_options();
+    test_value_containing_colon();
+    test_namespaced_options();
+    test_errors();
+    test_blank_entries_ignored();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all model_options checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/package.sh

@@ -0,0 +1,228 @@
+#!/bin/bash
+# Assemble backend/cpp/audio-cpp/package, which becomes the whole content of the
+# FROM scratch backend image. Nothing outside this directory exists at run time.
+set -euo pipefail
+
+CURDIR=$(dirname "$(realpath "$0")")
+REPO_ROOT="${CURDIR}/../../.."
+PACKAGE_DIR="$CURDIR/package"
+BUILD_DIR="$CURDIR/build"
+
+rm -rf "$PACKAGE_DIR"
+mkdir -p "$PACKAGE_DIR/lib" "$PACKAGE_DIR/assets"
+
+cp -avf "$CURDIR/grpc-server" "$PACKAGE_DIR/"
+cp -fv  "$CURDIR/run.sh"      "$PACKAGE_DIR/"
+
+# ENGINE_ENABLE_CPU_ALL_VARIANTS builds the ggml backends as shared objects that
+# are dlopened at run time, so ldd cannot see them and the dependency walk below
+# would leave the image with no CPU backend at all. They also cannot go in lib/:
+# ggml DISCOVERS them by listing dirname(/proc/self/exe) and the current
+# directory, so being on a library path is not enough, they have to be in a
+# directory ggml scans. run.sh execs the bundled loader from the package root
+# exactly so that directory is this one. cmake writes them to build/bin, not
+# next to build/grpc-server, which is why this reads from bin/.
+#
+# -a keeps the libggml.so -> libggml.so.0 -> libggml.so.0.12.0 symlink chain,
+# so the SONAME the binary asks for still names a file here.
+for pattern in '*.so*' '*.dylib*'; do
+    if compgen -G "$BUILD_DIR/bin/$pattern" > /dev/null; then
+        # shellcheck disable=SC2086
+        cp -avf "$BUILD_DIR/bin/"$pattern "$PACKAGE_DIR/"
+    fi
+done
+
+# Upstream ships silero_vad and marblenet_vad as small runtime assets.
+# resolve_model_path() expands "bundled:<name>" to
+# dirname(/proc/self/exe)/assets/<name>, so copying them here is what makes VAD
+# work with nothing downloaded.
+for asset in silero_vad marblenet_vad; do
+    src="$CURDIR/audio.cpp/assets/framework/models/$asset"
+    if [ -d "$src" ]; then
+        cp -rfv "$src" "$PACKAGE_DIR/assets/"
+    else
+        echo "package.sh: bundled asset missing: $src" >&2
+        echo "package.sh: run 'make audio.cpp' before packaging" >&2
+        exit 1
+    fi
+done
+
+# Everything below this point is Linux-only: a bundled ELF loader, an ldd walk
+# and an ld.so --list validation. The macOS equivalent is the otool -L closure in
+# scripts/build/audio-cpp-darwin.sh, which picks up from the exit 0 below.
+#
+# WARNING FOR ANYONE REWORKING THAT SCRIPT. The obvious move is to copy
+# scripts/build/privacy-filter-darwin.sh, and that script assembles its own
+# package under build/darwin and never calls package.sh at all. Adapted as-is it
+# will silently omit assets/, and the bundled: model path form then resolves to
+# nothing, which takes the only zero-download verification path in this backend
+# with it. That is why audio-cpp-darwin.sh copies THIS directory instead of
+# rebuilding one. The Darwin package needs the same root-level layout as the
+# Linux one: grpc-server, run.sh, the ggml dylibs and assets/ in ONE directory,
+# with lib/ for the rest. run.sh's Darwin branch execs grpc-server directly, so
+# _NSGetExecutablePath already names the package root; nothing else is needed
+# beyond putting the files there.
+UNAME_S=$(uname -s)
+if [ "$UNAME_S" = "Darwin" ]; then
+    echo "package.sh: Darwin dylib bundling is deferred to scripts/build/audio-cpp-darwin.sh"
+    ls -lah "$PACKAGE_DIR/" "$PACKAGE_DIR/assets/"
+    exit 0
+fi
+
+# The loader goes in the package ROOT, not in lib/. run.sh explains why at
+# length; the short version is that exec'ing it makes dirname(/proc/self/exe)
+# the directory it sits in, and both the ggml backend scan and the bundled:
+# asset lookup need that to be the package root.
+if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
+    cp -arfLv /lib64/ld-linux-x86-64.so.2 "$PACKAGE_DIR/ld.so"
+elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
+    cp -arfLv /lib/ld-linux-aarch64.so.1 "$PACKAGE_DIR/ld.so"
+else
+    echo "package.sh: unknown architecture" >&2
+    exit 1
+fi
+
+# THE LAYOUT ASSERTION. Everything else in this script checks that the package
+# can LINK. This checks that it can RESOLVE, which is a different property and
+# the one with no other guard on it.
+#
+# The loader, assets/ and the dlopened ggml objects only agree while they share
+# one directory, because run.sh execs the loader and all three are reached
+# through dirname(/proc/self/exe). A tidy-up that moves the loader into lib/,
+# following the llama-cpp layout, produces a package that builds, ships, and
+# then fails at run time with "model path does not exist: <pkg>/lib/assets/..."
+# or "Failed to initialize CPU backend". Fail the build instead.
+#
+# This sits immediately after the loader copy rather than at the end of the
+# script on purpose: everything below dereferences $PACKAGE_DIR/ld.so, so a
+# misplaced loader would otherwise surface as "No such file or directory" from
+# the validation gate and never reach an assertion that could explain it.
+if [ ! -f "$PACKAGE_DIR/ld.so" ]; then
+    echo "package.sh: the bundled loader must be at the package root, not in lib/." >&2
+    echo "package.sh: run.sh execs it, so its directory is dirname(/proc/self/exe)," >&2
+    echo "package.sh: which is where resolve_model_path looks for assets/ and where" >&2
+    echo "package.sh: ggml looks for the CPU variants." >&2
+    exit 1
+fi
+if [ ! -d "$PACKAGE_DIR/assets" ]; then
+    echo "package.sh: assets/ must sit beside the loader at the package root." >&2
+    exit 1
+fi
+# Only assert the ggml half when this build produced CPU variants at all: a
+# cublas or vulkan build links ggml statically and ships none.
+if compgen -G "$BUILD_DIR/bin/libggml-cpu-*.so" > /dev/null && \
+   ! compgen -G "$PACKAGE_DIR/libggml-cpu-*.so" > /dev/null; then
+    echo "package.sh: the build produced libggml-cpu-*.so but none reached the" >&2
+    echo "package.sh: package root, so ggml's scan of dirname(/proc/self/exe)" >&2
+    echo "package.sh: will find no CPU backend." >&2
+    exit 1
+fi
+
+# Libraries the host GPU driver stack owns. package_gpu_libs deliberately ships
+# the CUDA/Vulkan runtime but not the driver, because the driver has to match
+# the kernel module on whatever host runs the image. Copying the build host's
+# copy in would pin it to the build host instead.
+#
+# One regex, used by both the copy loop and the validation gate below. They have
+# to agree: exempting a library from the copy but not from the gate makes the
+# gate reject the very absence the copy loop just created.
+DRIVER_LIB_RE='^(libcuda\.so|libnvidia-)'
+# awk applies string-escape processing to a -v assignment before compiling the
+# regex, so a lone backslash is eaten and awk warns about it. Double them here
+# rather than keeping a second hand-written copy of the pattern, which is the
+# drift this single-source-of-truth exists to prevent.
+DRIVER_LIB_RE_AWK=${DRIVER_LIB_RE//\\/\\\\}
+is_driver_lib() {
+    [[ "$(basename "$1")" =~ $DRIVER_LIB_RE ]]
+}
+
+# Bundle the full dependency closure. grpc-server links the distro gRPC,
+# protobuf and absl stack; copying only the C/C++ runtime leaves the scratch
+# image unable to start. The walk runs over the PACKAGED binary, not the one in
+# $CURDIR, because its RUNPATH is $ORIGIN: only from inside the package does
+# libggml.so.0 resolve to the copy shipped above rather than to nothing.
+# The dlopened ggml objects are walked too, since a dependency of theirs that
+# grpc-server does not itself link would otherwise be missed.
+{
+    ldd "$PACKAGE_DIR/grpc-server"
+    for so in "$PACKAGE_DIR"/*.so*; do
+        [ -f "$so" ] || continue
+        ldd "$so"
+    done
+} | awk '$2 == "=>" && $3 ~ /^\// { print $3 }' | sort -u | \
+while read -r so; do
+    # Skip what is already inside the package: the ggml objects resolve through
+    # $ORIGIN and re-copying them into lib/ would ship two copies of each.
+    case "$so" in "$PACKAGE_DIR"/*) continue ;; esac
+    if is_driver_lib "$so"; then
+        echo "package.sh: leaving driver-owned library to the host: $so"
+        continue
+    fi
+    cp -arfLv "$so" "$PACKAGE_DIR/lib/"
+done
+
+GPU_LIB_SCRIPT="${REPO_ROOT}/scripts/build/package-gpu-libs.sh"
+if [ -f "$GPU_LIB_SCRIPT" ]; then
+    echo "Packaging GPU libraries for BUILD_TYPE=${BUILD_TYPE:-cpu}..."
+    # shellcheck source=/dev/null
+    source "$GPU_LIB_SCRIPT" "$PACKAGE_DIR/lib"
+    package_gpu_libs
+fi
+
+# Resolve every dependency through the same loader and library path the
+# from-scratch image uses. Two distinct failures are rejected, because the
+# loader can still fall back to the host's default directories: a dependency it
+# could not resolve at all, and one it resolved to a file OUTSIDE the package,
+# which would validate here and be absent in the image.
+#
+# The driver libraries are exempt from BOTH rejections, and that exemption is
+# load-bearing on GPU builds rather than tidiness. With BUILD_TYPE=cublas ggml
+# is static (no CPU_ALL_VARIANTS), and ggml/CMakeLists.txt defaults
+# GGML_CUDA_NO_VMM=OFF, so ggml-cuda links CUDA::cuda_driver and grpc-server
+# itself carries DT_NEEDED libcuda.so.1. The copy loop above deliberately leaves
+# that to the host, so inside the CUDA builder it resolves either to a host path
+# or to nothing. Without this exemption every cublas build would fail here and
+# CI would produce no image at all.
+#
+# LD_TRACE_LOADED_OBJECTS + LD_LIBRARY_PATH, NOT `ld.so --library-path --list`,
+# and the difference is not cosmetic. Measured on a stub object built to
+# DT_NEEDED an absent libcuda.so.1: `--list` refuses to trace at all, printing
+# "libdrivertest.so: error while loading shared libraries: libcuda.so.1: cannot
+# open shared object file" and exiting 127, so no per-library line is ever
+# produced and no exemption below could apply. The env form prints
+# "libcuda.so.1 => not found" and exits 0, which is what makes both the
+# unresolved rule and its driver exemption reachable. It is also closer to what
+# run.sh actually does, since run.sh exports LD_LIBRARY_PATH rather than passing
+# --library-path.
+validation_failed=0
+validate_object() {
+    local object="$1"
+    LD_TRACE_LOADED_OBJECTS=1 LD_LIBRARY_PATH="$PACKAGE_DIR/lib:$PACKAGE_DIR" \
+        "$PACKAGE_DIR/ld.so" "$object" | awk -v pkg="$PACKAGE_DIR/" -v obj="$object" \
+                                             -v driver_re="$DRIVER_LIB_RE_AWK" '
+        function base(p,   n, parts) { n = split(p, parts, "/"); return parts[n] }
+        $2 == "=>" && $3 == "not" {
+            if ($1 ~ driver_re) next
+            print "package.sh: unresolved dependency of " obj ": " $1 > "/dev/stderr"
+            bad = 1
+        }
+        $2 == "=>" && $3 ~ /^\// && index($3, pkg) != 1 {
+            if (base($3) ~ driver_re) next
+            print "package.sh: dependency of " obj " resolved outside the package: " $0 > "/dev/stderr"
+            bad = 1
+        }
+        END { exit bad }
+    '
+}
+
+validate_object "$PACKAGE_DIR/grpc-server" || validation_failed=1
+for so in "$PACKAGE_DIR"/*.so*; do
+    [ -f "$so" ] || continue
+    validate_object "$so" || validation_failed=1
+done
+if [ "$validation_failed" -ne 0 ]; then
+    exit 1
+fi
+
+echo "audio-cpp package contents:"
+ls -lah "$PACKAGE_DIR/" "$PACKAGE_DIR/lib/" "$PACKAGE_DIR/assets/"

backend/cpp/audio-cpp/result_map.cpp

@@ -0,0 +1,85 @@
+#include "result_map.h"
+
+#include "transcript_assembly.h"
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+void fill_transcript_result(const engine::runtime::TaskResult &result,
+                            int sample_rate, float duration_seconds,
+                            backend::TranscriptResult *out) {
+    // No null guard on `out`, deliberately. gRPC always hands a handler a
+    // response message, so a null here would be a programming error in a
+    // caller, and a guard that returned quietly would answer the client with an
+    // untouched, empty transcript and an OK status. That is the same
+    // indistinguishable-from-silence failure the rest of this unit exists to
+    // prevent; crashing on the developer's machine is the cheaper outcome.
+    std::vector<Span> speech_segments;
+    speech_segments.reserve(result.speech_segments.size());
+    for (const auto &segment : result.speech_segments) {
+        speech_segments.push_back(
+            Span{segment.span.start_sample, segment.span.end_sample});
+    }
+
+    std::vector<SpeakerSpan> speaker_turns;
+    speaker_turns.reserve(result.speaker_turns.size());
+    for (const auto &turn : result.speaker_turns) {
+        speaker_turns.push_back(
+            SpeakerSpan{Span{turn.span.start_sample, turn.span.end_sample},
+                        turn.speaker_id});
+    }
+
+    std::vector<WordSpan> words;
+    words.reserve(result.word_timestamps.size());
+    for (const auto &word : result.word_timestamps) {
+        words.push_back(
+            WordSpan{Span{word.span.start_sample, word.span.end_sample},
+                     word.word});
+    }
+
+    // The ONLY read of transcript text in this function, and the only one there
+    // may ever be. See THE RULE in the header.
+    const std::string text =
+        result.text_output.has_value() ? result.text_output->text : std::string();
+
+    const AssembledTranscript assembled = assemble_transcript(
+        text, speech_segments, speaker_turns, words, sample_rate);
+
+    out->set_text(assembled.text);
+    // language has no source inside transcript_assembly, which is span-shaped
+    // only, so it is read straight off the engine result here. Left untouched
+    // when the family reported no text output at all: an empty string would be
+    // indistinguishable from a family that genuinely detected no language, and
+    // the field is documented as optional.
+    if (result.text_output.has_value()) {
+        out->set_language(result.text_output->language);
+    }
+    out->set_duration(duration_seconds);
+
+    // Cleared rather than appended to. A caller that fills the same message
+    // twice (a stream's final_result being rebuilt, say) would otherwise emit
+    // every segment twice, and the second call's ids would restart at 0 and
+    // collide with the first call's.
+    out->clear_segments();
+    for (const auto &segment : assembled.segments) {
+        auto *out_segment = out->add_segments();
+        out_segment->set_id(segment.id);
+        // NANOSECONDS. TranscriptSegment and TranscriptWord are the only
+        // messages in backend.proto that use them; VADSegment and DiarizeSegment
+        // are float seconds. assemble_transcript has already converted.
+        out_segment->set_start(segment.start_ns);
+        out_segment->set_end(segment.end_ns);
+        out_segment->set_text(segment.text);
+        out_segment->set_speaker(segment.speaker);
+        for (const auto &word : segment.words) {
+            auto *out_word = out_segment->add_words();
+            out_word->set_start(word.start_ns);
+            out_word->set_end(word.end_ns);
+            out_word->set_text(word.text);
+        }
+    }
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/result_map.h

@@ -0,0 +1,37 @@
+#pragma once
+
+// Converts engine::runtime results into LocalAI proto messages. All of the
+// non-trivial shaping lives in transcript_assembly, which is stdlib-only and
+// unit tested; this unit is the thin engine-typed boundary around it.
+
+#include "backend.pb.h"
+
+#include "engine/framework/runtime/session.h"
+
+namespace audiocpp_backend {
+
+// Fills text, language, duration, segments and per-segment words.
+//
+// THE RULE: the top-level text is TaskResult.text_output verbatim. It is never
+// derived from segments or words. audio.cpp carries transcript text in
+// text_output and nowhere else: speech_segments, speaker_turns and
+// word_timestamps carry spans and labels and no text at all. Deriving the
+// transcript from them therefore returns an EMPTY text for every producer that
+// reports segments without word timing, which real VibeVoice diarized ASR does.
+// An earlier attempt at this backend shipped exactly that bug. assemble_transcript
+// enforces the rule and is heavily tested; this unit's job is not to re-derive
+// it but to not undo it at the proto boundary.
+//
+// `sample_rate` is the rate the result's spans are expressed in, which is the
+// rate of the AudioBuffer that was handed to the session, NOT the rate of the
+// file the caller uploaded. Those differ whenever read_audio_file resampled,
+// which is why the handler passes the buffer's rate rather than the file's.
+//
+// Segments are replaced, not appended to, so a message filled twice does not
+// accumulate. `out` must be non-null and is not checked; see the note at the
+// top of the implementation for why that is not an oversight.
+void fill_transcript_result(const engine::runtime::TaskResult &result,
+                            int sample_rate, float duration_seconds,
+                            backend::TranscriptResult *out);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/result_map_ctest.cpp

@@ -0,0 +1,247 @@
+// Tests for result_map, the engine-to-proto boundary.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE. backend/cpp/run-unit-tests.sh globs
+// every *_test.cpp under backend/cpp/ and compiles it as a single standalone
+// translation unit with no include path beyond its own directory. This file
+// needs backend.pb.h and the audio.cpp framework headers, so it is built and
+// run by ctest instead:
+//
+//   make -C backend/cpp/audio-cpp test-engine
+//
+// Renaming it to *_test.cpp would break the standalone suite for every backend.
+//
+// What is worth testing here is exactly one thing, and it is not the field
+// copying: THE RULE. TaskResult carries transcript text in text_output and
+// nowhere else, so the proto's text must be that string verbatim. An earlier
+// attempt at this backend derived it from the segments, which returns an empty
+// transcript for every producer that reports segments without word timing.
+// transcript_assembly already enforces the rule and is tested on its own; these
+// checks are here so that a future edit cannot undo it at the boundary.
+
+#include "result_map.h"
+
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+namespace rt = engine::runtime;
+
+static const int kRate = 16000;
+
+static rt::SpeechSegment speech(std::int64_t start, std::int64_t end) {
+    rt::SpeechSegment segment;
+    segment.span.start_sample = start;
+    segment.span.end_sample = end;
+    return segment;
+}
+
+static rt::SpeakerTurn turn(std::int64_t start, std::int64_t end,
+                            const std::string &speaker) {
+    rt::SpeakerTurn out;
+    out.span.start_sample = start;
+    out.span.end_sample = end;
+    out.speaker_id = speaker;
+    return out;
+}
+
+static rt::WordTimestamp word(std::int64_t start, std::int64_t end,
+                              const std::string &text) {
+    rt::WordTimestamp out;
+    out.span.start_sample = start;
+    out.span.end_sample = end;
+    out.word = text;
+    return out;
+}
+
+// THE REGRESSION. A diarized ASR result: real text, real speaker turns, and no
+// word timing at all. This is the vibevoice_asr shape, and it is the one that
+// came back empty before.
+static void test_text_survives_segments_without_words() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "hello there general kenobi";
+    transcript.language = "en";
+    result.text_output = transcript;
+    result.speaker_turns.push_back(turn(0, 16000, "speaker_0"));
+    result.speaker_turns.push_back(turn(16000, 32000, "speaker_1"));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 2.0f, &out);
+
+    check(out.text() == "hello there general kenobi",
+          "diarized result keeps text_output verbatim");
+    check(out.language() == "en", "language comes from text_output");
+    check(out.segments_size() == 2, "both speaker turns become segments");
+    if (out.segments_size() == 2) {
+        check(out.segments(0).speaker() == "speaker_0",
+              "first segment keeps its own speaker label");
+        check(out.segments(1).speaker() == "speaker_1",
+              "second segment keeps its own speaker label");
+        check(out.segments(1).start() == 1000000000LL,
+              "segment start is nanoseconds, not samples");
+        check(out.segments(1).end() == 2000000000LL,
+              "segment end is nanoseconds, not samples");
+    }
+}
+
+// The same rule seen from the other side: text present, spans present, and the
+// per-segment text empty because there is nothing truthful to split. A boundary
+// that derived the top-level text from these segments would produce "".
+static void test_speech_segments_do_not_supply_the_text() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "one two three";
+    result.text_output = transcript;
+    result.speech_segments.push_back(speech(0, 8000));
+    result.speech_segments.push_back(speech(8000, 16000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text() == "one two three",
+          "speech segments without words do not empty the transcript");
+    check(out.segments_size() == 2, "both speech segments are emitted");
+    if (out.segments_size() == 2) {
+        check(out.segments(0).text().empty() && out.segments(1).text().empty(),
+              "per-segment text stays empty when there is no word timing");
+    }
+}
+
+static void test_words_reach_the_proto_in_nanoseconds() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "hi there";
+    result.text_output = transcript;
+    result.word_timestamps.push_back(word(0, 8000, "hi"));
+    result.word_timestamps.push_back(word(8000, 16000, "there"));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text() == "hi there", "word-timed result keeps text_output");
+    check(out.segments_size() == 1, "words with no spans yield one covering segment");
+    if (out.segments_size() == 1) {
+        const auto &segment = out.segments(0);
+        check(segment.words_size() == 2, "both words are emitted");
+        if (segment.words_size() == 2) {
+            check(segment.words(0).text() == "hi", "first word text");
+            check(segment.words(0).start() == 0, "first word start");
+            check(segment.words(0).end() == 500000000LL,
+                  "first word end is 0.5 s in nanoseconds");
+            check(segment.words(1).start() == 500000000LL, "second word start");
+            check(segment.words(1).end() == 1000000000LL, "second word end");
+        }
+    }
+}
+
+// The buffer's rate, not the file's, is what the spans mean. Passing 8000 for
+// the same spans has to halve every timestamp, which is what makes resampling
+// the input at read time load-bearing rather than cosmetic.
+static void test_sample_rate_scales_the_timestamps() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "x";
+    result.text_output = transcript;
+    result.speech_segments.push_back(speech(0, 8000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, 8000, 1.0f, &out);
+
+    check(out.segments_size() == 1, "one segment at 8 kHz");
+    if (out.segments_size() == 1) {
+        check(out.segments(0).end() == 1000000000LL,
+              "8000 samples at 8 kHz is one second");
+    }
+}
+
+static void test_duration_is_carried_through() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "x";
+    result.text_output = transcript;
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 14.07f, &out);
+
+    check(out.duration() > 14.06f && out.duration() < 14.08f,
+          "duration is set from the argument");
+}
+
+// No text output at all. A VAD-shaped result reaching this boundary must not
+// invent a transcript, and must not overwrite a language the caller had already
+// decided on.
+static void test_missing_text_output_leaves_language_alone() {
+    rt::TaskResult result;
+    result.speech_segments.push_back(speech(0, 16000));
+
+    backend::TranscriptResult out;
+    out.set_language("it");
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text().empty(), "no text_output means no text");
+    check(out.language() == "it",
+          "a result with no text_output does not clear the language");
+    check(out.segments_size() == 1, "spans are still emitted");
+}
+
+// Filling the same message twice must replace, not accumulate: the second
+// call's ids restart at 0 and would collide with the first call's.
+static void test_refilling_replaces_the_segments() {
+    rt::TaskResult first;
+    rt::Transcript transcript;
+    transcript.text = "first";
+    first.text_output = transcript;
+    first.speech_segments.push_back(speech(0, 16000));
+    first.speech_segments.push_back(speech(16000, 32000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(first, kRate, 2.0f, &out);
+
+    rt::TaskResult second;
+    rt::Transcript replacement;
+    replacement.text = "second";
+    second.text_output = replacement;
+    second.speech_segments.push_back(speech(0, 16000));
+    fill_transcript_result(second, kRate, 1.0f, &out);
+
+    check(out.text() == "second", "the second fill replaces the text");
+    check(out.segments_size() == 1,
+          "the second fill replaces the segments instead of appending");
+}
+
+static void test_empty_result_is_empty() {
+    rt::TaskResult result;
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 0.0f, &out);
+
+    check(out.text().empty(), "empty result has no text");
+    check(out.segments_size() == 0, "empty result has no segments");
+}
+
+int main() {
+    test_text_survives_segments_without_words();
+    test_speech_segments_do_not_supply_the_text();
+    test_words_reach_the_proto_in_nanoseconds();
+    test_sample_rate_scales_the_timestamps();
+    test_duration_is_carried_through();
+    test_missing_text_output_leaves_language_alone();
+    test_refilling_replaces_the_segments();
+    test_empty_result_is_empty();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all result_map checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/run.sh

@@ -0,0 +1,58 @@
+#!/bin/bash
+# Entry point for the audio-cpp backend image and for BACKEND_BINARY mode.
+#
+# The image's final stage is FROM scratch, so the package root is / and there is
+# no system loader, no system libc and no fallback library path. Everything the
+# process opens has to be inside the package, and it has to be findable by the
+# two mechanisms that actually do the finding: the dynamic linker, and
+# audio.cpp's own directory scans.
+set -e
+
+CURDIR=$(dirname "$(realpath "$0")")
+
+if [ "$(uname -s)" = "Darwin" ]; then
+	export DYLD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$DYLD_LIBRARY_PATH"
+	exec "$CURDIR/grpc-server" "$@"
+fi
+
+# $CURDIR is on the path as well as $CURDIR/lib: the ggml shared objects the
+# CPU-all-variants build produces sit in the package root, next to the binary,
+# not in lib/. See the comment below for why they cannot live in lib/.
+export LD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$LD_LIBRARY_PATH"
+
+# THE BUNDLED LOADER IS AT THE PACKAGE ROOT, NOT AT lib/ld.so. DO NOT MOVE IT.
+#
+# Exec'ing the loader is what pins the bundled glibc to the matching ld.so, and
+# every other C++ backend here does it. The cost is that /proc/self/exe then
+# names the LOADER rather than grpc-server, and this backend has two consumers
+# of /proc/self/exe that both have to land on the package root:
+#
+#   - ggml's backend registry DISCOVERS the per-microarch libggml-cpu-*.so by
+#     scanning dirname(/proc/self/exe) and the current directory. Those files
+#     are dlopened, never linked, so no library path and no RUNPATH reaches
+#     them: they have to be in a directory ggml scans.
+#   - resolve_model_path() turns "bundled:<name>" into
+#     dirname(/proc/self/exe)/assets/<name>, which is how the bundled
+#     silero_vad and marblenet_vad models resolve with nothing downloaded.
+#
+# backend/cpp/llama-cpp/package.sh answers the first of these by keeping
+# lib/ld.so and moving the ggml objects INTO lib/. That does not generalise
+# here, because it would also drag assets/ into lib/ to keep the second
+# consumer working. Putting the loader in the package root instead makes
+# dirname(/proc/self/exe) the package root, so the binary, the ggml objects and
+# assets/ all sit in the one directory that all three mechanisms agree on.
+#
+# The ggml half has a second chance that the bundled: half does not: LocalAI
+# sets the backend process cwd to the directory holding run.sh
+# (pkg/model/process.go), so ggml's fs::current_path() fallback would find the
+# objects in normal operation whatever the loader's placement. That fallback is
+# worth little here. It holds only for the launcher that sets that cwd, it is
+# gone the moment anyone runs the binary by hand or through a wrapper that
+# chdirs, and resolve_model_path has no equivalent, which would leave the only
+# zero-download path in this backend resting on it. Rooting the loader is the
+# one layout where all three mechanisms agree without depending on the cwd.
+if [ -f "$CURDIR/ld.so" ]; then
+	exec "$CURDIR/ld.so" "$CURDIR/grpc-server" "$@"
+fi
+
+exec "$CURDIR/grpc-server" "$@"

backend/cpp/audio-cpp/stem_selection.cpp

@@ -0,0 +1,129 @@
+#include "stem_selection.h"
+
+#include <cstddef>
+#include <filesystem>
+#include <string>
+
+namespace audiocpp_backend {
+namespace {
+
+// The stem every separation family this backend can reach names its lead vocal
+// track, and the one a caller who names no stem almost always wants: it is what
+// the OpenAI-shaped "isolate the voice" request means. htdemucs (drums, bass,
+// other, vocals) and mel_band_roformer (vocals, instrumental) both have it, and
+// in htdemucs's case it is NOT the first output, which is the whole reason this
+// preference is written down rather than left as "take index 0".
+const char *const kPreferredStem = "vocals";
+
+std::string join_names(const std::vector<std::string> &names) {
+    std::string out;
+    for (const auto &name : names) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += name;
+    }
+    return out;
+}
+
+// Whether a model-supplied stem name can be used as one component of a file
+// name. Deliberately a whitelist of refusals rather than a sanitiser: silently
+// rewriting "vo/cals" to "cals" would make the file the caller receives
+// disagree with the name they would have to ask for.
+bool name_is_writable(const std::string &name) {
+    if (name.empty() || name == "." || name == "..") {
+        return false;
+    }
+    // Control bytes, NUL above all. GGUF strings are length prefixed and demucs
+    // reads its source names out of JSON, which can encode one, so a
+    // std::string holding an embedded NUL survives all the way here. Two such
+    // names differing only AFTER the NUL are distinct std::strings, so the
+    // duplicate check below waves them through, and then path::c_str()
+    // truncates both at the NUL and they open the same file: precisely the
+    // silent overwrite the duplicate check exists to prevent, with the ".wav"
+    // stripped off as well. The rest of the range goes with it, since a newline
+    // or an escape sequence in a file name is a terminal and log injection
+    // nuisance with no legitimate use.
+    for (const char byte : name) {
+        const auto value = static_cast<unsigned char>(byte);
+        if (value < 0x20 || value == 0x7f) {
+            return false;
+        }
+    }
+    // Both separators, not just the host's. A GGUF is a downloaded file and its
+    // strings are not this host's to trust, so a name written on Windows must
+    // not become a directory traversal wherever the check happens to run.
+    return name.find('/') == std::string::npos &&
+           name.find('\\') == std::string::npos;
+}
+
+} // namespace
+
+StemChoice select_named_output(const std::vector<std::string> &names,
+                               const std::string &requested) {
+    StemChoice choice;
+    if (names.empty()) {
+        // Not an error here. The caller distinguishes "this family produces one
+        // unnamed output" from "this family produced nothing", and only it can
+        // tell them apart.
+        return choice;
+    }
+
+    // Every name is checked, not merely the selected one, because every stem is
+    // written. A bad name in the fourth output would otherwise be discovered
+    // only after three files had already been created.
+    for (std::size_t i = 0; i < names.size(); ++i) {
+        if (!name_is_writable(names[i])) {
+            choice.error = "audio-cpp: this model names an output stem '" +
+                           names[i] +
+                           "' that cannot be used as a file name; stems: " +
+                           join_names(names);
+            return choice;
+        }
+        for (std::size_t seen = 0; seen < i; ++seen) {
+            if (names[seen] == names[i]) {
+                choice.error =
+                    "audio-cpp: this model produces two output stems both named '" +
+                    names[i] + "'; one would silently overwrite the other";
+                return choice;
+            }
+        }
+    }
+
+    if (!requested.empty()) {
+        for (std::size_t i = 0; i < names.size(); ++i) {
+            if (names[i] == requested) {
+                choice.index = static_cast<int>(i);
+                return choice;
+            }
+        }
+        choice.error = "audio-cpp: no stem named '" + requested +
+                       "' in this model's output; available stems: " +
+                       join_names(names);
+        return choice;
+    }
+
+    for (std::size_t i = 0; i < names.size(); ++i) {
+        if (names[i] == kPreferredStem) {
+            choice.index = static_cast<int>(i);
+            return choice;
+        }
+    }
+    choice.index = 0;
+    return choice;
+}
+
+std::string sibling_stem_path(const std::string &dst, const std::string &name) {
+    const std::filesystem::path path(dst);
+    // The dst extension is reused rather than forced to ".wav" so the siblings
+    // look like the file the caller named. write_audio_file writes WAV bytes
+    // whatever the extension says, for dst as much as for the siblings, so this
+    // keeps the set consistent instead of making the siblings honest about a
+    // format dst is already lying about.
+    const std::string extension =
+        path.has_extension() ? path.extension().string() : std::string(".wav");
+    return (path.parent_path() / (path.stem().string() + "." + name + extension))
+        .string();
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stem_selection.h

@@ -0,0 +1,63 @@
+#pragma once
+
+// Decides which of a separation model's named stems the AudioTransform response
+// carries in `dst`, and names the sibling files every other stem is written to.
+//
+// It exists because AudioTransformResult carries ONE dst while htdemucs and
+// mel_band_roformer produce several named outputs from a single run. Running
+// once per stem would cost four full inferences for a four stem model, so the
+// handler runs once, writes every stem beside dst, and puts the selected one in
+// dst itself.
+//
+// Standard library only, so backend/cpp/run-unit-tests.sh compiles and runs its
+// test without an audio.cpp checkout. grpc-server.cpp flattens the engine's
+// NamedAudioBuffer list into the plain name vector taken here; nothing in this
+// unit knows about engine::runtime.
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+struct StemChoice {
+    // Index into the `names` vector. -1 means nothing was chosen, which happens
+    // for an empty list (the family produced a single unnamed output) and for
+    // every refusal.
+    //
+    // THE CONTRACT THE CALLER INDEXES ON: when `error` is empty and `names` was
+    // not, this is always a valid index into `names`. It is never -1 in that
+    // case, so a caller that checks `error` first can index without a further
+    // guard, and a caller that does not check `error` first would index with a
+    // negative value. Check the error.
+    int index = -1;
+    // Non-empty when the request must be refused, and suitable verbatim as an
+    // INVALID_ARGUMENT message. Two things land here: the caller named a stem
+    // this model does not produce, and the model named stems that cannot both
+    // be written (an unusable file name, or two stems sharing one).
+    std::string error;
+};
+
+// Picks the stem that goes to dst. Preference order: an explicit `requested`,
+// then "vocals", then the first output.
+//
+// An explicit but unknown `requested` is an ERROR rather than a fallback. A
+// caller who asks for "drums" and silently receives "vocals" gets a 200 and a
+// wrong file, which is the failure mode nobody can see; the message therefore
+// lists the stem names this model really has.
+//
+// The names are also validated, because they come from the MODEL (htdemucs
+// reads them from the GGUF's config.sources) and each one becomes a component
+// of a file path this backend writes. A name carrying a path separator would
+// write outside the caller's output directory, and two stems sharing a name
+// would silently overwrite each other. Both are refused before anything is
+// written, which is also why selection has to happen before the first write
+// rather than after the loop: a refused request must leave no files behind.
+StemChoice select_named_output(const std::vector<std::string> &names,
+                               const std::string &requested);
+
+// "/generated/transform-1.wav" + "drums" -> "/generated/transform-1.drums.wav".
+// A dst with no extension gets ".wav", since that is what write_audio_file
+// produces whatever the caller called the file.
+std::string sibling_stem_path(const std::string &dst, const std::string &name);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stem_selection_test.cpp

@@ -0,0 +1,250 @@
+// Unit tests for stem_selection. Standard library only. The harness
+// (backend/cpp/run-unit-tests.sh) compiles this as a single translation unit,
+// so the implementation is included directly.
+//
+// What is actually at stake here: AudioTransformResult carries one dst, a
+// separation model produces several stems, and the caller cannot see which one
+// they got. Every check below is about a wrong file arriving with a 200.
+
+#include "stem_selection.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_equal(const std::string &got, const std::string &want,
+                        const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\", want \"" + want + "\")");
+}
+
+using namespace audiocpp_backend;
+
+// htdemucs's real source order, taken from its GGUF config.sources. vocals is
+// LAST, which is why "first output" is not the default.
+static const std::vector<std::string> kDemucs = {"drums", "bass", "other",
+                                                 "vocals"};
+// mel_band_roformer's, where vocals is first.
+static const std::vector<std::string> kRoformer = {"vocals", "instrumental"};
+
+static void test_default_selection() {
+    const auto demucs = select_named_output(kDemucs, "");
+    check(demucs.error.empty(), "an unrequested selection is not an error");
+    check(demucs.index == 3, "no stem asked for picks vocals, not the first output");
+
+    const auto roformer = select_named_output(kRoformer, "");
+    check(roformer.index == 0, "vocals is picked when it is already first");
+
+    // No vocals anywhere: the first output is the documented fallback.
+    const auto novocals = select_named_output({"accompaniment", "drums"}, "");
+    check(novocals.index == 0 && novocals.error.empty(),
+          "a family with no vocals stem falls back to the first output");
+
+    // Substring matches must not count: "vocals_2" is a different stem.
+    const auto near = select_named_output({"vocals_2", "backing"}, "");
+    check(near.index == 0 && near.error.empty(),
+          "'vocals_2' is not 'vocals', so the fallback and not the preference applies");
+    const auto near_second = select_named_output({"backing", "vocals_2"}, "");
+    check(near_second.index == 0,
+          "a near miss on the preferred name does not pull it to the front");
+}
+
+static void test_explicit_selection() {
+    for (int i = 0; i < 4; ++i) {
+        const auto choice = select_named_output(kDemucs, kDemucs[static_cast<size_t>(i)]);
+        check(choice.index == i && choice.error.empty(),
+              "explicit '" + kDemucs[static_cast<size_t>(i)] + "' selects its own index");
+    }
+    // Including the one the default would have chosen anyway: asking for it
+    // must not be treated as "no request".
+    const auto vocals = select_named_output(kDemucs, "vocals");
+    check(vocals.index == 3, "explicitly asking for vocals still selects vocals");
+}
+
+static void test_unknown_stem_is_refused() {
+    const auto choice = select_named_output(kDemucs, "kazoo");
+    check(choice.index == -1, "an unknown stem selects nothing");
+    check(!choice.error.empty(), "an unknown stem is refused rather than substituted");
+    check(choice.error.find("kazoo") != std::string::npos,
+          "the refusal names the stem that was asked for");
+    // The real names, so the caller can fix the request without guessing.
+    for (const auto &name : kDemucs) {
+        check(choice.error.find(name) != std::string::npos,
+              "the refusal lists the real stem '" + name + "'");
+    }
+    check(choice.error.find("drums, bass, other, vocals") != std::string::npos,
+          "the refusal lists the stems in the model's own order");
+
+    // Case matters: the engine's ids are exact, so a wrong case is a wrong name
+    // rather than a near miss to be forgiven.
+    const auto wrong_case = select_named_output(kDemucs, "Vocals");
+    check(wrong_case.index == -1 && !wrong_case.error.empty(),
+          "stem names are matched case sensitively");
+}
+
+static void test_no_named_outputs() {
+    const auto choice = select_named_output({}, "");
+    check(choice.index == -1, "an empty output list selects nothing");
+    check(choice.error.empty(),
+          "an empty output list is not an error here: the caller decides");
+    const auto requested = select_named_output({}, "vocals");
+    check(requested.index == -1 && requested.error.empty(),
+          "an empty output list stays the caller's decision even when a stem was asked for");
+}
+
+static void test_unwritable_names_are_refused() {
+    // Model-supplied names become file path components. A separator would write
+    // outside the caller's output directory.
+    const std::vector<std::string> traversal = {"vocals", "../../etc/passwd"};
+    const auto escaped = select_named_output(traversal, "vocals");
+    check(escaped.index == -1 && !escaped.error.empty(),
+          "a stem name containing a path separator is refused");
+    check(escaped.error.find("../../etc/passwd") != std::string::npos,
+          "the refusal names the offending stem");
+
+    check(!select_named_output({"vo\\cals", "drums"}, "").error.empty(),
+          "a backslash separator is refused too");
+    check(!select_named_output({"drums", ""}, "").error.empty(),
+          "an empty stem name is refused");
+    check(!select_named_output({"drums", "."}, "").error.empty(),
+          "a stem named '.' is refused");
+    check(!select_named_output({"drums", ".."}, "").error.empty(),
+          "a stem named '..' is refused");
+
+    // The check covers EVERY name, not only the selected one: all of them are
+    // written, so a bad fourth name must not be found after three files exist.
+    const auto late = select_named_output({"vocals", "drums", "bass", "a/b"}, "vocals");
+    check(late.index == -1 && !late.error.empty(),
+          "an unwritable name after the selected one still refuses the whole request");
+
+    // Control bytes, and the NUL case is why the whole range is refused. These
+    // two names are DIFFERENT std::strings, so the duplicate check does not
+    // fire, yet both truncate to "vocals" at path::c_str() and would open one
+    // file: the silent overwrite the duplicate check exists to prevent, with
+    // the ".wav" stripped off into the bargain.
+    const std::string nul_a("vocals\0drums", 12);
+    const std::string nul_b("vocals\0bass", 11);
+    check(nul_a != nul_b, "the two NUL names really are distinct std::strings");
+    check(std::string(nul_a.c_str()) == "vocals" &&
+              std::string(nul_b.c_str()) == "vocals",
+          "and both truncate to the same C string, which is the hazard");
+    const auto nul_pair = select_named_output({nul_a, nul_b}, "");
+    check(nul_pair.index == -1 && !nul_pair.error.empty(),
+          "two stem names differing only after an embedded NUL are refused");
+    check(!select_named_output({"drums", std::string("vo\0cals", 7)}, "").error.empty(),
+          "a single embedded NUL is refused on its own");
+    check(!select_named_output({"drums", "voc\nals"}, "").error.empty(),
+          "a newline in a stem name is refused");
+    check(!select_named_output({"drums", "voc\tals"}, "").error.empty(),
+          "a tab in a stem name is refused");
+    check(!select_named_output({"drums", "voc\033[31mals"}, "").error.empty(),
+          "an escape sequence in a stem name is refused");
+    check(!select_named_output({"drums", "voc\177als"}, "").error.empty(),
+          "DEL in a stem name is refused");
+
+    // The boundary below the refused range is the space, which is an ordinary
+    // file name character and must stay usable, or this check would be
+    // refusing real stem names.
+    const auto spaced = select_named_output({"lead vocals", "drums"}, "lead vocals");
+    check(spaced.index == 0 && spaced.error.empty(),
+          "a space is not a control character and stays usable");
+    // And every byte above DEL: a UTF-8 stem name is ordinary, and signed char
+    // would make those bytes compare as negative.
+    const auto utf8 = select_named_output({"vocals", "b\xc3\xa4sse"}, "b\xc3\xa4sse");
+    check(utf8.index == 1 && utf8.error.empty(),
+          "a UTF-8 stem name is not mistaken for a control character");
+
+    // A leading dot is not a traversal and must stay usable.
+    const auto dotted = select_named_output({".vocals", "drums"}, ".vocals");
+    check(dotted.index == 0 && dotted.error.empty(),
+          "a leading dot in a stem name is allowed");
+}
+
+static void test_duplicate_names_are_refused() {
+    const auto choice = select_named_output({"vocals", "drums", "vocals"}, "vocals");
+    check(choice.index == -1 && !choice.error.empty(),
+          "two stems sharing a name are refused: one file would overwrite the other");
+    check(choice.error.find("vocals") != std::string::npos,
+          "the duplicate refusal names the repeated stem");
+}
+
+static void test_sibling_paths() {
+    check_equal(sibling_stem_path("/generated/transform-1.wav", "drums"),
+                "/generated/transform-1.drums.wav", "sibling beside an absolute dst");
+    check_equal(sibling_stem_path("sep.wav", "vocals"), "sep.vocals.wav",
+                "sibling of a bare file name has no directory");
+    check_equal(sibling_stem_path("/out/sep", "vocals"), "/out/sep.vocals.wav",
+                "an extensionless dst gets .wav");
+    check_equal(sibling_stem_path("/out/take.2.wav", "bass"), "/out/take.2.bass.wav",
+                "only the final extension is treated as the extension");
+    check_equal(sibling_stem_path("/out/sep.WAV", "bass"), "/out/sep.bass.WAV",
+                "the caller's extension spelling is preserved");
+    check_equal(sibling_stem_path("/a b/c d.wav", "other"), "/a b/c d.other.wav",
+                "spaces in the destination survive");
+
+    // The property that matters: no stem can ever be written over dst itself,
+    // or the "dst holds the selected stem" contract would depend on write order.
+    const std::string dst = "/out/sep.wav";
+    for (const auto &name : kDemucs) {
+        check(sibling_stem_path(dst, name) != dst,
+              "the sibling for '" + name + "' is not dst itself");
+    }
+    // Distinct stems must land in distinct files.
+    check(sibling_stem_path(dst, "drums") != sibling_stem_path(dst, "bass"),
+          "two stems get two different sibling paths");
+}
+
+// The contract grpc-server.cpp indexes on: an accepted choice over a non-empty
+// name list is always in range, so the handler needs no bounds guard of its own.
+// A -1 reaching the subscript would become a colossal size_t.
+static void test_accepted_index_is_always_in_range() {
+    const std::vector<std::vector<std::string>> lists = {
+        kDemucs, kRoformer, {"solo"}, {"accompaniment", "drums"}, {"a", "b", "c"}};
+    const std::vector<std::string> requests = {"", "vocals", "drums", "solo", "c",
+                                               "kazoo", "..", "a/b"};
+    for (const auto &names : lists) {
+        for (const auto &requested : requests) {
+            const auto choice = select_named_output(names, requested);
+            if (!choice.error.empty()) {
+                check(choice.index == -1,
+                      "a refusal never carries an index (request '" + requested + "')");
+                continue;
+            }
+            check(choice.index >= 0 &&
+                      choice.index < static_cast<int>(names.size()),
+                  "an accepted choice is in range (request '" + requested + "')");
+            // And the selected name is the one that was asked for, when one was.
+            if (!requested.empty()) {
+                check(names[static_cast<size_t>(choice.index)] == requested,
+                      "an accepted explicit request selects that exact name");
+            }
+        }
+    }
+}
+
+int main() {
+    test_accepted_index_is_always_in_range();
+    test_default_selection();
+    test_explicit_selection();
+    test_unknown_stem_is_refused();
+    test_no_named_outputs();
+    test_unwritable_names_are_refused();
+    test_duplicate_names_are_refused();
+    test_sibling_paths();
+    if (failures != 0) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all stem_selection checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/stream_delta.cpp

@@ -0,0 +1,204 @@
+#include "stream_delta.h"
+
+namespace audiocpp_backend {
+namespace {
+
+// True when `text` begins with `prefix`. An empty prefix matches everything,
+// which is what makes the first fragment take the cumulative branch and the
+// incremental branch alike: they agree there.
+bool starts_with(const std::string &text, const std::string &prefix) {
+    return text.size() >= prefix.size() &&
+           text.compare(0, prefix.size(), prefix) == 0;
+}
+
+// Number of leading bytes of `text` that cannot BEGIN a character, i.e. orphan
+// continuation bytes with no lead byte in front of them.
+//
+// They are unrecoverable rather than early: the byte that would have led them
+// has already gone past, and nothing can be prepended to a fragment after the
+// fact. Holding them would stall the stream for good, and emitting them puts
+// invalid UTF-8 on the wire, so the caller DROPS them. Losing a byte keeps the
+// stream alive; emitting one ends it, and takes the final_result still to come
+// with it.
+std::size_t utf8_orphan_prefix_length(const std::string &text) {
+    std::size_t index = 0;
+    while (index < text.size() &&
+           (static_cast<unsigned char>(text[index]) & 0xC0) == 0x80) {
+        ++index;
+    }
+    return index;
+}
+
+// Length of the longest prefix of `text` that does NOT end inside a multi-byte
+// UTF-8 sequence, i.e. the most that can go on the wire without splitting a
+// character in half.
+//
+// Only the TRAILING sequence is examined; the leading side is
+// utf8_orphan_prefix_length's job. Bytes in the MIDDLE are neither one's
+// business: a family that emitted a malformed sequence inside its own text
+// cannot be repaired here without deleting part of that transcript, and the
+// stream is lost anyway, because final_result.text carries the same bytes
+// through the same proto3 string field.
+//
+// Anything that can never complete is reported as complete, so it goes out
+// rather than being held forever: a lead byte the encoding does not define, and
+// a run of five or more continuation bytes, are both passed through. A tracker
+// that stalled on undecodable input would turn one bad byte into a permanently
+// silent stream, which is worse than the bad byte.
+std::size_t utf8_complete_prefix_length(const std::string &text) {
+    std::size_t index = text.size();
+    std::size_t continuations = 0;
+    while (index > 0 && continuations < 4) {
+        const auto byte = static_cast<unsigned char>(text[index - 1]);
+        if ((byte & 0xC0) == 0x80) {
+            --index;
+            ++continuations;
+            continue;
+        }
+        std::size_t needed = 1;
+        if ((byte & 0x80) == 0x00) {
+            needed = 1;
+        } else if ((byte & 0xE0) == 0xC0) {
+            needed = 2;
+        } else if ((byte & 0xF0) == 0xE0) {
+            needed = 3;
+        } else if ((byte & 0xF8) == 0xF0) {
+            needed = 4;
+        } else {
+            // Not a lead byte this encoding defines, so nothing is waiting on
+            // it and it must not be held.
+            needed = 1;
+        }
+        if (continuations + 1 >= needed) {
+            return text.size();
+        }
+        // The trailing sequence is short by at least one byte: cut before its
+        // lead byte and keep the rest for the next fragment.
+        return index - 1;
+    }
+    return text.size();
+}
+
+} // namespace
+
+std::string TranscriptDeltaTracker::release(const std::string &fragment) {
+    // Held-back bytes go in FRONT of whatever arrived next, or the character
+    // they begin is reassembled in the wrong order.
+    std::string candidate = pending_ + fragment;
+    // A fragment must not BEGIN mid-character either. pending_ always starts on
+    // a lead byte, so this only bites when nothing was held and the caller's
+    // rules dropped the lead byte somewhere upstream; it is the backstop that
+    // makes "no delta this class returns is ever invalid UTF-8" true of the
+    // FRONT as well as the back, independently of those rules being right.
+    candidate.erase(0, utf8_orphan_prefix_length(candidate));
+    const std::size_t cut = utf8_complete_prefix_length(candidate);
+    pending_ = candidate.substr(cut);
+    std::string emitted = candidate.substr(0, cut);
+    assembled_ += emitted;
+    return emitted;
+}
+
+std::string TranscriptDeltaTracker::observe(const std::string &partial_text) {
+    if (partial_text.empty()) {
+        return {};
+    }
+    // The comparisons run against everything KNOWN, delivered plus held back,
+    // rather than against the delivered text alone. Comparing against the
+    // delivered text would treat the held-back byte as new on the very next
+    // report and emit it twice.
+    const std::string known = assembled_ + pending_;
+    // An EXACT repeat, and nothing looser. This absorbs the duplicate delivery
+    // voxtral_realtime produces, which is the ONLY thing rule 2 was ever needed
+    // for: process_available_stream_chunks hands each event it produces to the
+    // sink from inside its loop and RETURNS the last of the batch
+    // (session.cpp:385-386), so that last event arrives twice with byte-equal
+    // text both times. A duplicate IS an exact repeat, so equality covers it.
+    //
+    // It used to discard any report the known text merely STARTED WITH, and that
+    // cost far more than it bought. Two separate defects came out of it, and
+    // both were found by randomized traces rather than by reading:
+    //
+    //   1. A short INCREMENTAL fragment that happens to be a byte prefix of the
+    //      transcript so far was read as a repeat and dropped, losing text with
+    //      a 200 and no diagnostic. Both incremental families emit fragments
+    //      that small routinely: nemotron_asr cuts at a byte offset
+    //      (decoder.cpp:550) and vibevoice_asr at a common prefix
+    //      (session.cpp:89-105). 9.50% of randomized pure-ASCII traces and
+    //      29.12% of French ones ended with a corrupted transcript.
+    //   2. When such a fragment was the LEAD BYTE of a multi-byte character, its
+    //      continuation bytes then arrived alone and began the next delta, which
+    //      is invalid UTF-8, which the Go runtime refuses to unmarshal, which
+    //      ends the stream and the final_result with it.
+    //
+    // What the narrowing gives up is the shrinking-hypothesis case: a cumulative
+    // report SHORTER than what is known is now read as an incremental fragment
+    // and duplicates those bytes at the client. No pinned family produces one.
+    // voxtral is the only cumulative reporter, and its hypothesis is
+    // tokenizer_.decode(streaming_token_ids_) over a vector that is only ever
+    // push_back'ed (session.cpp:436) and cleared by reset() (session.cpp:257),
+    // so within a stream it can only grow. Measured: narrowing this changed not
+    // one byte of 30,000 randomized cumulative traces.
+    //
+    // KEPT DELIBERATELY THOUGH NO TEST CAN SEE IT. Once narrowed to equality
+    // this rule became redundant with rule 3 below: an equal partial has an
+    // empty suffix, so rule 3 would call release("") and emit nothing either
+    // way. Deleting it is therefore an equivalent mutation, and the mutation
+    // harness reports it as a survivor, which is the honest result and not a
+    // gap in the tests. It stays for two reasons: it states the duplicate
+    // absorption where the citation for it lives, and it is independent of rule
+    // 3's condition. A future tightening of rule 3 to, say, require a STRICTLY
+    // longer partial would otherwise send every duplicate down the incremental
+    // branch and put the whole transcript on the wire a second time.
+    if (partial_text == known) {
+        return {};
+    }
+    if (starts_with(partial_text, known)) {
+        // Cumulative: the report is the whole transcript so far.
+        return release(partial_text.substr(known.size()));
+    }
+    // Incremental: the fragment is new text to append.
+    //
+    // A family that REWRITES its hypothesis lands here too, and the client's
+    // view is then wrong in a way nothing downstream can fix. nemotron's
+    // decoder has such a branch (decoder.cpp:552-554): when the new text is not
+    // an extension of what it already emitted, it emits the whole new text. So
+    // "the cat sat" followed by "the cat sap" leaves the client holding
+    // "the cat satthe cat sap", and reconcile then correctly refuses to append
+    // to a contradicted assembly, which leaves concat(deltas) != final with no
+    // signal on the wire. This is NOT repaired here, and the reason is that a
+    // delta stream has no retraction: emitting only the differing suffix would
+    // read as "sap" appended to "the cat sat", which is a different wrong
+    // answer, and emitting a correction would need a wire field that does not
+    // exist. final_result carries the authoritative text either way. It did not
+    // fire in a 331 delta run, because an RNN-T decode is monotonic in practice.
+    return release(partial_text);
+}
+
+std::string TranscriptDeltaTracker::reconcile(const std::string &final_text) {
+    if (final_text.empty() || final_text == assembled_) {
+        // Nothing further is owed. Held-back bytes are dropped rather than
+        // flushed: they are not in the authoritative text, so sending them
+        // would contradict it.
+        pending_.clear();
+        return {};
+    }
+    if (!starts_with(final_text, assembled_)) {
+        // Contradicted. Nothing sent can be taken back, so nothing more is
+        // sent; final_result carries the authoritative text.
+        pending_.clear();
+        return {};
+    }
+    // Compared against the DELIVERED text, so the fragment below already
+    // contains whatever was held back. pending_ is therefore cleared rather
+    // than prepended, or those bytes would go out twice.
+    //
+    // The fragment ends on a character boundary whenever final_text is
+    // well-formed, which is the normal case and the reason a held-back sequence
+    // is always flushed here. It is still cut, so a family handing back a final
+    // text that is itself truncated mid-character cannot put a partial sequence
+    // on the wire through this path either.
+    pending_.clear();
+    return release(final_text.substr(assembled_.size()));
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stream_delta.h

@@ -0,0 +1,133 @@
+#pragma once
+
+// Turns whatever a streaming session calls a "partial transcript" into the
+// incremental deltas AudioTranscriptionStream is contracted to send. Standard
+// library only, so it is tested without an audio.cpp checkout.
+//
+// THIS UNIT EXISTS BECAUSE THE FAMILIES DISAGREE, and the disagreement is
+// invisible at the interface: StreamEvent::partial_text is a Transcript either
+// way. Read out of the pinned upstream, one family at a time:
+//
+//   nemotron_asr      INCREMENTAL. decoder.cpp emits
+//                     current_text.substr(emitted_text.size()) per non-blank
+//                     token, and only through the stream event SINK, during
+//                     finalize(). process_audio_chunk returns empty events.
+//   vibevoice_asr     INCREMENTAL. process_audio_chunk returns
+//                     text.substr(common_prefix_size(...)); the sink is
+//                     deliberately swapped out around its internal run_single,
+//                     so the fragment arrives once, on the return value.
+//   higgs_audio_stt   INCREMENTAL. Same shape as vibevoice_asr.
+//   voxtral_realtime  CUMULATIVE. partial_text is
+//                     tokenizer_.decode(streaming_token_ids_), the whole
+//                     hypothesis so far. process_available_stream_chunks hands
+//                     every event it produces to the sink from INSIDE its loop
+//                     (session.cpp:385-386) and RETURNS only the last of the
+//                     batch, so the last event of each batch arrives twice and
+//                     the others arrive once.
+//
+// Applying either convention to the other family corrupts the transcript: read
+// a cumulative report as a delta and the client sees the transcript repeated on
+// every event; read an incremental fragment as cumulative and the suffix
+// arithmetic eats the front of it. So the tracker decides per fragment, from
+// what it has already delivered, and the one rule it enforces is that TEXT THE
+// CLIENT HAS ALREADY BEEN SENT IS NEVER SENT AGAIN.
+//
+// The cumulative reading is provably safe for voxtral, which is the family it
+// matters for: its decode is a pure concatenation of per-token byte strings
+// (tokenizer_text.cpp:171-183), so decode(ids[0..n]) is an unconditional BYTE
+// PREFIX of decode(ids[0..n+1]) and one of its reports can never be mistaken
+// for an incremental fragment.
+//
+// UTF-8 IS THE OTHER HALF OF THAT SAME FACT. Because that decode concatenates
+// raw token BYTES, a multi-byte character is split across token boundaries, and
+// the difference between two consecutive cumulative reports is then a lone
+// continuation byte. TranscriptStreamResponse.delta is a proto3 `string`, whose
+// wire format REQUIRES valid UTF-8: the C++ runtime serializes an invalid one
+// with at most a warning, but the Go runtime refuses to unmarshal it, and the
+// client loses every remaining delta AND the final_result. So no fragment this
+// class returns ever BEGINS OR ENDS inside a character: an incomplete trailing
+// sequence is held back and merged into the next fragment, and a leading orphan
+// continuation byte, which nothing can ever complete, is dropped.
+//
+// It is NOT a voxtral-only concern, which is what the first attempt at this
+// assumed. The incremental families split characters by the same arithmetic:
+// nemotron_asr's decoder cuts at a BYTE offset (decoder.cpp:550) and
+// vibevoice_asr's common_prefix_size compares BYTES (session.cpp:80-86). Nor is
+// European text the worst case: a Japanese transcript, whose every character is
+// three bytes, carried at least one invalid delta in 33.74% of traces until
+// rule 2 below learned to leave an incomplete fragment alone.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+class TranscriptDeltaTracker {
+public:
+    // Takes one StreamEvent::partial_text and returns the fragment to put on
+    // the wire, empty when there is nothing new.
+    //
+    // The rules, in order, all of them against everything KNOWN (delivered
+    // plus held back), never against the delivered text alone:
+    //   1. An empty partial says nothing.
+    //   2. A partial IDENTICAL to the known text is a repeat: nothing is
+    //      emitted. Identical, not merely a prefix of it. That absorbs voxtral's
+    //      repeat of the last event in each batch, which is the only duplicate
+    //      any pinned family produces and which carries byte-equal text both
+    //      times. Discarding a mere PREFIX used to swallow an incremental
+    //      fragment that coincided with the start of the transcript, corrupting
+    //      the text silently and, when that fragment was a character's lead
+    //      byte, killing the stream outright; see the note at the rule in the
+    //      implementation.
+    //   3. A partial that EXTENDS the known text is a cumulative report: only
+    //      its new suffix is emitted.
+    //   4. Anything else is an incremental fragment: it is emitted whole and
+    //      appended.
+    //
+    // Rule 3 is the one judgement call, since a fragment that happens to begin
+    // with the entire transcript so far is indistinguishable from a cumulative
+    // report. It is read as cumulative because every cumulative family produces
+    // that shape on EVERY event, while an incremental family produces it only
+    // when one fragment repeats everything before it, which no tokenizer output
+    // does in practice.
+    //
+    // What comes back is the fragment MINUS any incomplete trailing UTF-8
+    // sequence, which is carried into the next call, and minus any leading
+    // orphan continuation byte, which is dropped. So an empty return can also
+    // mean "the only new bytes were half a character", and the caller needs no
+    // knowledge of that: writing nothing is exactly right.
+    std::string observe(const std::string &partial_text);
+
+    // Reconciles against TaskResult::text_output, which is authoritative, and
+    // returns the fragment that makes appending every delta equal it.
+    //
+    // This is what makes the OFFLINE FALLBACK a single line rather than its own
+    // branch: with no partials observed, the assembly is empty and the whole
+    // final text comes back as one delta.
+    //
+    // It is also what FLUSHES a held-back UTF-8 sequence, and it can always do
+    // so: the final text is complete, so the fragment from the last delivered
+    // byte to its end ends on a character boundary.
+    //
+    // A final text that CONTRADICTS what was already sent returns empty. A
+    // fragment on the wire cannot be retracted, so the alternative would be to
+    // send the transcript a second time and let the client hold it twice.
+    // final_result carries the authoritative text either way.
+    std::string reconcile(const std::string &final_text);
+
+    // Everything the client has been sent, concatenated. Held-back bytes are
+    // deliberately NOT included: this is what the client holds, not what the
+    // tracker knows.
+    const std::string &assembled() const noexcept { return assembled_; }
+
+private:
+    // Appends the emittable prefix of `fragment` to assembled_ and returns it,
+    // keeping any incomplete trailing UTF-8 sequence in pending_.
+    std::string release(const std::string &fragment);
+
+    std::string assembled_;
+    // An incomplete trailing UTF-8 sequence, computed but not sent. Always a
+    // proper prefix of one character, so at most three bytes.
+    std::string pending_;
+};
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stream_delta_test.cpp

@@ -0,0 +1,486 @@
+// Unit tests for stream_delta. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+//
+// The traces below are transcribed from the pinned upstream sessions rather
+// than invented, because the whole reason this unit exists is that the four
+// streaming ASR families do NOT agree on what partial_text means.
+
+#include "stream_delta.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+using audiocpp_backend::TranscriptDeltaTracker;
+
+// Feeds a whole trace and returns what a client appending every emitted
+// fragment would end up holding, which is the only property that matters.
+static std::string client_view(TranscriptDeltaTracker &tracker,
+                               const std::vector<std::string> &partials,
+                               std::vector<std::string> *emitted = nullptr) {
+    std::string view;
+    for (const auto &partial : partials) {
+        const std::string fragment = tracker.observe(partial);
+        if (emitted != nullptr && !fragment.empty()) {
+            emitted->push_back(fragment);
+        }
+        view += fragment;
+    }
+    return view;
+}
+
+// nemotron_asr: decoder.cpp emits current_text.substr(emitted_text.size()) on
+// every non-blank token, i.e. INCREMENTAL fragments, through the stream event
+// sink during finalize().
+static void test_incremental_family() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view =
+        client_view(tracker, {"Local", " AI", " now", " speaks."}, &emitted);
+    check_eq(view, "Local AI now speaks.", "incremental deltas concatenate");
+    check(emitted.size() == 4, "incremental family emits one fragment per partial");
+    check_eq(tracker.assembled(), "Local AI now speaks.",
+             "incremental family assembles the whole transcript");
+    check_eq(tracker.reconcile("Local AI now speaks."), "",
+             "a final result the deltas already cover adds nothing");
+}
+
+// voxtral_realtime: process_one_stream_chunk sets partial_text to
+// tokenizer_.decode(streaming_token_ids_), the WHOLE accumulated hypothesis,
+// and process_available_stream_chunks then hands the SAME event to both the
+// sink and the caller, so every partial arrives twice.
+static void test_cumulative_family_with_duplicate_delivery() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view = client_view(
+        tracker, {"Local", "Local", "Local AI", "Local AI", "Local AI now",
+                  "Local AI now"},
+        &emitted);
+    check_eq(view, "Local AI now", "cumulative partials are not repeated to the client");
+    check(emitted.size() == 3,
+          "the duplicate delivery of each cumulative event emits nothing twice");
+    check_eq(emitted.empty() ? "" : emitted[0], "Local", "first cumulative fragment");
+    check_eq(emitted.size() < 2 ? "" : emitted[1], " AI", "second cumulative fragment");
+    check_eq(emitted.size() < 3 ? "" : emitted[2], " now", "third cumulative fragment");
+}
+
+// The rule that separates the two: a report IDENTICAL to everything known is a
+// repeat and is never sent again.
+//
+// Only an identical one. Rule 2 used to discard any report the known text merely
+// STARTED WITH, and that cost more than it bought: see
+// test_a_short_fragment_is_not_mistaken_for_a_repeat.
+static void test_an_exact_repeat_is_never_resent() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("hello world"), "hello world", "first fragment");
+    check_eq(tracker.observe("hello world"), "", "an identical repeat emits nothing");
+    check_eq(tracker.observe("hello world"), "", "and a third delivery emits nothing");
+    check_eq(tracker.assembled(), "hello world", "the assembly is unchanged by repeats");
+}
+
+// THE TRADE, pinned so it is a decision rather than a surprise. A CUMULATIVE
+// report that SHRINKS is no longer absorbed: it is read as an incremental
+// fragment and duplicates a few bytes at the client.
+//
+// No pinned family produces one. voxtral_realtime is the only cumulative
+// reporter, and its hypothesis is tokenizer_.decode(streaming_token_ids_) over a
+// vector that is only ever push_back'ed (session.cpp:436) and cleared by reset()
+// (session.cpp:257), so within a stream it grows and never shrinks. The
+// duplicate delivery rule 2 really exists for is an EXACT repeat, which the test
+// above still covers.
+static void test_a_shrinking_hypothesis_is_read_as_incremental() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("hello world"), "hello world", "first fragment");
+    check_eq(tracker.observe("hello"), "hello",
+             "a shortened hypothesis is now read as an incremental fragment");
+    check_eq(tracker.assembled(), "hello worldhello",
+             "which duplicates those bytes at the client: the accepted cost");
+}
+
+static void test_empty_partials_are_ignored() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe(""), "", "an empty partial emits nothing");
+    check_eq(tracker.observe("a"), "a", "a real partial after an empty one still emits");
+    check_eq(tracker.observe(""), "", "a later empty partial emits nothing");
+    check_eq(tracker.assembled(), "a", "empty partials do not disturb the assembly");
+}
+
+// The offline fallback: a family with no streaming ASR runs once, so nothing is
+// ever observed and the reconciliation IS the single delta the RPC promises.
+static void test_offline_fallback_is_one_delta() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.reconcile("the whole transcript"), "the whole transcript",
+             "with no partials the final text is emitted whole");
+    check_eq(tracker.assembled(), "the whole transcript",
+             "the reconciliation is recorded as delivered");
+    check_eq(tracker.reconcile("the whole transcript"), "",
+             "reconciling twice does not duplicate");
+}
+
+// A streaming family whose partials stopped short of the final text: the tail
+// is emitted so that appending every delta still equals final_result.text.
+static void test_reconcile_emits_the_tail() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("Local AI"), "Local AI", "partial arrives");
+    check_eq(tracker.reconcile("Local AI now speaks."), " now speaks.",
+             "the untold tail of the final text is emitted");
+    check_eq(tracker.assembled(), "Local AI now speaks.", "tail is recorded");
+}
+
+// Divergence. nemotron's decoder has a rewrite branch: when the new hypothesis
+// is NOT an extension of what it already emitted, it emits the whole new text.
+// Nothing can retract a fragment already written to the wire, so the tracker
+// must not try: it emits nothing further and leaves final_result authoritative.
+static void test_divergent_final_text_is_not_appended() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("the cat"), "the cat", "first hypothesis");
+    check_eq(tracker.reconcile("the dog"), "",
+             "a final text that contradicts the deltas is not appended to them");
+    check_eq(tracker.assembled(), "the cat",
+             "a contradicted assembly is left as it was actually sent");
+}
+
+static void test_empty_final_text() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("something"), "something", "partial arrives");
+    check_eq(tracker.reconcile(""), "", "an empty final text emits nothing");
+    check_eq(tracker.assembled(), "something", "an empty final text changes nothing");
+}
+
+// A whitespace-only fragment is real text: the space between two words is
+// exactly what an incremental family delivers on its own.
+static void test_whitespace_fragments_survive() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("one"), "one", "word");
+    check_eq(tracker.observe(" "), " ", "a bare separator is emitted");
+    check_eq(tracker.observe("two"), "two", "next word");
+    check_eq(tracker.assembled(), "one two", "separator is kept in the assembly");
+}
+
+// The ambiguity this unit cannot resolve, pinned so that a future reader sees
+// the choice rather than rediscovering it: a fragment that EXTENDS everything
+// delivered so far is read as a cumulative report, because that is what every
+// cumulative family produces on every event, while an incremental family
+// producing one is the rare coincidence of a fragment repeating the whole
+// transcript so far.
+static void test_prefix_extension_is_read_as_cumulative() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("I"), "I", "first fragment");
+    check_eq(tracker.observe("I'm"), "'m",
+             "a fragment extending the assembly is treated as a cumulative report");
+    check_eq(tracker.assembled(), "I'm", "cumulative reading assembles once");
+}
+
+// --------------------------------------------------------------------------
+// UTF-8 boundaries
+//
+// TranscriptStreamResponse.delta is a proto3 `string`, and the wire format
+// REQUIRES a string field to be valid UTF-8. The C++ runtime serializes an
+// invalid one with at most a warning; the Go runtime refuses to unmarshal it,
+// so the client loses every delta AND the final_result still to come.
+//
+// Not hypothetical. voxtral_realtime reports the whole hypothesis as
+// tokenizer_.decode(streaming_token_ids_), and that decode is a pure
+// concatenation of raw token BYTES (tokenizer_text.cpp:171-183), so a
+// multi-byte character is split across token boundaries and the cumulative
+// difference between two consecutive reports is a lone continuation byte.
+// --------------------------------------------------------------------------
+
+// True when `text` is well-formed UTF-8. Written out here rather than reused
+// from the implementation on purpose: a test that shares the implementation's
+// idea of a boundary cannot catch the implementation's idea being wrong.
+static bool is_valid_utf8(const std::string &text) {
+    size_t i = 0;
+    while (i < text.size()) {
+        const auto lead = static_cast<unsigned char>(text[i]);
+        size_t length = 0;
+        if ((lead & 0x80) == 0x00) {
+            length = 1;
+        } else if ((lead & 0xE0) == 0xC0) {
+            length = 2;
+        } else if ((lead & 0xF0) == 0xE0) {
+            length = 3;
+        } else if ((lead & 0xF8) == 0xF0) {
+            length = 4;
+        } else {
+            return false;
+        }
+        if (i + length > text.size()) {
+            return false;
+        }
+        for (size_t k = 1; k < length; ++k) {
+            if ((static_cast<unsigned char>(text[i + k]) & 0xC0) != 0x80) {
+                return false;
+            }
+        }
+        i += length;
+    }
+    return true;
+}
+
+// Written as byte escapes so the test does not depend on the encoding of this
+// source file.
+static const std::string kEAcute = "\xC3\xA9";        // 2 bytes
+static const std::string kEuro = "\xE2\x82\xAC";      // 3 bytes
+static const std::string kEmoji = "\xF0\x9F\x8E\xA7"; // 4 bytes
+
+// A cumulative family advancing its hypothesis one BYTE at a time, which is
+// what voxtral_realtime does across a multi-byte character.
+static void test_cumulative_split_multibyte_character() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string full = "5" + kEuro;
+    std::vector<std::string> partials;
+    for (size_t n = 1; n <= full.size(); ++n) {
+        partials.push_back(full.substr(0, n));
+    }
+    const std::string view = client_view(tracker, partials, &emitted);
+
+    check_eq(view, full, "a byte-at-a-time cumulative report still assembles");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "cumulative fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check_eq(tracker.reconcile(full), "", "the final text adds nothing");
+}
+
+// An incremental family splitting a character across two fragments.
+static void test_incremental_split_multibyte_character() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view = client_view(
+        tracker, {"caf" + kEAcute.substr(0, 1), kEAcute.substr(1), " au lait"},
+        &emitted);
+
+    check_eq(view, "caf" + kEAcute + " au lait",
+             "an incremental split character still assembles");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "incremental fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check(emitted.size() == 3, "one fragment out per partial, none swallowed");
+    if (emitted.size() == 3) {
+        check_eq(emitted[0], "caf", "the lead byte of the character is held back");
+        check_eq(emitted[1], kEAcute,
+                 "the held byte is merged into the next fragment, not sent alone");
+        check_eq(emitted[2], " au lait", "the rest follows unchanged");
+    }
+}
+
+// A 4 byte character split three ways, so the held-back buffer has to survive
+// more than one round.
+static void test_four_byte_character_split_three_ways() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view =
+        client_view(tracker,
+                    {"listen " + kEmoji.substr(0, 1), kEmoji.substr(1, 2),
+                     kEmoji.substr(3), " now"},
+                    &emitted);
+    check_eq(view, "listen " + kEmoji + " now",
+             "a 4 byte character survives three splits");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "4 byte fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+}
+
+// The held-back bytes must reach the client. reconcile can always flush them,
+// because the final text is complete by construction.
+static void test_reconcile_flushes_a_held_back_sequence() {
+    TranscriptDeltaTracker tracker;
+    const std::string first = tracker.observe("done" + kEuro.substr(0, 2));
+    check_eq(first, "done", "the incomplete trailing sequence is held back");
+    check(is_valid_utf8(first), "what was emitted is valid UTF-8");
+    const std::string tail = tracker.reconcile("done" + kEuro);
+    check_eq(tail, kEuro, "reconcile flushes the completed character");
+    check(is_valid_utf8(tail), "the flushed tail is valid UTF-8");
+    check_eq(tracker.assembled(), "done" + kEuro, "the client holds the whole text");
+}
+
+// Held-back bytes are not lost track of: the next cumulative report emits the
+// whole character rather than only the bytes that just arrived.
+static void test_held_bytes_join_the_next_fragment() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("a" + kEuro.substr(0, 1)), "a",
+             "only the complete prefix goes out");
+    check_eq(tracker.observe("a" + kEuro), kEuro,
+             "the next cumulative report emits the whole character at once");
+    check_eq(tracker.assembled(), "a" + kEuro, "assembly is correct");
+}
+
+// A whole multi-byte character arriving at once must NOT be held back: holding
+// a complete sequence would stall every stream by one character.
+static void test_a_complete_character_is_not_held() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("x" + kEuro), "x" + kEuro,
+             "a fragment ending on a boundary is emitted immediately");
+    check_eq(tracker.observe("x" + kEuro + kEmoji), kEmoji,
+             "and so is the next one");
+}
+
+// Bytes that can never complete must not be held forever: a stray continuation
+// byte or an invalid lead is passed through rather than stalling the stream.
+// Repairing a family's malformed output is not something a delta tracker can do
+// without altering the transcript.
+static void test_undecodable_bytes_are_not_held_forever() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe(std::string("ok\x80")), std::string("ok\x80"),
+             "a stray continuation byte is passed through, not held");
+    check_eq(tracker.observe(std::string("ok\x80") + "next"), "next",
+             "the stream continues");
+
+    // A lead byte the encoding does not define (0xF8 and above). Nothing can
+    // ever complete it, so holding it would stall the stream for good.
+    TranscriptDeltaTracker invalid_lead;
+    check_eq(invalid_lead.observe(std::string("ok\xFE")), std::string("ok\xFE"),
+             "an undefined lead byte is passed through, not held");
+    check_eq(invalid_lead.observe(std::string("ok\xFE") + "more"), "more",
+             "the stream continues past an undefined lead byte");
+
+    // The same byte followed by continuation bytes, which is the shape that
+    // looks most like a real sequence waiting to be completed.
+    TranscriptDeltaTracker invalid_run;
+    check_eq(invalid_run.observe(std::string("\xFE\x80\x80")), std::string("\xFE\x80\x80"),
+             "an undefined lead with continuations is passed through");
+
+    // Five continuation bytes with no lead in sight. They are DROPPED, not
+    // held: nothing can ever precede them, so holding would stall the stream
+    // for good, and emitting them would put invalid UTF-8 on the wire. See
+    // test_a_fragment_never_begins_mid_character.
+    TranscriptDeltaTracker orphans;
+    check_eq(orphans.observe(std::string("\x80\x80\x80\x80\x80")), "",
+             "a run of orphan continuation bytes is dropped, not emitted");
+    check_eq(orphans.observe("after"), "after",
+             "and the stream continues past them");
+}
+
+// THE SECOND HALF OF THE SAME BUG, and the one that survived fix round 1.
+//
+// Rule 2 discards a fragment the known text already starts with. When that
+// fragment is the LEAD BYTE of a NEW character it looks exactly like a repeat of
+// an earlier character beginning with the same byte, so it was discarded and
+// never held. Its continuation bytes then arrived on their own and began the
+// next delta, which is invalid UTF-8 at the FRONT, and utf8_complete_prefix_length
+// only ever inspected the TRAILING sequence.
+//
+// Reachable from shipping families, not synthetic: nemotron_asr's
+// decoder.cpp:550 cuts at a BYTE offset (current_text.substr(emitted_text.size()))
+// and vibevoice_asr's common_prefix_size (session.cpp:80-86) compares BYTES, so
+// both split characters mid-sequence. The trace below is exactly how they split
+// "ssee" spelled with the German sharp s, an e-acute, a euro sign and an o-grave,
+// three of which begin with the same 0xC3 lead byte.
+static void test_a_repeated_lead_byte_is_not_swallowed() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string sharp_s = "\xC3\x9F";  // U+00DF
+    const std::string e_acute = "\xC3\xA9";  // U+00E9
+    const std::string euro = "\xE2\x82\xAC"; // U+20AC
+    const std::string o_grave = "\xC3\xB2";  // U+00F2
+    const std::string full = sharp_s + e_acute + euro + o_grave;
+
+    const std::string view = client_view(tracker,
+                                         {sharp_s.substr(0, 1), sharp_s.substr(1),
+                                          e_acute.substr(0, 1),
+                                          e_acute.substr(1) + euro,
+                                          o_grave.substr(0, 1), o_grave.substr(1)},
+                                         &emitted);
+
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "repeated-lead fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check_eq(view, full, "no character is lost to a repeated lead byte");
+    check_eq(tracker.reconcile(full), "", "the final text adds nothing");
+}
+
+// The same shape one layer down, as a backstop: a fragment that BEGINS with
+// orphan continuation bytes must never go on the wire, whatever produced it.
+// Dropping bytes keeps the stream alive; emitting them ends it, and takes the
+// final_result that was still to come with it.
+static void test_a_fragment_never_begins_mid_character() {
+    TranscriptDeltaTracker tracker;
+    const std::string first = tracker.observe(std::string("\xA9") + "rest");
+    check(is_valid_utf8(first), "a leading orphan continuation byte is not emitted");
+    check_eq(first, "rest", "the rest of the fragment still goes out");
+
+    TranscriptDeltaTracker all_orphans;
+    check_eq(all_orphans.observe(std::string("\x82\xAC")), "",
+             "a fragment that is nothing but orphans emits nothing");
+    check_eq(all_orphans.assembled(), "",
+             "and nothing is recorded as delivered");
+    check_eq(all_orphans.observe("after"), "after", "the stream continues");
+}
+
+// PURE ASCII, no multi-byte character anywhere, and the transcript still comes
+// out wrong: a short incremental fragment that happens to be a byte prefix of
+// everything known was read as an already-delivered repeat and discarded.
+//
+// This is the shape both incremental families produce. nemotron_asr emits
+// current_text.substr(emitted_text.size()) per non-blank token (decoder.cpp:550)
+// and vibevoice_asr emits text.substr(common_prefix_size(...)) (session.cpp:89-105),
+// so a one-character fragment is ordinary output, and any of the transcript's
+// own leading characters will eventually arrive as one.
+//
+// Measured over 5,000 randomized traces per transcript before this was fixed:
+// 9.50% of pure-ASCII traces and 29.12% of French ones ended with the client
+// holding something other than final_result.text, with a 200 and no diagnostic.
+static void test_a_short_fragment_is_not_mistaken_for_a_repeat() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string full = "pure ascii transcript";
+    const std::string view = client_view(
+        tracker, {"pure ", "ascii ", "trans", "c", "ri", "p", "t"}, &emitted);
+
+    check_eq(view, full, "the whole transcript reaches the client");
+    check_eq(tracker.assembled(), full, "and the assembly agrees with it");
+    check_eq(tracker.reconcile(full), "",
+             "the final text adds nothing, because nothing was lost");
+    check(emitted.size() == 7, "every fragment produced exactly one delta");
+}
+
+int main() {
+    test_incremental_family();
+    test_cumulative_family_with_duplicate_delivery();
+    test_an_exact_repeat_is_never_resent();
+    test_a_shrinking_hypothesis_is_read_as_incremental();
+    test_a_short_fragment_is_not_mistaken_for_a_repeat();
+    test_empty_partials_are_ignored();
+    test_offline_fallback_is_one_delta();
+    test_reconcile_emits_the_tail();
+    test_divergent_final_text_is_not_appended();
+    test_empty_final_text();
+    test_whitespace_fragments_survive();
+    test_prefix_extension_is_read_as_cumulative();
+    test_cumulative_split_multibyte_character();
+    test_incremental_split_multibyte_character();
+    test_four_byte_character_split_three_ways();
+    test_reconcile_flushes_a_held_back_sequence();
+    test_held_bytes_join_the_next_fragment();
+    test_a_complete_character_is_not_held();
+    test_undecodable_bytes_are_not_held_forever();
+    test_a_repeated_lead_byte_is_not_swallowed();
+    test_a_fragment_never_begins_mid_character();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all stream_delta checks passed\n");
+    return 0;
+}