fix(whisper-medusa): pin available portable Python

The standalone Python release used by libbackend does not publish a 3.11.18 artifact. Pin Whisper-Medusa to the available 3.11.13 build and cover the generated download URL.

Assisted-by: Codex:gpt-5 [systematic-debugging]

.agents/ci-caching.md

@@ -122,18 +122,89 @@ The per-backend prefix match only sees files under a backend's own directory, so
 
 | Changed path | Rebuilds |
 |---|---|
-| `backend/backend.proto` | everything (all languages compile or copy it) |
+| `backend/backend.proto` | nothing if the edit is additive-only, otherwise everything (see below) |
 | `backend/Dockerfile.<x>` | the Linux entries whose `dockerfile:` names it |
 | `backend/python/common/` | Python, Linux + Darwin |
-| `scripts/build/package-gpu-libs.sh` | Python, Linux only |
+| `scripts/build/package-gpu-libs.sh` | every Linux entry (Python, Go and C++ all run it) |
 | `scripts/build/<lang>-darwin.sh` | the Darwin entries that build target routes to |
 | `.github/workflows/backend_build[_darwin].yml` | everything on that OS |
 | anything else under `scripts/build/` (except `*_test.sh`) | everything — conservative default for unclassified packaging inputs |
 
 Deliberately excluded: `backend/index.yaml` (gallery metadata, never enters an image), `.github/backend-matrix.yml` (adding a backend would rebuild all of them), `backend/Dockerfile.base-grpc-builder` (owned by `base-images.yml`), and the root `Makefile` (touched in ~11% of commits, and its backend-relevant edits arrive alongside the backend directory anyway). `make test-ci-scripts` pins all of this.
 
+#### `backend/backend.proto` is content-filtered, not path-filtered
+
+Every language consumes the proto, so a path rule for it can only ever say "rebuild all 473 images". It changes in ~1.3% of commits, and that was enough to make it the single largest CI cost driver in the repo: on 2026-07-29 four runs totalling 935 queued jobs traced to nothing but a proto edit, one of which (#11158) was a six-line diff adding `bool cache_prompt = 8;`.
+
+An additive proto edit cannot change how a backend that never references the new symbol behaves, so `filterMatrix()` suppresses the rule for one. `changed-backends.js` fetches `backend/backend.proto` at the base revision (same contents-API pattern as `.github/backend-matrix.yml`) and hands both texts to `protoChangeIsAdditive()`, which compares them structurally rather than textually:
+
+- **Additive, rebuilds nothing**: a new field with an unused number, a new message, a new enum value, a new RPC. Comment, whitespace and ordering changes also land here.
+- **Breaking, rebuilds everything**: a removed, renumbered, retyped or renamed field, a dropped RPC, a changed `option` or `package`. So does an unresolvable base revision, matching the run-all posture used for a truncated diff.
+
+Checked against every proto commit in the preceding six months, all nine resolvable ones classify as additive. Note the tradeoff this accepts: generated stubs do change for an additive edit, so image bytes would differ on a rebuild even though behavior does not. That is the same standard already applied when the filter declines to rebuild on unrelated `pkg/` changes, and the weekly cron remains the backstop.
+
 The Sunday 06:00 UTC cron on `backend.yml` exists specifically because path filtering can leave Python backends frozen on stale wheels. `DEPS_REFRESH` (below) only fires when the build actually runs, so an untouched Python backend would never re-resolve its unpinned deps. The weekly cron is the safety net.
 
+## Content-blind PRs skip the workflows that cannot see them
+
+`backend_pr.yml` and `test-extra.yml` filter themselves (matrix generation and a `detect-changes` job), so a gallery-only or docs-only PR costs them about one job each. The Go and image workflows had no filter of any kind, so a one-line `gallery/index.yaml` edit queued 20 jobs, and a docs-only PR queued the same.
+
+This is worth more than it looks. Measured over the week to 2026-07-30, **97% of CI wall-clock is queueing, 3% is execution** (median queue ~5h against a 4-20min median job). Cutting job count is therefore the only lever that shortens feedback time; making individual jobs faster moves 3%.
+
+The volume is real: 13 gallery-only PRs merged that week with 10 open at once, and 78 of the 137 PRs opened were bot-generated.
+
+`paths-ignore` on the PR trigger of `image-pr.yml` (7 jobs), `build-test.yaml` (3), `lint.yml` (2) and `tests-e2e.yml` (1) drops 13 of those 20. The excluded set:
+
+| Path | Why no image or Go build can see it |
+|---|---|
+| `gallery/**` | Model-gallery metadata, parsed at runtime, never copied into an image |
+| `docs/**`, `examples/**`, `**/*.md` | Never enter an image or a binary. `lint.yml` already excluded these before gallery was added |
+
+### `backend/{cpp,go,python}/**` on `image-pr.yml` and `build-test.yaml` only
+
+Version-pin bumps dominate PR volume: 48 `update/*` PRs in the week to 2026-07-30, from 16 pins, each a two-line diff. Most edit nothing but one `backend/*/<name>/Makefile`.
+
+Neither of those two workflows can observe such a change. `make build` is `go build ./cmd/local-ai`, GoReleaser builds the same plus `./cmd/launcher`, and the core image's final stage ships only `entrypoint.sh`, `healthcheck.sh` and that binary. The per-backend trees are copied into the builder but nothing in them reaches the output.
+
+What still triggers a full run, because none of it lives under those prefixes:
+
+- `backend/backend.proto` — feeds `protogen-go`, so it does change the binary.
+- `go.mod` / `go.sum` — the `go mod tidy` before-hook.
+- `backend/Dockerfile.*` and anything else directly under `backend/`.
+
+Deliberately **not** applied to:
+
+| Workflow | Why it must keep seeing `backend/**` |
+|---|---|
+| `test.yml` | `TEST_PATHS` explicitly includes `./backend/go/cloud-proxy/...`, `./backend/go/local-store/...` and `./backend/go/valkey-store/...` |
+| `lint.yml` | `.golangci.yml` carries `backend/`-scoped rules, so golangci-lint covers that tree |
+| `tests-e2e.yml` | The e2e suite drives real backends over gRPC |
+| `backend_pr.yml` | This is the workflow whose entire job is to rebuild the changed backend |
+
+What still runs, and why it has to:
+
+| Workflow | Why it keeps running |
+|---|---|
+| `test.yml` (`tests`) | `core/gallery/variants_lint_test.go` reads the real `gallery/index.yaml` and asserts the index invariants (no duplicate entry names, no build claimed by two parents). This is the only schema-level check the gallery has. |
+| `yaml-check.yml` (`Yamllint`) | Lints `gallery/` for syntax. |
+| `backend_pr.yml`, `test-extra.yml` | Already self-filtering; they stop after the detect step. |
+
+Two properties this relies on:
+
+- `paths-ignore` skips a run only when **every** changed file matches, so a PR touching the gallery *and* Go code still runs everything. That is what makes the exclusion safe rather than a hole.
+- `master` carries no branch protection and no rulesets, so a skipped workflow reports no status and nothing waits on it. If required status checks are ever introduced, these four entries must be excluded from the required set or PRs will hang on "Expected — Waiting for status to be reported".
+
+### `image.yml` on master push is gated too, by a job rather than a path filter
+
+The same reasoning applies to master pushes, and the volume is larger there: on 2026-07-30, **12 of the 23 queued `image.yml` runs** were commits like "add 1 new model to gallery" or a docs fix, each rebuilding all 18 container images.
+
+`image.yml` now has a `changes` job that decides once whether the push can affect any image; the other 11 jobs carry `needs: changes` plus an `if:` on its output. Verified against the shipped `Dockerfile`: the final stage copies only `entrypoint.sh`, `healthcheck.sh` and the `local-ai` binary, there is no `go:embed` of `gallery/` or `docs/`, and the gallery is fetched at runtime from `github:mudler/LocalAI/gallery/index.yaml@master`. A gallery-only commit therefore produces byte-identical images, and the gallery change reaches users through GitHub immediately whether or not an image is rebuilt.
+
+Two properties to preserve if you touch it:
+
+- **It is a job gate, not `paths-ignore`.** `paths-ignore` on `push` also applies to tag pushes, and a tag created on an existing commit carries an empty commits list, which would silently skip the release image build. The gate short-circuits to "build" for `refs/tags/*`, and for any push whose base commit is missing, zero, or unresolvable.
+- **The merge jobs must name the gate explicitly.** They use `if: ${{ !cancelled() && ... }}`, and `!cancelled()` is true when a dependency is *skipped*, so without the extra condition they would run and try to merge manifest lists for images that were never built.
+
 ## The `DEPS_REFRESH` cache-buster (Python backends)
 
 Every Python backend goes through the shared `backend/Dockerfile.python`, which ends with:
@@ -169,15 +240,38 @@ RUN --mount=type=cache,target=/root/.ccache,id=<backend>-ccache-${TARGETARCH}-${
     bash /usr/local/sbin/compile.sh
 ```
 
-The compile script exports `CMAKE_C/CXX/CUDA_COMPILER_LAUNCHER=ccache` so CMake threads ccache through gcc/g++/nvcc. `cache-to: type=registry,mode=max` exports the cache mount data into the registry cache, so subsequent builds restore it.
+The compile script exports `CMAKE_C/CXX/CUDA_COMPILER_LAUNCHER=ccache` so CMake threads ccache through gcc/g++/nvcc. Cache scope is per `(TARGETARCH, BUILD_TYPE)` so e.g. cublas-12 doesn't share with cublas-13 (their CUDA headers differ; cross-pollination would just be cache misses anyway).
 
-On a `LLAMA_VERSION` bump, most translation units are byte-identical to the previous version's preprocessed source — ccache returns the previous `.o` and skips the real compile. Same for LocalAI source changes that don't actually touch llama.cpp's CMake inputs. Cache scope is per `(TARGETARCH, BUILD_TYPE)` so e.g. cublas-12 doesn't share with cublas-13 (their CUDA headers differ; cross-pollination would just be cache misses anyway).
+### ⚠️ This ccache does nothing in CI today
+
+This section previously claimed that `cache-to: type=registry,mode=max` "exports the cache mount data into the registry cache, so subsequent builds restore it". **That is not true.** BuildKit does not export the contents of a `--mount=type=cache` to a registry cache export. A cache mount lives in the builder's local state, and every CI job gets a fresh runner with a fresh builder, so `/root/.ccache` starts empty on every single build.
+
+Measured on 2026-07-30 from the `ccache -s` output the compile script already prints (it runs `ccache -z` first, so the numbers are per-build):
+
+| Job | Commit touched | Build time | ccache |
+|---|---|---|---|
+| 89766266951 (llama-cpp, cublas 13) | `backend/go/magpie-tts-cpp/Makefile` only | 6369s | **0 / 889 hits**, and 0 / 1778 |
+| 89766267281 (llama-cpp, hipblas) | same commit | 8160s | **0 / 537 hits** |
+| 90210828110 (llama-cpp, cublas 12.8) | `LLAMA_VERSION` bump | 5673s | **0 / 813 hits** |
+
+The first two are the decisive control: commit `90355cd44` changed exactly one file, `backend/go/magpie-tts-cpp/Makefile`, nowhere near llama.cpp. The engine source was byte-identical to the previous build, which is precisely the case this section says ccache should serve, and the hit rate was still **0.00%**. A cache that was being restored but merely matching poorly would show partial hits; 0-of-N is the signature of an empty cache.
+
+So the paragraph above about `LLAMA_VERSION` bumps reusing previous `.o` files describes an intended design that is not in effect. `Dockerfile.{llama-cpp,ik-llama-cpp,turboquant,bonsai,ds4,privacy-filter}` pay the ccache wrapper overhead and get nothing back. Multi-hour C++ rebuilds are recompiling identical translation units from scratch.
+
+**Do not "fix" this by adding cache mounts to more Dockerfiles.** Wiring the same mount into `Dockerfile.golang` (215 of the 434 matrix entries) was measured locally at 18% faster on a rebuild after a source edit, with a 71.5% ccache hit rate — but only because the local test reused one builder across both builds. In CI it would be a no-op for exactly the reason above.
+
+Making this actually work needs the cache to live outside the builder. The options, none of them free:
+
+- **ccache `remote_storage`** (ccache ≥ 4.4, HTTP or Redis backend) or **sccache** with an S3/GCS/Redis backend. Genuinely works across runners; needs a cache service to point at. quay.io is a registry, not a blob store, so the existing infra does not cover it.
+- **Round-trip the cache dir through `actions/cache` on the runner**: restore it, pass it in, and export it back out via a build stage output. No external infra, but clunky, and the repo already sits at GitHub's 10 GB cache ceiling while the llama-cpp ccache alone is capped at 5 GB.
+
+Until one of those lands, treat C++ backend builds as always-cold and spend the effort on not running them instead (path filtering, see above).
 
 ## Composite actions
 
 Two composite actions handle runner-side prep:
 
-- **`.github/actions/free-disk-space/action.yml`** — wraps `jlumbroso/free-disk-space@main` plus an explicit apt purge of dotnet/android/ghc/mono/etc. Reclaims ~6–10 GB on `ubuntu-latest`. No-op on self-hosted runners. Used by `backend_build.yml`, `image_build.yml`, `test.yml`, `tests-aio.yml`, etc.
+- **`.github/actions/free-disk-space/action.yml`** — wraps `jlumbroso/free-disk-space@main` plus an explicit apt purge of dotnet/android/ghc/mono/etc. Reclaims ~6–10 GB on `ubuntu-latest`. No-op on self-hosted runners. Used by `backend_build.yml`, `image_build.yml` and `base-images.yml` — the jobs that actually build images. Deliberately **not** used by `test.yml`, which runs no buildx step.
 - **`.github/actions/setup-build-disk/action.yml`** — relocates Docker's data-root to `/mnt` on hosted X64 runners. GHA hosted `ubuntu-latest` ships ~75 GB of unused space at `/mnt`; combined with the free-disk-space cleanup this gives ~100 GB working space — enough for ROCm dev image + vLLM torch install + flash-attn intermediate layers. No-op on self-hosted and on non-X64 hosted runners. Used by `backend_build.yml`, `image_build.yml`, `base-images.yml`.
 
 Both actions run before any docker buildx step.
@@ -218,10 +312,20 @@ Eviction is rarely needed in normal operation — `DEPS_REFRESH` handles weekly
 
 ## What the cache does **not** cover
 
-- The `free-disk-space` and `setup-build-disk` composite actions run on every job — these reclaim runner-state, not Docker layers, so BuildKit caches don't apply.
+- The `free-disk-space` and `setup-build-disk` composite actions run on every job — these reclaim runner-state, not Docker layers, so BuildKit caches don't apply. `test.yml` deliberately does **not** use `free-disk-space`: it runs no buildx step, and the multi-GB fixture downloads that once justified it left `make test` in the test-suite reorg.
 - Intermediate artifacts of `Build (PR)` are not pushed anywhere — PRs only build for verification.
 - Darwin builds (see below) — macOS runners have no Docker daemon, so the registry-backed BuildKit cache cannot apply.
 
+### The Linux Go workflows set `cache: false` on purpose
+
+`test.yml`, `lint.yml`, `tests-e2e.yml` and friends pass `cache: false` to `actions/setup-go@v5`, unlike the darwin jobs. This looks like an oversight and is not.
+
+Measured over the week to 2026-07-30, the `Set up Go` step has a **median of 11 seconds** on these runners. There is essentially nothing to win: the module download is not where the time goes. The expensive steps are compilation and test execution (`Test (with coverage gate)` at ~18.6min, `Test Backend E2E` at ~14.5min), and Go's build cache would have to survive across runners to touch those.
+
+Enabling it also has a real cost. GitHub caps Actions cache at **10 GB per repo and the repo already sits at that ceiling** (31 entries), so every `setup-go` entry written by a branch with a distinct `go.sum` (222-375 MB on Linux, up to 1.4 GB on macOS) evicts something else. See the darwin cache budget below.
+
+Before re-enabling this, measure `Set up Go` again and confirm it has actually become slow. If room is needed in the 10 GB budget, the cheapest evictions are the `docker.io--tonistiigi--binfmt` entries (~30 MB each, trivially re-fetched).
+
 ## Darwin native caches
 
 `backend_build_darwin.yml` runs natively on `macOS-14` GitHub-hosted runners — there is no Docker, no BuildKit, no cross-job registry cache. Instead, the reusable workflow uses `actions/cache@v4` for four native caches that mirror the spirit of the Linux cache (warm by default, weekly refresh for unpinned Python deps, PRs read-only).
@@ -255,6 +359,26 @@ GitHub Actions caches are limited to 10 GB per repo. Steady-state worst case: ~8
 
 One residual self-hosted reference remains in `test-extra.yml` (`tests-vibevoice-cpp-grpc-transcription` uses `bigger-runner` for the 30s JFK-decode timeout headroom). That's a separate concern.
 
+### Small always-on jobs routed to `arc-runner-set`
+
+The hosted pool is shared across the whole *account*, not per repo, so a burst in one repo starves the others. On 2026-07-31 it went to **zero scheduled jobs for 35 consecutive minutes** with 39 jobs queued, while `arc-runner-set` completed 12 jobs without interruption over the same window. Actions was healthy globally at the time (other public repos were scheduling normally), so this is an account-level throttle, not an outage.
+
+`gh-pages.yml` (`build` + `deploy`) is therefore routed to `arc-runner-set` when `github.repository == 'mudler/LocalAI'`. It needs no fork-safety clause because it only triggers on push-to-master and `workflow_dispatch`, so it never executes pull-request code. The repository guard keeps forks (which have no such runner label) from queueing forever. It fetches its own toolchains via `setup-go` / `actions-hugo` and uses no `sudo`/`apt`.
+
+#### What the `arc-runner-set` image actually contains
+
+Measured 2026-07-31 on run `30637392862` by a preflight step, not assumed:
+
+| present | **absent** |
+|---|---|
+| `git`, `curl`, `unzip`, `tar`, `ldd`, `python3` | **`make`**, **`gcc`** |
+
+That is why `lint.yml` is **not** on the self-hosted pool. Both of its jobs were routed there and both failed in one second: `golangci-lint` needs `make` (for `make protogen-go`, itself needing `curl`+`unzip` to fetch protoc, and for `make lint`), and `build-scripts` additionally needs a C toolchain because the packaging-script tests compile a throwaway binary and inspect it with `ldd`. Both jobs are back on `ubuntu-latest`.
+
+The preflight steps were deliberately left in place. They cost about a second on the hosted pool and mean that whenever the runner image gains `make` + `gcc`, re-routing is one `runs-on:` line per job and any remaining gap reports itself by name rather than as an opaque mid-build failure.
+
+Note for any future re-route: `lint.yml` also triggers on `pull_request`, and a fork PR runs untrusted contributor code. That must never reach a persistent self-hosted runner, so any re-route has to stay push-only, e.g. `${{ (github.event_name == 'push' && github.repository == 'mudler/LocalAI') && 'arc-runner-set' || 'ubuntu-latest' }}`.
+
 ## Touching the cache pipeline
 
 When changing `image_build.yml`, `backend_build.yml`, any of the `backend/Dockerfile.*` files, `Dockerfile.base-grpc-builder`, `.docker/install-base-deps.sh`, `.docker/<backend>-compile.sh`, or `scripts/changed-backends.js`:

.agents/coding-style.md

@@ -70,3 +70,37 @@ The project documentation is located in `docs/content`. When adding new features
 - **Configuration**: If you modify configuration options, update the relevant sections in `docs/content/`.
 - **Examples**: providing concrete examples (like YAML configuration blocks) is highly encouraged to help users get started quickly.
 - **Shortcodes**: Use `{{% notice note %}}`, `{{% notice tip %}}`, or `{{% notice warning %}}` for callout boxes. Do **not** use `{{% alert %}}` — that shortcode does not exist in this project's Hugo theme and will break the docs build.
+
+## React UI styling
+
+The React UI ships a design system in `core/http/react-ui/src/App.css`: design
+tokens, form grids, data tables, stat cards, callouts, plus a small semantic
+primitive layer (`.stack`, `.hstack`, `.text-note`, `.text-meta`, `.tone-*`,
+`.icon-chip`). **Use it instead of `style={{ ... }}`.** Inline styles are a
+spacing or colour decision made in one file, so no two pages end up sharing a
+rhythm, which is the main reason the app reads as unfinished.
+
+Inline styles are still correct for values that are genuinely computed at
+runtime: `width: ${pct}%`, a data-driven `background`, a tooltip's coordinates.
+Everything else belongs in a class.
+
+A ratchet enforces this:
+
+```sh
+cd core/http/react-ui
+npm run lint:inline-styles          # fails if the count went UP
+npm run lint:inline-styles:report   # per-file counts, worst first
+npm run lint:inline-styles:write    # refresh the baseline after converting
+```
+
+The gate also fails on **duplicate `className` attributes on one element**. JSX
+keeps the last and silently drops the first, so `<i className={icon}
+className="text-xs" />` loses its icon while passing lint, the build and the e2e
+suite. Converting a style to a class on an element that already has a
+`className` is the usual way to introduce one; merge them into a single
+attribute instead.
+
+When converting a page, prefer naming the shapes it actually has
+(`.p2p-diagram`, `.usage-tile`) over adding more utilities, and check whether an
+existing block already covers it: the Nodes page reuses the P2P setup shapes,
+and Model Editor reuses the Settings section rail.

.agents/preparing-a-release.md

@@ -0,0 +1,26 @@
+# Preparing a Release
+
+A release is not finished when the tag is pushed. The GitHub release, the blog post and the demo clips ship together, because the changelog says what moved and the post and the clips are what make anyone care.
+
+## What a release must include
+
+1. **Labels on the merged PRs.** GitHub generates the raw notes from PR labels, so label first, generate second. Wrong labels mean a miscategorised changelog that has to be edited by hand.
+2. **`RELEASE_NOTES_vX.Y.Z.md`** at the repository root, in the house style: what changed, why it matters, PR numbers so people can read the diffs.
+3. **A blog post under `website/content/blog/`.** One post per release, front matter with `title`, `date`, `author`, `category: "Release"`, `tags`, `summary` and `extracss: ["blog.css"]`. Cover the two or three changes that alter what a user does day to day, not the whole changelog, and link the PR numbers. See `website/content/blog/what-landed-in-localai-4-8.md` for the shape.
+4. **Demo clips for the notable features.** Anything visible (a new backend, a UI change, a new endpoint, a measured speedup) gets a short screen recording. Put the file in `website/static/media/`, reference it from the blog post, and reuse it on the marketing pages where it fits.
+
+A release without a post and without clips is incomplete, in the same way a user-facing code change without a docs update is incomplete.
+
+## Clip conventions
+
+- MP4, H.264, no audio track unless the feature is about audio. Keep them short (10 to 30 seconds) and loopable.
+- Record the real thing. A clip from the engine's own benchmark suite or a real session, never a mockup.
+- Where the change is a speedup, record both sides on the same machine on the same input, so the comparison is honest.
+- Name the file after the feature, not the release (`vllm-race.mp4`, not `v4-8-demo.mp4`), so it stays reusable once the release is old.
+- The marketing site plays clips with `muted loop playsinline preload="none"` and a `data-lazy` attribute, which the site's IntersectionObserver uses to play and pause them on scroll. Follow that pattern for anything you add.
+
+## Order of work
+
+Label the PRs, generate and edit the release notes, cut the draft release, record the clips while the branch is still fresh in your head, then write the post against the notes and the clips. Publishing the release and merging the post should happen on the same day.
+
+The `creating-localai-releases` skill drives steps 1 to 3 and captures the React UI screenshots that go into the notes.

.agents/vllm-backend.md

@@ -21,6 +21,17 @@ options:
   - reasoning_parser:qwen3
 ```
 
+## `Options[]` doubles as CLI-style engine flags
+
+Beyond the parser names above, `Options[]` carries `--` prefixed engine flags (`--enable-prefix-caching`, `--kv-cache-dtype:fp8_e5m2`). `apply_options_to_engine_args` in `backend/python/common/vllm_utils.py` maps them onto `AsyncEngineArgs` fields, and it must run **before** `AsyncLLMEngine.from_engine_args()` - applying them afterwards is a silent no-op, which is exactly what issue #11130 was.
+
+Things to keep straight when touching this:
+
+- Precedence is typed proto fields → `options:` → `engine_args:`. `applyEngineArgDefaults` in `core/config/hooks_vllm.go` therefore skips seeding a production default whose key the user already set as an option, otherwise the later `engine_args:` pass would silently override them.
+- Only `--` prefixed entries are engine flags; `tool_parser:`/`reasoning_parser:` and friends keep their meaning. Parser lookups accept both spellings via `normalize_option_key`.
+- Unknown or uncoercible flags warn and are skipped, unlike `engine_args:` which is strict - `Options[]` is a shared bag and knows entries this mapping doesn't.
+- Field types come from the annotation's *base* (`Literal["auto","float16"]` is not a float). The helper's tests are stdlib-only: `make test-python-helpers`.
+
 Auto-defaults for known model families live in `core/config/parser_defaults.json` and are applied:
 - at gallery import time by `core/gallery/importers/vllm.go`
 - at model load time by the `vllm` / `vllm-omni` backend hook in `core/config/hooks_vllm.go`

.docker/install-base-deps.sh

@@ -113,6 +113,54 @@ if [ "${BUILD_TYPE:-}" = "vulkan" ] && [ "${SKIP_DRIVERS:-false}" = "false" ]; t
     rm -rf /var/lib/apt/lists/*
 fi
 
+# --- 2b. Intel graphics driver (BUILD_TYPE=sycl*) ---
+# The Intel oneAPI base image brings the compilers and the oneAPI libraries, but
+# not the driver that talks to the graphics card. The packaging step copies that
+# driver into the backend, so that the backend works on a machine which has no
+# Intel graphics packages of its own, for the same reason the Vulkan section
+# above installs the Mesa drivers. Install it here so there is something to copy.
+#
+# Only the sycl builds are covered, because those are the ones whose packaging
+# copies the driver. See package_intel_libs in scripts/build/package-gpu-libs.sh.
+#
+# The driver comes from Intel's own package repository, not from the Ubuntu
+# archive. The archive has 23.43 from late 2023, which does not know any card
+# released since, so a machine with a recent Intel GPU would end up carrying a
+# driver that cannot drive it. Intel's repository has 25.18 for the same Ubuntu
+# release.
+#
+# Anything that goes wrong here fails the build, on purpose. An unreachable
+# repository is a passing problem that a retry fixes, whereas carrying a
+# different driver than intended, or none, is a difference nobody would notice
+# until a user reports an idle GPU.
+if case "${BUILD_TYPE:-}" in sycl*) true;; *) false;; esac \
+    && [ "${SKIP_DRIVERS:-false}" = "false" ]; then
+    # Ubuntu release name, which is what the repository is indexed by.
+    ubuntu_codename=$(. /etc/os-release && echo "${VERSION_CODENAME:-}")
+    if [ -z "$ubuntu_codename" ]; then
+        echo "ERROR: cannot tell which Ubuntu release this image is, so cannot pick the Intel driver repository" >&2
+        exit 1
+    fi
+
+    # The key is armored text, which apt reads directly from a .asc file, so
+    # there is no need for gnupg here. "unified" is the component Intel ships
+    # its current driver in.
+    mkdir -p /usr/share/keyrings
+    curl -fsSL https://repositories.intel.com/gpu/intel-graphics.key \
+        -o /usr/share/keyrings/intel-graphics.asc
+    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.asc] https://repositories.intel.com/gpu/ubuntu ${ubuntu_codename} unified" \
+        > /etc/apt/sources.list.d/intel-graphics.list
+    apt-get update
+    # The first package holds the driver OpenCL talks to, the second the driver
+    # Level Zero talks to. Between them they pull in the compiler and the memory
+    # manager that both need.
+    apt-get install -y --no-install-recommends \
+        intel-opencl-icd \
+        libze-intel-gpu1
+    apt-get clean
+    rm -rf /var/lib/apt/lists/*
+fi
+
 # --- 3. CUDA toolkit (BUILD_TYPE=cublas|l4t) ---
 if { [ "${BUILD_TYPE:-}" = "cublas" ] || [ "${BUILD_TYPE:-}" = "l4t" ]; } && [ "${SKIP_DRIVERS:-false}" = "false" ]; then
     apt-get update

.docker/llama-cpp-build-target.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+arch=${1:?target architecture is required}
+build_type=${2-}
+
+# GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
+# to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
+# builder images can supply that compiler.
+if [ "$arch" = "arm64" ] && [ -n "$build_type" ]; then
+  echo llama-cpp-fallback
+else
+  echo llama-cpp-cpu-all
+fi

.docker/llama-cpp-compile.sh

@@ -18,10 +18,12 @@ if [[ -n "${CUDA_DOCKER_ARCH:-}" ]]; then
 fi
 
 cd /LocalAI/backend/cpp/llama-cpp
-if [ -z "${BUILD_TYPE:-}" ]; then
-  # Pure CPU image (BUILD_TYPE empty): one build with ggml CPU_ALL_VARIANTS replaces the
-  # per-microarch binaries (x86: avx/avx2/avx512/fallback; arm64: armv8.x/armv9.x). ggml
-  # dlopens the best libggml-cpu-*.so at runtime by probing host CPU features.
+BUILD_TARGET=$(/LocalAI/.docker/llama-cpp-build-target.sh "${TARGETARCH}" "${BUILD_TYPE:-}")
+if [ "$BUILD_TARGET" = "llama-cpp-cpu-all" ]; then
+  # One build with ggml CPU_ALL_VARIANTS replaces the per-microarch binaries (x86:
+  # avx/avx2/avx512/fallback; arm64: armv8.x/armv9.x). BUILD_TYPE remains in the
+  # environment, so GPU builds retain their accelerator backend while ggml dlopens the
+  # best CPU library when work is offloaded to the host.
   #
   # arm64: the CPU_ALL_VARIANTS table includes armv9.2 SME variants whose -march=...+sme is
   # rejected by the Ubuntu 24.04 default gcc-13. gcc-14 accepts it, so build the arm64
@@ -35,14 +37,8 @@ if [ -z "${BUILD_TYPE:-}" ]; then
     apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
     export CC=gcc-14 CXX=g++-14
   fi
-  make llama-cpp-cpu-all
-else
-  # GPU build (cublas/hipblas/sycl/vulkan/...): the accelerator does the compute, so a
-  # single fallback CPU build is enough - no per-microarch CPU variants needed. (This also
-  # keeps the heavy GPU backend compile from also building the whole CPU variant matrix,
-  # and avoids the gcc-14 apt step on GPU base images such as nvidia l4t.)
-  make llama-cpp-fallback
 fi
+make "$BUILD_TARGET"
 make llama-cpp-grpc
 make llama-cpp-rpc-server
 

.docker/turboquant-build-target.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+arch=${1:?target architecture is required}
+build_type=${2-}
+
+# GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
+# to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
+# builder images can supply that compiler.
+if [ "$arch" = "arm64" ] && [ -n "$build_type" ]; then
+  echo turboquant-fallback
+else
+  echo turboquant-cpu-all
+fi

.docker/turboquant-compile.sh

@@ -19,20 +19,18 @@ fi
 
 cd /LocalAI/backend/cpp/turboquant
 
-if [ -z "${BUILD_TYPE:-}" ]; then
-  # Pure CPU image: one ggml CPU_ALL_VARIANTS build replaces the per-microarch binaries.
+BUILD_TARGET=$(/LocalAI/.docker/turboquant-build-target.sh "${TARGETARCH}" "${BUILD_TYPE:-}")
+if [ "$BUILD_TARGET" = "turboquant-cpu-all" ]; then
+  # BUILD_TYPE remains in the environment, so GPU builds retain their accelerator while
+  # ggml selects the best CPU library when model work is offloaded to the host.
   # arm64: the armv9.2 SME variants need gcc-14 (gcc-13 rejects +sme).
   if [ "${TARGETARCH}" = "arm64" ]; then
+    sh /LocalAI/.docker/apt-mirror.sh || true
     apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
     export CC=gcc-14 CXX=g++-14
   fi
-  make turboquant-cpu-all
-else
-  # GPU build (cublas/hipblas/sycl/vulkan/...): single fallback CPU build, the accelerator
-  # does the compute. Keeps the GPU compile from also building the CPU variant matrix and
-  # avoids the gcc-14 apt step on GPU base images such as nvidia l4t.
-  make turboquant-fallback
 fi
+make "$BUILD_TARGET"
 make turboquant-grpc
 make turboquant-rpc-server
 

.dockerignore

@@ -40,6 +40,16 @@ backend/cpp/privacy-filter/build
 backend/cpp/privacy-filter/grpc-server
 backend/cpp/privacy-filter/package
 
+# audio-cpp: same in-place pattern. The Makefile clones audio.cpp at the pinned
+# AUDIO_CPP_VERSION and the `audio.cpp:` target is the directory itself, so a
+# stale host checkout COPY'd in makes the build compile against whatever commit
+# the host had. build/ is worse than stale: its CMakeCache.txt records the host
+# source, prefix and compiler paths, and cmake refuses to reconfigure from it.
+backend/cpp/audio-cpp/audio.cpp
+backend/cpp/audio-cpp/build
+backend/cpp/audio-cpp/grpc-server
+backend/cpp/audio-cpp/package
+
 # Rust backend build output (sources are tracked; target/ is generated)
 backend/rust/*/target
 

.github/backend-matrix.yml

@@ -166,6 +166,19 @@ include:
     dockerfile: "./backend/Dockerfile.python"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-whisper-medusa'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "whisper-medusa"
+    dockerfile: "./backend/Dockerfile.python"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -626,6 +639,19 @@ include:
     dockerfile: "./backend/Dockerfile.python"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "1"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-12-whisper-medusa'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "whisper-medusa"
+    dockerfile: "./backend/Dockerfile.python"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "8"
@@ -756,6 +782,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "8"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-12-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "8"
@@ -1716,6 +1755,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-13-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "13"
     cuda-minor-version: "0"
@@ -1729,6 +1781,19 @@ include:
     backend: "stablediffusion-ggml"
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/arm64'
+    skip-drivers: 'false'
+    tag-latest: 'auto'
+    tag-suffix: '-nvidia-l4t-cuda-13-arm64-trellis2cpp'
+    base-image: "ubuntu:24.04"
+    ubuntu-version: '2404'
+    runs-on: 'ubuntu-24.04-arm'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
   - build-type: 'cublas'
     cuda-major-version: "13"
     cuda-minor-version: "0"
@@ -3104,6 +3169,97 @@ include:
     dockerfile: "./backend/Dockerfile.privacy-filter"
     context: "./"
     ubuntu-version: '2404'
+  # audio-cpp: 0xShug0/audio.cpp, a multi-family ggml audio engine (TTS, ASR,
+  # VAD, diarization, source separation, music generation).
+  #
+  # These entries deliberately carry NO builder-base-image, unlike the
+  # privacy-filter and llama-cpp blocks above. The prebuilt
+  # quay.io/go-skynet/ci-cache:base-grpc-* images ship a from-source gRPC whose
+  # protobuf is v26, and protobuf has depended on abseil since v22. audio.cpp
+  # links sentencepiece with SPM_PROTOBUF_PROVIDER=package (needed to stop
+  # sentencepiece's vendored protobuf 3.14 from colliding with the 3.21 the
+  # generated backend.pb.cc is built against, which broke every nested-message
+  # parse), so sentencepiece then sees real abseil's
+  # `absl::lts_20240116::internal` alongside its own vendored plain
+  # `absl::internal` and every `absl::internal::` reference becomes ambiguous.
+  # Verified, not theorised: building against base-grpc-amd64 fails at
+  # sentencepiece-static.dir/error.cc.o with "reference to 'internal' is
+  # ambiguous". Dockerfile.audio-cpp therefore installs Ubuntu Noble's apt
+  # gRPC/protobuf 3.21.12 itself and has a single `builder` stage, so the
+  # BUILDER_BASE_IMAGE / BUILDER_TARGET / SKIP_DRIVERS build-args are never
+  # consumed. Same reason CUDA needs its toolkit in base-image rather than in a
+  # builder image: this is the ds4 shape, not the llama-cpp one.
+  #
+  # No ROCm entry: upstream has no HIP configuration. No CUDA arm64 or L4T
+  # entry: upstream documents and validates CUDA on x86 only. Darwin/Metal is in
+  # the includeDarwin matrix below, built by scripts/build/audio-cpp-darwin.sh.
+  #
+  # No vulkan entry either, though Dockerfile.audio-cpp and the backend Makefile
+  # both handle BUILD_TYPE=vulkan for local builds. Every other vulkan backend
+  # gets its Mesa ICD drivers from .docker/install-base-deps.sh, which installs
+  # mesa-vulkan-drivers so package-gpu-libs.sh can bundle them; this Dockerfile
+  # calls neither, so the image would ship a Vulkan loader that finds no GPU. No
+  # CI job runs a vulkan image against real hardware, so it would pass green and
+  # fail in users' hands. The entry comes back once the ICD question is settled.
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-audio-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-audio-cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
+  # cuda-major-version is forwarded into the build (Dockerfile.audio-cpp -> the
+  # backend Makefile) and picks the CMAKE_CUDA_ARCHITECTURES list, which upstream
+  # otherwise sets to `native` and no CI runner can enumerate. cuda-minor-version
+  # and the base-image tag encode the same toolkit and must move together;
+  # nothing checks that for you.
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "8"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-12-audio-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "nvidia/cuda:12.8.1-devel-ubuntu24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "13"
+    cuda-minor-version: "0"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-13-audio-cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "nvidia/cuda:13.0.0-devel-ubuntu24.04"
+    skip-drivers: 'true'
+    backend: "audio-cpp"
+    dockerfile: "./backend/Dockerfile.audio-cpp"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -3267,6 +3423,35 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  # trellis2cpp
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-trellis2cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   # sam3-cpp
   - build-type: ''
     cuda-major-version: ""
@@ -3592,6 +3777,34 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'vulkan'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-vulkan-trellis2cpp'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: 'vulkan'
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-vulkan-trellis2cpp'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "0"
@@ -3605,6 +3818,19 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2204'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "0"
+    platforms: 'linux/arm64'
+    skip-drivers: 'false'
+    tag-latest: 'auto'
+    tag-suffix: '-nvidia-l4t-arm64-trellis2cpp'
+    base-image: "nvcr.io/nvidia/l4t-jetpack:r36.4.0"
+    runs-on: 'ubuntu-24.04-arm'
+    backend: "trellis2cpp"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2204'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "0"
@@ -5436,6 +5662,35 @@ include:
     dockerfile: "./backend/Dockerfile.golang"
     context: "./"
     ubuntu-version: '2404'
+  # valkey-store
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    platform-tag: 'amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-valkey-store'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "valkey-store"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/arm64'
+    platform-tag: 'arm64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-valkey-store'
+    runs-on: 'ubuntu-24.04-arm'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "valkey-store"
+    dockerfile: "./backend/Dockerfile.golang"
+    context: "./"
+    ubuntu-version: '2404'
   # rfdetr
   - build-type: ''
     cuda-major-version: ""
@@ -5977,6 +6232,10 @@ includeDarwin:
     tag-suffix: "-metal-darwin-arm64-stablediffusion-ggml"
     build-type: "metal"
     lang: "go"
+  - backend: "trellis2cpp"
+    tag-suffix: "-metal-darwin-arm64-trellis2cpp"
+    build-type: "metal"
+    lang: "go"
   - backend: "whisper"
     tag-suffix: "-metal-darwin-arm64-whisper"
     build-type: "metal"
@@ -6058,6 +6317,18 @@ includeDarwin:
   - backend: "privacy-filter"
     tag-suffix: "-metal-darwin-arm64-privacy-filter"
     lang: "go"
+  # audio-cpp is the same shape: a C++/ggml backend built by a bespoke darwin
+  # script (make backends/audio-cpp-darwin), which reuses the backend's own
+  # package.sh so the Darwin package keeps the root-level layout the Linux image
+  # has (grpc-server, run.sh and assets/ in one directory, dylibs in lib/).
+  # No build-type: the backend Makefile turns ENGINE_ENABLE_METAL on from
+  # uname -s. lang=go drives runner/toolchain selection only - there is no
+  # backend/go/audio-cpp, which is why backend_build_darwin.yml and
+  # DARWIN_BESPOKE_BUILDERS in scripts/lib/backend-filter.mjs both route this
+  # backend away from the generic Go path.
+  - backend: "audio-cpp"
+    tag-suffix: "-metal-darwin-arm64-audio-cpp"
+    lang: "go"
   # LocalVQE has no Metal path; on Apple Silicon it builds CPU-only (GGML_METAL
   # OFF) but is still a native arm64 image. Uses the darwin/metal build profile.
   - backend: "localvqe"
@@ -6154,6 +6425,10 @@ includeDarwin:
     tag-suffix: "-metal-darwin-arm64-cloud-proxy"
     build-type: "metal"
     lang: "go"
+  - backend: "valkey-store"
+    tag-suffix: "-metal-darwin-arm64-valkey-store"
+    build-type: "metal"
+    lang: "go"
   - backend: "llama-cpp-quantization"
     tag-suffix: "-metal-darwin-arm64-llama-cpp-quantization"
     build-type: "mps"

.github/ci/gen-redirects.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+#
+# Generate client-side redirects for the documentation URLs that used to live at
+# the site root.
+#
+# Until this site existed, the Hugo docs site WAS localai.io, so pages
+# were published at /features/..., /getting-started/..., /faq/ and so on. The
+# docs now build under /docs/, and GitHub Pages serves static files only: there
+# is no server-side rewrite, no .htaccess, no _redirects. The only way to keep
+# every published, bookmarked and search-indexed URL alive is to leave a real
+# HTML file at the old address that sends the browser to the new one.
+#
+# Anything the main site already publishes wins: it owns /, /engines/,
+# /blog/ and friends, so an existing file is never replaced.
+#
+# Usage: gen-redirects.sh <public-dir> [base-url]
+#   public-dir  merged output directory (main site with docs/ inside it)
+#   base-url    absolute or root-relative prefix the deployment is served from,
+#               trailing slash optional (default "/")
+
+set -euo pipefail
+
+PUBLIC_DIR=${1:?usage: gen-redirects.sh <public-dir> [base-url]}
+BASE_URL=${2:-/}
+
+# Normalise to exactly one trailing slash so concatenation below is predictable.
+BASE_URL="${BASE_URL%/}/"
+
+DOCS_DIR="${PUBLIC_DIR}/docs"
+
+if [ ! -d "$DOCS_DIR" ]; then
+  echo "gen-redirects: no docs output at ${DOCS_DIR}" >&2
+  exit 1
+fi
+
+created=0
+skipped=0
+
+# Every .html file is a reachable old URL, not just directory indexes: the
+# generated model gallery ships as a bare gallery.html and used to sit at the
+# root too.
+while IFS= read -r src; do
+  rel=${src#"$DOCS_DIR"/}
+  dst="${PUBLIC_DIR}/${rel}"
+
+  if [ -e "$dst" ]; then
+    skipped=$((skipped + 1))
+    continue
+  fi
+
+  # Link to the directory, not to its index.html, so the redirect target is the
+  # canonical URL the docs site itself advertises.
+  target="${BASE_URL}docs/${rel%index.html}"
+
+  mkdir -p "$(dirname "$dst")"
+  printf '%s' '<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>Moved</title>
+<link rel="canonical" href="'"$target"'">
+<meta name="robots" content="noindex">
+<meta http-equiv="refresh" content="0; url='"$target"'">
+</head>
+<body>
+<p>This page moved to <a href="'"$target"'">'"$target"'</a>.</p>
+</body>
+</html>
+' > "$dst"
+
+  created=$((created + 1))
+done <<EOF
+$(find "$DOCS_DIR" -type f -name '*.html' | sort)
+EOF
+
+echo "gen-redirects: ${created} redirect(s) written, ${skipped} path(s) left to the main site"

.github/ci/refresh-site-counters.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# Refreshes the counters shown on the landing page from the GitHub API.
+#
+# The numbers used to be typed into the templates by hand, which meant they
+# only moved when somebody remembered, and a stale star count on the front
+# page is worse than no star count. Everything the API can answer for lives
+# in website/data/stats.yaml and is rewritten wholesale by this script.
+#
+# Anything the API cannot answer for (the Discord member count) is read back
+# out of the existing file and carried through untouched.
+set -euo pipefail
+
+REPO="${REPO:-mudler/LocalAI}"
+OUT="${OUT:-website/data/stats.yaml}"
+
+# The contributors and releases endpoints are paginated and never report a
+# total. Asking for one item per page makes the last page number equal to the
+# item count, which the Link header hands over.
+count_via_link_header() {
+  local path="$1" link last
+  link=$(gh api -i "${path}?per_page=1" 2>/dev/null | tr -d '\r' | grep -i '^link:' || true)
+  if [ -z "$link" ]; then
+    # No Link header means a single page, so count that page directly.
+    gh api "${path}?per_page=100" --jq 'length'
+    return
+  fi
+  last=$(sed -n 's/.*[?&]page=\([0-9]*\)>; rel="last".*/\1/p' <<<"$link")
+  [ -n "$last" ] || { gh api "${path}?per_page=100" --jq 'length'; return; }
+  printf '%s\n' "$last"
+}
+
+read -r stars forks < <(gh api "repos/${REPO}" --jq '"\(.stargazers_count) \(.forks_count)"')
+contributors=$(count_via_link_header "repos/${REPO}/contributors")
+releases=$(count_via_link_header "repos/${REPO}/releases")
+
+# Not derivable from the GitHub API, so keep whatever is already on disk.
+discord=$(sed -n 's/^discord: *\([0-9]*\).*/\1/p' "$OUT" 2>/dev/null | head -1)
+discord="${discord:-0}"
+
+for n in stars forks contributors releases; do
+  v="${!n}"
+  [[ "$v" =~ ^[0-9]+$ ]] && [ "$v" -gt 0 ] || {
+    echo "refusing to write: ${n} came back as '${v}'" >&2
+    exit 1
+  }
+done
+
+cat > "$OUT" <<YAML
+# Counters shown on the landing page.
+#
+# The four GitHub fields are rewritten by .github/ci/refresh-site-counters.sh,
+# which runs weekly from .github/workflows/refresh-site-counters.yml. Editing
+# them by hand works but will be overwritten on the next run.
+stars: ${stars}
+forks: ${forks}
+contributors: ${contributors}
+releases: ${releases}
+
+# The GitHub API cannot answer for this one, so it is maintained by hand and
+# the refresh script carries it through untouched.
+discord: ${discord}
+YAML
+
+echo "stars=${stars} forks=${forks} contributors=${contributors} releases=${releases} discord=${discord}"

.github/workflows/backend_build_darwin.yml

@@ -244,8 +244,19 @@ jobs:
           make protogen-go
           make backends/privacy-filter-darwin
 
+      # audio-cpp is a C++/ggml backend like ds4 and privacy-filter - a single
+      # grpc-server with otool dylib bundling, plus bundled VAD assets - so it
+      # gets its own bespoke darwin script rather than the generic
+      # build-darwin-go-backend path, which would look for a backend/go/audio-cpp
+      # that does not exist. Keep this set in sync with DARWIN_BESPOKE_BUILDERS
+      # in scripts/lib/backend-filter.mjs.
+      - name: Build audio-cpp backend (Darwin Metal)
+        if: inputs.backend == 'audio-cpp'
+        run: |
+          make backends/audio-cpp-darwin
+
       - name: Build ${{ inputs.backend }}-darwin
-        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4' && inputs.backend != 'privacy-filter'
+        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4' && inputs.backend != 'privacy-filter' && inputs.backend != 'audio-cpp'
         run: |
           make protogen-go
           BACKEND=${{ inputs.backend }} BUILD_TYPE=${{ inputs.build-type }} USE_PIP=${{ inputs.use-pip }} make build-darwin-${{ inputs.lang }}-backend

.github/workflows/build-test.yaml

@@ -5,6 +5,24 @@ on:
     branches:
       - master
   pull_request:
+    # GoReleaser and the darwin launcher take no gallery, docs or markdown
+    # input, so a diff confined to these paths cannot change either binary.
+    # The darwin job matters here: macOS is the scarcest runner class.
+    #
+    # backend/{cpp,go,python}/**: GoReleaser builds ./cmd/local-ai and the
+    # launcher builds ./cmd/launcher; neither compiles a backend. The
+    # before-hooks still matter, but their input is backend/backend.proto
+    # (protogen-go) and go.mod/go.sum (go mod tidy), none of which live under
+    # these prefixes, so a change to any of those still triggers a full run.
+    # See .agents/ci-caching.md.
+    paths-ignore:
+      - 'gallery/**'
+      - 'docs/**'
+      - 'examples/**'
+      - '**/*.md'
+      - 'backend/cpp/**'
+      - 'backend/go/**'
+      - 'backend/python/**'
 
 # Supersede an in-flight run when a PR gets a new push. Keyed on the PR number
 # so every push to the same PR shares a group; on a master push the key falls
@@ -26,9 +44,15 @@ jobs:
         uses: actions/setup-go@v5
         with:
           go-version: 1.25
+      # A PR builds only the host target. The three-platform cross-compile
+      # (linux/amd64, linux/arm64, darwin/arm64) is the bulk of this job's
+      # ~6.6min median and no PR consumes the resulting binaries. The
+      # before-hooks (protogen-go, react-ui, go mod tidy) run either way, so the
+      # "is the release build broken" signal is unchanged. master and tags still
+      # build everything.
       - name: Run GoReleaser
         run: |
-          make dev-dist
+          make ${{ github.event_name == 'pull_request' && 'dev-dist-single' || 'dev-dist' }}
   launcher-build-darwin:
     runs-on: macos-latest
     steps:

.github/workflows/bump_deps.yaml

@@ -30,6 +30,10 @@ jobs:
             variable: "DS4_VERSION"
             branch: "main"
             file: "backend/cpp/ds4/Makefile"
+          - repository: "0xShug0/audio.cpp"
+            variable: "AUDIO_CPP_VERSION"
+            branch: "main"
+            file: "backend/cpp/audio-cpp/Makefile"
           - repository: "meituan-longcat/LongCat-Video"
             variable: "LONGCAT_VIDEO_VERSION"
             branch: "main"
@@ -78,6 +82,10 @@ jobs:
             variable: "STABLEDIFFUSION_GGML_VERSION"
             branch: "master"
             file: "backend/go/stablediffusion-ggml/Makefile"
+          - repository: "localai-org/trellis2cpp"
+            variable: "TRELLIS2CPP_VERSION"
+            branch: "pbr-textures"
+            file: "backend/go/trellis2cpp/Makefile"
           - repository: "mudler/go-piper"
             variable: "PIPER_VERSION"
             branch: "master"
@@ -102,10 +110,14 @@ jobs:
             variable: "LOCATEANYTHING_VERSION"
             branch: "master"
             file: "backend/go/locate-anything-cpp/Makefile"
-          - repository: "ServeurpersoCom/qwentts.cpp"
-            variable: "QWEN3TTS_CPP_VERSION"
-            branch: "master"
-            file: "backend/go/qwen3-tts-cpp/Makefile"
+          # qwentts.cpp is held, not tracked: upstream master hangs in synthesis
+          # (see the comment on QWEN3TTS_CPP_VERSION in the backend Makefile).
+          # Leaving it here would re-bump the pin back onto the hang every night.
+          # Restore this entry once the upstream fix lands.
+          # - repository: "ServeurpersoCom/qwentts.cpp"
+          #   variable: "QWEN3TTS_CPP_VERSION"
+          #   branch: "master"
+          #   file: "backend/go/qwen3-tts-cpp/Makefile"
           - repository: "ServeurpersoCom/omnivoice.cpp"
             variable: "OMNIVOICE_VERSION"
             branch: "master"

.github/workflows/gh-pages.yml

@@ -1,4 +1,4 @@
-name: Deploy docs to GitHub Pages
+name: Deploy site to GitHub Pages
 
 on:
   push:
@@ -6,9 +6,11 @@ on:
       - master
     paths:
       - 'docs/**'
+      - 'website/**'
       - 'gallery/**'
       - 'images/**'
       - '.github/ci/modelslist.go'
+      - '.github/ci/gen-redirects.sh'
       - '.github/workflows/gh-pages.yml'
   workflow_dispatch:
 
@@ -23,7 +25,20 @@ concurrency:
 
 jobs:
   build:
-    runs-on: ubuntu-latest
+    # Self-hosted. This workflow is push-to-master + workflow_dispatch only, so
+    # it never executes pull-request code and a fork cannot reach the runner
+    # with untrusted changes. The repository guard keeps forks (whose own master
+    # pushes would otherwise queue forever against a label they do not have) on
+    # the hosted pool.
+    #
+    # Why: the GitHub-hosted pool is shared account-wide and has repeatedly
+    # starved (2026-07-31: 35 consecutive minutes at zero scheduled jobs, while
+    # arc-runner-set kept completing work throughout). Publishing the site is
+    # small, frequent, and must not sit behind a saturated hosted queue.
+    #
+    # Needs only git, tar and curl on the runner: setup-go and actions-hugo
+    # fetch their own toolchains, and no step uses sudo, apt, make or unzip.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     env:
       HUGO_VERSION: "0.146.3"
     steps:
@@ -49,25 +64,46 @@ jobs:
         id: pages
         uses: actions/configure-pages@v6
 
+      # The gallery page is generated from the model index and shipped as a
+      # static asset of the docs site, so it has to exist before Hugo runs.
       - name: Generate gallery
         run: go run ./.github/ci/modelslist.go ./gallery/index.yaml > docs/static/gallery.html
 
-      - name: Build site
+      # Two Hugo sites, one Pages artifact: the main site owns the root,
+      # the docs site is nested under /docs/.
+      - name: Build the main site
+        working-directory: website
+        run: hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+
+      - name: Build documentation site
         working-directory: docs
         run: |
           mkdir -p layouts/_default
-          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/docs/"
+
+      - name: Merge documentation into the main site
+        run: |
+          mkdir -p website/public/docs
+          cp -R docs/public/. website/public/docs/
+
+      # Keeps the pre-split URLs alive; see the script header.
+      - name: Generate legacy URL redirects
+        run: .github/ci/gen-redirects.sh website/public "${{ steps.pages.outputs.base_url }}/"
 
       - name: Upload artifact
         uses: actions/upload-pages-artifact@v5
         with:
-          path: docs/public
+          path: website/public
 
   deploy:
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}
-    runs-on: ubuntu-latest
+    # Same routing as build: a hosted slot for a ~10s deploy is exactly the kind
+    # of job that should not block on a starved pool. deploy-pages authenticates
+    # with the job's OIDC token (id-token: write above), which self-hosted
+    # runners issue the same way hosted ones do.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     needs: build
     steps:
       - name: Deploy to GitHub Pages

.github/workflows/image-pr.yml

@@ -3,7 +3,28 @@
   
   on:
     pull_request:
-  
+      # None of these seven image builds can observe a diff confined to these
+      # paths. Gallery metadata is parsed at runtime and never copied into an
+      # image; docs and markdown never enter one at all. Gallery content is
+      # still checked by yaml-check.yml and by
+      # core/gallery/variants_lint_test.go under 'tests'.
+      #
+      # backend/{cpp,go,python}/**: this workflow builds the core image, whose
+      # only compiled output is `make build` -> `go build ./cmd/local-ai`. The
+      # per-backend trees are copied into the builder but nothing in them reaches
+      # the binary or the final stage. backend/backend.proto is deliberately not
+      # listed: it feeds protogen-go and so does change the binary, and it does
+      # not live under any of these prefixes, so it still triggers a full run.
+      # See .agents/ci-caching.md.
+      paths-ignore:
+        - 'gallery/**'
+        - 'docs/**'
+        - 'examples/**'
+        - '**/*.md'
+        - 'backend/cpp/**'
+        - 'backend/go/**'
+        - 'backend/python/**'
+
   concurrency:
     group: ci-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
     cancel-in-progress: ${{ github.event_name == 'pull_request' }}

.github/workflows/image.yml

@@ -13,8 +13,53 @@
     cancel-in-progress: ${{ github.event_name == 'pull_request' }}
   
   jobs:
-    hipblas-jobs:
+    # Decide once whether this push can change any image. Gallery metadata is
+    # fetched at runtime and never baked into an image, and docs/markdown never
+    # enter one, so a push confined to those paths produces byte-identical
+    # images. On 2026-07-30, 12 of the 23 queued runs of this workflow were
+    # commits like "add 1 new model to gallery" or a docs fix, each rebuilding
+    # all 18 images.
+    #
+    # A job-level gate rather than `paths-ignore` on the trigger: paths-ignore
+    # would also apply to tag pushes, and a tag created on an existing commit
+    # carries an empty commits list, which would silently skip the release image
+    # build. Tags short-circuit to "build" below, as does a push whose base
+    # commit cannot be resolved -- the same run-everything posture the backend
+    # matrix filter takes for a truncated diff.
+    changes:
       if: github.repository == 'mudler/LocalAI'
+      runs-on: ubuntu-latest
+      outputs:
+        build: ${{ steps.decide.outputs.build }}
+      steps:
+        - uses: actions/checkout@v7
+          with:
+            fetch-depth: 0
+        - id: decide
+          env:
+            BEFORE: ${{ github.event.before }}
+            AFTER: ${{ github.sha }}
+          run: |
+            set -euo pipefail
+            emit() { echo "$2"; echo "build=$1" >> "$GITHUB_OUTPUT"; exit 0; }
+            case "${GITHUB_REF}" in
+              refs/tags/*) emit true "tag push: building every image" ;;
+            esac
+            if [ -z "${BEFORE:-}" ] || [ "${BEFORE}" = "0000000000000000000000000000000000000000" ] \
+               || ! git cat-file -e "${BEFORE}^{commit}" 2>/dev/null; then
+              emit true "no resolvable base commit: building every image"
+            fi
+            files="$(git diff --name-only "${BEFORE}" "${AFTER}")"
+            echo "changed files:"; echo "${files:-<none>}"
+            [ -z "${files}" ] && emit true "empty diff: building every image"
+            if echo "${files}" | grep -qvE '^(gallery/|docs/|examples/)|\.md$'; then
+              emit true "push touches image-visible content: building"
+            fi
+            emit false "only gallery/docs/markdown changed: images identical, skipping"
+
+    hipblas-jobs:
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}
@@ -47,7 +92,8 @@
               ubuntu-codename: 'noble'
 
     core-image-build:
-      if: github.repository == 'mudler/LocalAI'
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}
@@ -155,8 +201,8 @@
       # merge whenever any matrix cell of the parent build fails or is
       # cancelled. Same fix as backend.yml's merge jobs — we still want to
       # publish the manifest list for tag-suffixes whose legs all succeeded.
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -168,8 +214,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-vulkan-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -187,8 +233,8 @@
     # Each merge job needs only its parent build matrix and is filtered by
     # tag-suffix in image_merge.yml's artifact-download pattern.
     gpu-nvidia-cuda-12-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -200,8 +246,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-nvidia-cuda-13-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -213,8 +259,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-intel-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: core-image-build
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, core-image-build]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -226,8 +272,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gpu-hipblas-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: hipblas-jobs
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, hipblas-jobs]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -239,8 +285,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     nvidia-l4t-arm64-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: gh-runner
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, gh-runner]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -252,8 +298,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     nvidia-l4t-arm64-cuda-13-image-merge:
-      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' }}
-      needs: gh-runner
+      if: ${{ !cancelled() && github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true' }}
+      needs: [changes, gh-runner]
       uses: ./.github/workflows/image_merge.yml
       with:
         tag-latest: 'auto'
@@ -265,7 +311,8 @@
         quayPassword: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
 
     gh-runner:
-      if: github.repository == 'mudler/LocalAI'
+      needs: changes
+      if: github.repository == 'mudler/LocalAI' && needs.changes.outputs.build == 'true'
       uses: ./.github/workflows/image_build.yml
       with:
         tag-latest: ${{ matrix.tag-latest }}

.github/workflows/lint.yml

@@ -8,6 +8,9 @@ on:
       - 'examples/**'
       - 'README.md'
       - '**/*.md'
+      # golangci-lint runs new-from-merge-base, so a diff with no touched Go
+      # lines can only ever be a no-op. See .agents/ci-caching.md.
+      - 'gallery/**'
   push:
     branches:
       - master
@@ -18,8 +21,41 @@ concurrency:
 
 jobs:
   golangci-lint:
+    # Self-hosted for PUSH only, and only in the canonical repo.
+    #
+    # This workflow also runs on pull_request, which for a fork PR means
+    # executing untrusted contributor code. That must never land on a
+    # self-hosted runner, so anything that is not a push to mudler/LocalAI stays
+    # on the ephemeral hosted pool. Pushes to master are trusted code that has
+    # already been reviewed and merged.
+    #
+    # Why at all: the hosted pool is shared account-wide and starved for 35
+    # straight minutes on 2026-07-31 while arc-runner-set kept completing jobs.
+    # Lint is small and runs on every commit, so it is a good candidate to move
+    # off the contended pool.
+    # REVERTED to hosted: the arc-runner-set image has git, curl, unzip, tar,
+    # ldd and python3, but NOT make (nor gcc). Measured on run 30637392862,
+    # where the preflight below named both. Re-route here once the runner image
+    # ships a C toolchain and make; the preflight stays so the next attempt
+    # fails by name in one second instead of opaquely mid-build.
     runs-on: ubuntu-latest
     steps:
+      - name: Preflight - required host tools
+        # The hosted images ship these; a self-hosted container image may not.
+        # Check up front so a missing tool reports itself by name instead of
+        # surfacing as an opaque failure inside `make protogen-go` (which needs
+        # curl + unzip for protoc) or `make lint`.
+        run: |
+          missing=""
+          for t in git curl unzip make tar; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
       - uses: actions/checkout@v7
         with:
           # Full history so golangci-lint's new-from-merge-base can reach
@@ -52,8 +88,30 @@ jobs:
     # container build (a missing transitive dep, a partial cuDNN family). Their
     # shell tests need nothing but bash + gcc + ldd, so run them on every PR
     # rather than waiting on a multi-GB cross-arch backend image build.
+    #
+    # Push-only self-hosted routing, same fork-safety reasoning as
+    # golangci-lint above.
+    # REVERTED to hosted: the arc-runner-set image has git, curl, unzip, tar,
+    # ldd and python3, but NOT make (nor gcc). Measured on run 30637392862,
+    # where the preflight below named both. Re-route here once the runner image
+    # ships a C toolchain and make; the preflight stays so the next attempt
+    # fails by name in one second instead of opaquely mid-build.
     runs-on: ubuntu-latest
     steps:
+      - name: Preflight - required host tools
+        # This job additionally needs a C toolchain: the packaging-script tests
+        # compile a throwaway binary and inspect it with ldd.
+        run: |
+          missing=""
+          for t in git make gcc ldd python3; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
       - uses: actions/checkout@v7
       - name: run packaging script tests
         run: make test-build-scripts
@@ -66,3 +124,9 @@ jobs:
           node-version: '20'
       - name: run CI script tests
         run: make test-ci-scripts
+
+      # The shared python backend helpers (Options[] parsing, engine-arg
+      # mapping, model reference resolution) are stdlib-only, so their tests
+      # ride along here instead of waiting on a multi-GB backend image build.
+      - name: run shared python backend helper tests
+        run: make test-python-helpers

.github/workflows/refresh-site-counters.yml

@@ -0,0 +1,44 @@
+name: Refresh site counters
+
+# The landing page shows a star count, a contributor count and a release
+# count. They were typed in by hand, so they drifted the moment somebody
+# forgot. This pulls the real numbers once a week and commits them only when
+# they have actually moved, which in turn triggers the usual Pages deploy.
+
+on:
+  schedule:
+    # Mondays, 06:17 UTC. Off the hour on purpose, since the scheduler queues
+    # everything that asks for :00 and drops what it cannot run.
+    - cron: '17 6 * * 1'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  group: refresh-site-counters
+  cancel-in-progress: false
+
+jobs:
+  refresh:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Read the counts off the GitHub API
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: ./.github/ci/refresh-site-counters.sh
+
+      - name: Commit only if something moved
+        run: |
+          if git diff --quiet -- website/data/stats.yaml; then
+            echo "counters unchanged, nothing to commit"
+            exit 0
+          fi
+          git diff --unified=0 -- website/data/stats.yaml
+          git config user.name  "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git add website/data/stats.yaml
+          git commit -m "chore(website): refresh the counters"
+          git push

.github/workflows/secscan.yaml

@@ -28,9 +28,9 @@ jobs:
     steps:
       - name: Checkout Source
         uses: actions/checkout@v7
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
       - name: Run Gosec Security Scanner
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
         uses: securego/gosec@v2.27.1
         with:
           # we let the report trigger content trigger a failure using the GitHub Security features.
@@ -39,7 +39,7 @@ jobs:
           # noise, G104 unhandled errors) are inherent to that upstream code, not ours to rewrite.
           args: '-no-fail -exclude-dir=backend/go/supertonic -fmt sarif -out results.sarif ./...'
       - name: Upload SARIF file
-        if: ${{ github.actor != 'dependabot[bot]' }}
+        if: ${{ !github.repository.fork && github.actor != 'dependabot[bot]' }}
         uses: github/codeql-action/upload-sarif@v4
         with:
           # Path to SARIF file relative to the root of the repository

.github/workflows/test-extra.yml

@@ -38,6 +38,7 @@ jobs:
       acestep-cpp: ${{ steps.detect.outputs.acestep-cpp }}
       qwen3-tts-cpp: ${{ steps.detect.outputs.qwen3-tts-cpp }}
       magpie-tts-cpp: ${{ steps.detect.outputs.magpie-tts-cpp }}
+      trellis2cpp: ${{ steps.detect.outputs.trellis2cpp }}
       rfdetr-cpp: ${{ steps.detect.outputs.rfdetr-cpp }}
       locate-anything-cpp: ${{ steps.detect.outputs.locate-anything-cpp }}
       vibevoice-cpp: ${{ steps.detect.outputs.vibevoice-cpp }}
@@ -935,6 +936,41 @@ jobs:
       - name: Test rfdetr-cpp
         run: |
           make --jobs=5 --output-sync=target -C backend/go/rfdetr-cpp test
+  # Weight-free packaged-backend smoke for trellis2cpp. Starting run.sh loads
+  # libtrellis2 + ggml, resolves the complete C ABI (including remeshing), and
+  # answers gRPC Health without downloading or loading the multi-GB model set.
+  tests-trellis2cpp:
+    needs: detect-changes
+    if: needs.detect-changes.outputs.trellis2cpp == 'true' || needs.detect-changes.outputs.run-all == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          submodules: true
+      - name: Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential cmake curl unzip
+      - name: Setup Go
+        uses: actions/setup-go@v5
+      - name: Display Go version
+        run: go version
+      - name: Proto Dependencies
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
+          unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Build trellis2cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/trellis2cpp
+      - name: Test trellis2cpp
+        run: |
+          make --jobs=5 --output-sync=target -C backend/go/trellis2cpp test
   # Per-backend e2e for locate-anything-cpp: builds the .so + Go binary and
   # runs `make -C backend/go/locate-anything-cpp test`. test.sh fetches the
   # locate-anything-q8_0 GGUF (~6.3 GB, NVIDIA LocateAnything-3B) from the

.github/workflows/test.yml

@@ -24,8 +24,13 @@ jobs:
         uses: actions/checkout@v7
         with:
           submodules: true
-      - name: Free disk space
-        uses: ./.github/actions/free-disk-space
+      # No free-disk-space step here on purpose. That action exists to make room
+      # for docker buildx layers, and this job runs no buildx step. It was also
+      # sized for a `make test` that downloaded multi-GB GGUF/whisper fixtures
+      # and built llama-cpp/whisper/stablediffusion-ggml; after the test-suite
+      # reorg it does neither (see the Makefile test target). It cost ~3min of
+      # every run, and its tool-cache:true wipe also forced setup-go and
+      # setup-node to re-download toolchains that ship preinstalled.
       - name: Setup Go ${{ matrix.go-version }}
         uses: actions/setup-go@v5
         with:

.github/workflows/tests-e2e.yml

@@ -3,6 +3,14 @@ name: 'E2E Backend Tests'
 
 on:
   pull_request:
+    # The e2e suite drives backends over gRPC directly and reads none of these
+    # paths, so a diff confined to them cannot move it.
+    # See .agents/ci-caching.md.
+    paths-ignore:
+      - 'gallery/**'
+      - 'docs/**'
+      - 'examples/**'
+      - '**/*.md'
   push:
     branches:
       - master

.gitignore

@@ -30,6 +30,7 @@ LocalAI
 # Go backend packages whose main lives under backend/go/.
 /cloud-proxy
 /local-store
+/valkey-store
 # prevent above rules from omitting the helm chart
 !charts/*
 # prevent above rules from omitting the api/localai folder
@@ -61,6 +62,11 @@ prepare
 /ggml-metal.metal
 docs/static/gallery.html
 
+# Hugo build output and lock files (docs/ and website/)
+docs/public/
+website/public/
+.hugo_build.lock
+
 # Protobuf generated files
 *.pb.go
 *pb2.py

ADOPTERS.md

@@ -0,0 +1,48 @@
+# Adopters
+
+Organisations running LocalAI, listed by the people who run it.
+
+If your organisation uses LocalAI and you are happy to say so publicly, open a
+pull request adding a row to the table below. That pull request is how we know
+we have permission to list you, which is why we do not add anybody ourselves.
+
+You do not need to be a large company, and you do not need to disclose anything
+sensitive. A sentence on what you use it for is more useful to other readers
+than a logo.
+
+## How to add yourself
+
+1. Add a row to the table, in alphabetical order.
+2. Use your organisation's usual name and a link to your site.
+3. Say briefly what you use LocalAI for, and whether it is in production.
+4. Open the pull request from an account that makes it plausible you speak for
+   the organisation, or say in the description who you are. We may ask.
+
+To be removed, open a pull request deleting your row, or email
+[info@localai.io](mailto:info@localai.io). We will not ask why.
+
+## Who is using LocalAI
+
+<!-- Keep alphabetical. Columns: Organisation | What for | Status -->
+
+| Organisation | What they use it for | Status |
+|---|---|---|
+| _Your organisation here_ | | |
+
+## What this list is not
+
+This is not a list of everyone who has ever starred the repository, and it is
+not a list of the employers of people who have contributed a patch. Both of
+those are easy to scrape and neither means what a logo wall implies.
+
+The website shows two separate things, both of which are checkable without
+anybody's permission:
+
+- **Engineers from these companies have contributed code.** Evidence is the
+  commit history plus the employer on that person's public GitHub profile. It
+  is a claim about a person, not about their employer.
+- **These projects integrate LocalAI.** Evidence is a reference to LocalAI in
+  that project's own repository or documentation.
+
+Those two lists live in [`website/data/ecosystem.yaml`](website/data/ecosystem.yaml).
+This file is the third, stronger thing: organisations that chose to say so.

AGENTS.md

@@ -32,6 +32,7 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 | [.agents/adding-gallery-models.md](.agents/adding-gallery-models.md) | Adding GGUF models from HuggingFace to the model gallery |
 | [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) | LocalAI Assistant chat modality — adding admin tools to the in-process MCP server, editing skill prompts, keeping REST + MCP + skills in sync |
 | [.agents/backend-signing.md](.agents/backend-signing.md) | Backend OCI image signing (keyless cosign + sigstore-go) — producer-side CI setup, consumer-side gallery `verification:` block, strict mode (`LOCALAI_REQUIRE_BACKEND_INTEGRITY`), revocation via `not_before` |
+| [.agents/preparing-a-release.md](.agents/preparing-a-release.md) | Cutting a release: PR labels, `RELEASE_NOTES_vX.Y.Z.md`, the blog post under `website/content/blog/`, and the demo clips under `website/static/media/` |
 
 ## Quick Reference
 
@@ -42,6 +43,7 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 - **Docs (docs-with-code rule)**: When you change user-facing behavior (API endpoints, CLI flags, config keys, or features), update the corresponding page under `docs/content/` in the SAME change, not as a follow-up. A user-facing change without a matching docs update is incomplete. See also the documentation conventions in [.agents/coding-style.md](.agents/coding-style.md).
 - **New API endpoints**: LocalAI advertises its capability surface in several independent places — swagger `@Tags`, `/api/instructions` registry, auth `RouteFeatureRegistry`, React UI `capabilities.js`, docs. Read [.agents/api-endpoints-and-auth.md](.agents/api-endpoints-and-auth.md) and follow its checklist — missing any surface means clients, admins, and the UI won't know the endpoint exists.
 - **Admin endpoints → MCP tool**: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in `pkg/mcp/localaitools/`. The LocalAI Assistant chat modality and the standalone `local-ai mcp-server` consume that package; drift between REST and MCP is a real risk. Read [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) — the `TestToolHTTPRouteMappingComplete` test fails until you wire the new tool and update the route map.
+- **Releases ship with a post and clips**: a release is not done at the tag. It needs labelled PRs, `RELEASE_NOTES_vX.Y.Z.md`, a blog post under `website/content/blog/`, and a short demo clip in `website/static/media/` for each notable feature. See [.agents/preparing-a-release.md](.agents/preparing-a-release.md).
 - **Build**: Inspect `Makefile` and `.github/workflows/` — ask the user before running long builds
 - **Backend OS coverage**: a new backend must target every OS it can build for, not just Linux. `.github/backend-matrix.yml` has two matrices — `include:` (Linux) and `includeDarwin:` (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the `includeDarwin` entry + `backend/index.yaml` `metal:` entries, or say in the PR why an OS is unsupported. See the darwin checklist in [.agents/adding-backends.md](.agents/adding-backends.md).
 - **Gallery variant ranking**: a gallery entry can declare `variants` (alternative builds of the same weights), and LocalAI ranks the ones a host can run by engine preference first, size second. A new backend that should be preferred on some hardware must be listed in `engineNamePreferenceRules` in `pkg/system/capabilities.go`; the sibling `backendBuildTagPreferenceRules` speaks build tags rather than engine names, and using the wrong table matches nothing without erroring. See [.agents/adding-backends.md](.agents/adding-backends.md).

Makefile

@@ -1,5 +1,5 @@
 # Disable parallel execution for backend builds
-.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/magpie-tts-cpp backends/vllm-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
+.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/trellis2cpp backends/trellis2cpp-darwin backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/whisper-medusa backends/silero-vad backends/local-store backends/valkey-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/magpie-tts-cpp backends/vllm-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin backends/audio-cpp backends/audio-cpp-darwin
 
 GOCMD=go
 GOTEST=$(GOCMD) test
@@ -69,7 +69,7 @@ else
 	GORELEASER=$(shell which goreleaser)
 endif
 
-TEST_PATHS?=./api/... ./pkg/... ./core/... ./backend/go/cloud-proxy/... ./backend/go/local-store/...
+TEST_PATHS?=./api/... ./pkg/... ./core/... ./backend/go/cloud-proxy/... ./backend/go/local-store/... ./backend/go/valkey-store/...
 
 ## Coverage output and the committed baseline that CI compares against.
 ## The gate is strict: total coverage must never decrease (no tolerance).
@@ -172,6 +172,15 @@ build-dev: ## Run LocalAI in dev mode with live reload
 dev-dist:
 	$(GORELEASER) build --snapshot --clean
 
+## PR-time variant of dev-dist: builds only the host platform instead of all
+## three release targets (linux/amd64, linux/arm64, darwin/arm64). The point of
+## running goreleaser on a PR is to catch a broken config or a broken
+## before-hook (protogen-go, react-ui, go mod tidy), and --single-target still
+## exercises every one of those. Nothing consumes a PR's cross-compiled
+## binaries. master pushes and tags still run the full dev-dist/dist.
+dev-dist-single:
+	$(GORELEASER) build --snapshot --clean --single-target
+
 dist:
 	$(GORELEASER) build --clean
 
@@ -222,6 +231,14 @@ test-build-scripts:
 test-ci-scripts:
 	@set -e; for t in scripts/lib/*_test.mjs; do echo "== $$t"; node --test "$$t"; done
 
+## Runs the unit tests for the shared python backend helpers. These modules are
+## pure stdlib on purpose so they run without any backend venv; the list is
+## explicit because their siblings (model_identity_test) import grpc and the
+## generated protobufs, which only exist inside a built backend.
+PYTHON_HELPER_TESTS?=python_utils_test vllm_utils_test model_utils_test mlx_utils_test parent_watch_test
+test-python-helpers:
+	cd backend/python/common && python3 -m unittest $(PYTHON_HELPER_TESTS)
+
 ## Runs the core suite ($(TEST_PATHS)) with statement-coverage instrumentation
 ## and writes a merged profile to $(COVERAGE_PROFILE). Deliberately omits
 ## --fail-fast so a single failure doesn't truncate the coverage number, and
@@ -386,6 +403,15 @@ test-stores: backends/local-store
 	BACKENDS_PATH=$(abspath ./)/backends \
 	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) -v -r tests/integration
 
+## Valkey-backed vector-store integration. Requires a running Valkey Search
+## server (valkey/valkey-bundle:9.1.0) reachable at $$VALKEY_ADDR — the suite
+## skips itself when VALKEY_ADDR is unset. Builds the backend on demand and
+## points the model loader at it via BACKENDS_PATH. Label-filtered to the
+## valkey specs so it does not also run the in-memory local-store suite.
+test-valkey-store: backends/valkey-store
+	BACKENDS_PATH=$(abspath ./)/backends \
+	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) --label-filter='valkey' -v -r tests/integration
+
 test-opus:
 	@echo 'Running opus backend tests'
 	$(MAKE) -C backend/go/opus libopusshim.so
@@ -585,6 +611,7 @@ prepare-test-extra: protogen-python
 	$(MAKE) -C backend/python/nemo
 	$(MAKE) -C backend/python/voxcpm
 	$(MAKE) -C backend/python/faster-whisper
+	$(MAKE) -C backend/python/whisper-medusa
 	$(MAKE) -C backend/python/whisperx
 	$(MAKE) -C backend/python/ace-step
 	$(MAKE) -C backend/python/trl
@@ -594,6 +621,8 @@ prepare-test-extra: protogen-python
 	$(MAKE) -C backend/rust/kokoros kokoros-grpc
 	$(MAKE) -C backend/go/rfdetr-cpp
 	$(MAKE) -C backend/go/locate-anything-cpp
+	$(MAKE) -C backend/go/trellis2cpp
+	$(MAKE) -C backend/go/valkey-store
 
 test-extra: prepare-test-extra
 	$(MAKE) -C backend/python/transformers test
@@ -614,6 +643,7 @@ test-extra: prepare-test-extra
 	$(MAKE) -C backend/python/nemo test
 	$(MAKE) -C backend/python/voxcpm test
 	$(MAKE) -C backend/python/faster-whisper test
+	$(MAKE) -C backend/python/whisper-medusa test
 	$(MAKE) -C backend/python/whisperx test
 	$(MAKE) -C backend/python/ace-step test
 	$(MAKE) -C backend/python/trl test
@@ -626,6 +656,8 @@ test-extra: prepare-test-extra
 	$(MAKE) -C backend/go/depth-anything-cpp test
 	$(MAKE) -C backend/go/supertonic test
 	$(MAKE) -C backend/go/vllm-cpp test
+	$(MAKE) -C backend/go/trellis2cpp test
+	$(MAKE) -C backend/go/valkey-store test
 
 ##
 ## End-to-end gRPC tests that exercise a built backend container image.
@@ -1188,6 +1220,10 @@ backends/privacy-filter-darwin: build
 	bash ./scripts/build/privacy-filter-darwin.sh
 	./local-ai backends install "ocifile://$(abspath ./backend-images/privacy-filter.tar)"
 
+backends/audio-cpp-darwin: build
+	bash ./scripts/build/audio-cpp-darwin.sh
+	./local-ai backends install "ocifile://$(abspath ./backend-images/audio-cpp.tar)"
+
 build-darwin-python-backend: build
 	bash ./scripts/build/python-darwin.sh
 
@@ -1218,6 +1254,10 @@ backends/stablediffusion-ggml-darwin:
 	BACKEND=stablediffusion-ggml BUILD_TYPE=metal $(MAKE) build-darwin-go-backend
 	./local-ai backends install "ocifile://$(abspath ./backend-images/stablediffusion-ggml.tar)"
 
+backends/trellis2cpp-darwin:
+	BACKEND=trellis2cpp BUILD_TYPE=metal $(MAKE) build-darwin-go-backend
+	./local-ai backends install "ocifile://$(abspath ./backend-images/trellis2cpp.tar)"
+
 backend-images:
 	mkdir -p backend-images
 
@@ -1241,14 +1281,21 @@ BACKEND_DS4 = ds4|ds4|.|false|false
 # openai-privacy-filter PII/NER token classifier) — the TokenClassify RPC for
 # the PII redactor tier, on stock ggml with no llama.cpp carry-patches.
 BACKEND_PRIVACY_FILTER = privacy-filter|privacy-filter|.|false|false
+# audio-cpp wraps 0xShug0/audio.cpp, a multi-family ggml audio inference engine
+# (TTS, ASR, VAD, diarization, source separation, music generation). Builds
+# against apt gRPC/protobuf rather than a prebuilt base-grpc image; the reason
+# is on the audio-cpp block in .github/backend-matrix.yml.
+BACKEND_AUDIO_CPP = audio-cpp|audio-cpp|.|false|false
 
 # Golang backends
 BACKEND_PIPER = piper|golang|.|false|true
 BACKEND_LOCAL_STORE = local-store|golang|.|false|true
+BACKEND_VALKEY_STORE = valkey-store|golang|.|false|true
 BACKEND_CLOUD_PROXY = cloud-proxy|golang|.|false|true
 BACKEND_HUGGINGFACE = huggingface|golang|.|false|true
 BACKEND_SILERO_VAD = silero-vad|golang|.|false|true
 BACKEND_STABLEDIFFUSION_GGML = stablediffusion-ggml|golang|.|--progress=plain|true
+BACKEND_TRELLIS2CPP = trellis2cpp|golang|.|--progress=plain|true
 BACKEND_WHISPER = whisper|golang|.|false|true
 BACKEND_CRISPASR = crispasr|golang|.|false|true
 BACKEND_PARAKEET_CPP = parakeet-cpp|golang|.|false|true
@@ -1272,6 +1319,7 @@ BACKEND_RERANKERS = rerankers|python|.|false|true
 BACKEND_TRANSFORMERS = transformers|python|.|false|true
 BACKEND_OUTETTS = outetts|python|.|false|true
 BACKEND_FASTER_WHISPER = faster-whisper|python|.|false|true
+BACKEND_WHISPER_MEDUSA = whisper-medusa|python|.|false|true
 BACKEND_COQUI = coqui|python|.|false|true
 BACKEND_RFDETR = rfdetr|python|.|false|true
 BACKEND_INSIGHTFACE = insightface|python|.|false|true
@@ -1342,12 +1390,15 @@ $(eval $(call generate-docker-build-target,$(BACKEND_TURBOQUANT)))
 $(eval $(call generate-docker-build-target,$(BACKEND_BONSAI)))
 $(eval $(call generate-docker-build-target,$(BACKEND_DS4)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PRIVACY_FILTER)))
+$(eval $(call generate-docker-build-target,$(BACKEND_AUDIO_CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PIPER)))
 $(eval $(call generate-docker-build-target,$(BACKEND_LOCAL_STORE)))
+$(eval $(call generate-docker-build-target,$(BACKEND_VALKEY_STORE)))
 $(eval $(call generate-docker-build-target,$(BACKEND_CLOUD_PROXY)))
 $(eval $(call generate-docker-build-target,$(BACKEND_HUGGINGFACE)))
 $(eval $(call generate-docker-build-target,$(BACKEND_SILERO_VAD)))
 $(eval $(call generate-docker-build-target,$(BACKEND_STABLEDIFFUSION_GGML)))
+$(eval $(call generate-docker-build-target,$(BACKEND_TRELLIS2CPP)))
 $(eval $(call generate-docker-build-target,$(BACKEND_WHISPER)))
 $(eval $(call generate-docker-build-target,$(BACKEND_CRISPASR)))
 $(eval $(call generate-docker-build-target,$(BACKEND_PARAKEET_CPP)))
@@ -1359,6 +1410,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_RERANKERS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_TRANSFORMERS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_OUTETTS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_FASTER_WHISPER)))
+$(eval $(call generate-docker-build-target,$(BACKEND_WHISPER_MEDUSA)))
 $(eval $(call generate-docker-build-target,$(BACKEND_COQUI)))
 $(eval $(call generate-docker-build-target,$(BACKEND_RFDETR)))
 $(eval $(call generate-docker-build-target,$(BACKEND_INSIGHTFACE)))
@@ -1408,7 +1460,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_SUPERTONIC)))
 docker-save-%: backend-images
 	docker save local-ai-backend:$* -o backend-images/$*.tar
 
-docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-magpie-tts-cpp docker-build-vllm-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter
+docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-whisper-medusa docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-magpie-tts-cpp docker-build-vllm-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter docker-build-trellis2cpp docker-build-valkey-store docker-build-audio-cpp
 
 ########################################################
 ### Mock Backend for E2E Tests
@@ -1500,7 +1552,12 @@ swagger:
 gen-assets:
 	$(GOCMD) run core/dependencies_manager/manager.go webui_static.yaml core/http/static/assets
 
-## Documentation
+## Documentation and website
+# The published site is two Hugo sites: website/ owns the root, docs/ is nested
+# under /docs/. Serve them separately while editing; use `make site` to get the
+# merged tree (including the legacy URL redirects) that GitHub Pages deploys.
+SITE_BASE_URL?=http://localhost:8000
+
 docs/layouts/_default:
 	mkdir -p docs/layouts/_default
 
@@ -1512,12 +1569,30 @@ docs/public: docs/layouts/_default docs/static/gallery.html
 
 docs-clean:
 	rm -rf docs/public
+	rm -rf website/public
 	rm -rf docs/static/gallery.html
 
 .PHONY: docs
 docs: docs/static/gallery.html
 	cd docs && hugo serve
 
+.PHONY: website
+website:
+	cd website && hugo serve
+
+.PHONY: site
+site: docs/static/gallery.html
+	rm -rf website/public docs/public
+	cd website && hugo --minify --baseURL "$(SITE_BASE_URL)/"
+	cd docs && hugo --minify --baseURL "$(SITE_BASE_URL)/docs/"
+	mkdir -p website/public/docs
+	cp -R docs/public/. website/public/docs/
+	./.github/ci/gen-redirects.sh website/public "$(SITE_BASE_URL)/"
+
+.PHONY: site-serve
+site-serve: site
+	cd website/public && python3 -m http.server 8000
+
 ########################################################
 ## Platform-specific builds
 ########################################################

README.md

@@ -238,13 +238,14 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native
 | [magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp) | C++/GGML port of NVIDIA's Magpie TTS Multilingual 357M: 22.05 kHz mono text-to-speech in 5 voices and 9+ languages, with the NanoCodec neural codec and tokenizer/G2P embedded in a single GGUF |
 | [ced.cpp](https://github.com/localai-org/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
 | [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
-| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Voxtral Realtime 4B speech-to-text in pure C |
+| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Mistral Voxtral-4B-TTS text-to-speech in pure C: 20 preset voices across 9 languages, 24 kHz WAV output, no dependencies beyond libc |
 | [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization |
 | [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
 | [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) |
 | [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose estimation |
 | [face-detect.cpp](https://github.com/mudler/face-detect.cpp) | Face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace), replacing the Python insightface backend |
 | [free-splatter.cpp](https://github.com/localai-org/free-splatter.cpp) | Pose-free 3D reconstruction (FreeSplatter): turns a handful of plain photos into 3D Gaussians, no camera poses or GPU required |
+| [trellis2.cpp](https://github.com/localai-org/trellis2cpp) | C++/GGML port of Microsoft TRELLIS.2: single-image to textured 3D mesh (GLB with PBR materials) |
 | [privacy-filter.cpp](https://github.com/localai-org/privacy-filter.cpp) | Standalone GGML PII/NER token-classification engine powering LocalAI's PII redaction tier |
 | [LocalVQE](https://github.com/localai-org/LocalVQE) | Joint acoustic echo cancellation, noise suppression, and dereverberation |
 | [local-store](https://github.com/mudler/LocalAI) | Local-first vector database for embeddings (shipped in-tree) |

backend/Dockerfile.audio-cpp

@@ -0,0 +1,120 @@
+ARG BASE_IMAGE=ubuntu:24.04
+ARG APT_MIRROR=""
+ARG APT_PORTS_MIRROR=""
+
+# audio-cpp: 0xShug0/audio.cpp, a ggml audio inference framework covering TTS,
+# ASR, VAD, diarization, source separation and music generation, wrapped as a
+# LocalAI gRPC backend.
+#
+# BASE_IMAGE is ubuntu:24.04 for cpu and vulkan builds, or
+# nvidia/cuda:<ver>-devel-ubuntu24.04 for cublas builds; both ship apt and
+# Ubuntu Noble packages, and the CUDA base additionally provides
+# /usr/local/cuda. BUILD_TYPE selects the engine backend in the Makefile:
+# "" = portable CPU with all ggml CPU variants, "cublas" ->
+# -DENGINE_ENABLE_CUDA=ON, "vulkan" -> -DENGINE_ENABLE_VULKAN=ON. Darwin
+# (Metal) builds bypass this Dockerfile entirely.
+#
+# Upstream needs GCC 13 or newer, which ubuntu:24.04 and the CUDA 12/13
+# devel-ubuntu24.04 images all provide.
+#
+# THIS BACKEND CANNOT USE .docker/install-base-deps.sh OR THE PREBUILT
+# quay.io/go-skynet/ci-cache:base-grpc-* IMAGES, AND THAT IS NOT A STYLE CHOICE.
+#
+# Both supply gRPC v1.65 built from source at /opt/grpc, which downstream
+# Dockerfiles copy to /usr/local. That gRPC vendors protobuf v26, and protobuf
+# has depended on abseil since v22: google/protobuf/message_lite.h includes
+# absl/strings/cord.h. audio.cpp links sentencepiece, and our CMakeLists sets
+# SPM_PROTOBUF_PROVIDER=package so sentencepiece uses the same protobuf the
+# generated backend.pb.cc was built against (the alternative broke every
+# nested-message parse; the full account is in backend/cpp/audio-cpp/CMakeLists.txt).
+# That makes sentencepiece's init.h include the external message_lite.h while it
+# still includes its own vendored mini-abseil from third_party/absl. The vendored
+# copy declares `namespace absl { namespace internal { ... } }` and real abseil
+# declares `namespace absl { inline namespace lts_20240116 { namespace internal
+# { ... } } }`, so every `absl::internal::` reference becomes ambiguous and the
+# compile dies in absl/base/casts.h. Verified, not theorised: building this image
+# against the base-grpc-amd64 prebuilt fails at
+# sentencepiece-static/error.cc.o with "reference to 'internal' is ambiguous".
+#
+# Ubuntu Noble's apt protobuf is 3.21.12, which predates the abseil dependency,
+# so message_lite.h pulls in no abseil and the vendored copy is the only one in
+# scope. That is also the exact protobuf/gRPC pair every unit and end-to-end run
+# of this backend has been verified against. Keep it: a from-source gRPC here
+# does not buy a faster build, it buys a broken one.
+#
+# The install-base-deps path is additionally unsafe because it drops protoc 27.1
+# into /usr/local/bin, which shadows apt's protoc on PATH and would generate
+# protobuf-27 sources to be compiled against 3.21 headers.
+FROM ${BASE_IMAGE} AS builder
+ARG BUILD_TYPE
+ARG TARGETARCH
+ARG TARGETVARIANT
+ARG APT_MIRROR
+ARG APT_PORTS_MIRROR
+# Selects the CUDA architecture list in backend/cpp/audio-cpp/Makefile. It has
+# to be forwarded: upstream compiles engine_runtime for `native` when
+# CMAKE_CUDA_ARCHITECTURES is unset, and no CI runner has a GPU to enumerate.
+# The value is the same cuda-major-version the matrix entry declares.
+ARG CUDA_MAJOR_VERSION
+
+ENV BUILD_TYPE=${BUILD_TYPE} \
+    CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} \
+    APT_MIRROR=${APT_MIRROR} \
+    APT_PORTS_MIRROR=${APT_PORTS_MIRROR} \
+    DEBIAN_FRONTEND=noninteractive \
+    PATH=/usr/local/cuda/bin:${PATH}
+
+WORKDIR /build
+
+# gRPC/protobuf from apt, deliberately; see the block above. libgrpc++-dev ships
+# a CMake config so find_package(gRPC CONFIG) resolves, and libprotobuf-dev
+# lands in the layout CMake's FindProtobuf module expects, which matters because
+# sentencepiece runs a bare find_package(Protobuf REQUIRED) with no CONFIG
+# fallback of its own.
+#
+# BUILD_TYPE=vulkan additionally needs the loader headers and glslc; both are in
+# Noble. The CUDA toolkit for BUILD_TYPE=cublas comes from BASE_IMAGE.
+RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mirror \
+    sh /usr/local/sbin/apt-mirror && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends \
+        git cmake build-essential pkg-config ca-certificates \
+        libgrpc++-dev libprotobuf-dev protobuf-compiler protobuf-compiler-grpc && \
+    if [ "${BUILD_TYPE}" = "vulkan" ]; then \
+        apt-get install -y --no-install-recommends libvulkan-dev glslc; \
+    fi && \
+    if [ "${TARGETARCH}" = "arm64" ]; then \
+        apt-get install -y --no-install-recommends gcc-14 g++-14; \
+    fi && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+COPY . /LocalAI
+
+# gcc-14 on arm64, for the same reason llama-cpp does it in
+# .docker/llama-cpp-compile.sh: ggml's CPU_ALL_VARIANTS table includes armv9.2
+# variants built with -march=...+sme, and Noble's default gcc-13 rejects that
+# feature modifier outright ("invalid feature modifier 'sme'"). Every variant in
+# the table has to COMPILE even though a host only ever dlopens the one its own
+# CPU supports, so one unbuildable variant fails the whole image.
+#
+# ON EVERY arm64 BUILD_TYPE, which is where this differs from llama-cpp's script
+# and why that difference is spelled out rather than assumed. llama-cpp only
+# needs gcc-14 for its pure-CPU image because its GPU builds run
+# llama-cpp-fallback, which has no variant table at all. This backend's Makefile
+# sets ENGINE_ENABLE_CPU_ALL_VARIANTS for every non-Darwin build, GPU included,
+# so an arm64 GPU image would hit the identical compile error. Gating this on an
+# empty BUILD_TYPE would leave that trap armed for the first arm64 GPU entry
+# added to the matrix, which today has none.
+RUN --mount=type=cache,target=/root/.ccache,id=audio-cpp-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
+    if [ "${TARGETARCH}" = "arm64" ]; then \
+        export CC=gcc-14 CXX=g++-14; \
+    fi && \
+    make -C /LocalAI/backend/cpp/audio-cpp BUILD_TYPE=${BUILD_TYPE} \
+        CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} NATIVE=false grpc-server package
+
+# The package directory is the whole image: run.sh, grpc-server, the dlopened
+# ggml CPU variants, the bundled loader and its library closure, and the
+# bundled silero_vad / marblenet_vad assets. Nothing else exists at run time.
+FROM scratch
+COPY --from=builder /LocalAI/backend/cpp/audio-cpp/package/. ./

backend/Dockerfile.golang

@@ -248,10 +248,48 @@ RUN <<EOT bash
     fi
 EOT
 
-COPY . /LocalAI
-
 RUN git config --global --add safe.directory /LocalAI
 
+# Prebuild the native engine from a layer that depends on this backend's own
+# directory and nothing else.
+#
+# The expensive part of a C++ backend build is the engine: each of these
+# Makefiles clones an upstream repo at a pinned SHA and compiles it once per
+# SIMD variant (depth-anything-cpp builds four: avx, avx2, avx512, fallback),
+# and those variant targets depend only on the clone. They cannot observe a
+# change anywhere else in the LocalAI tree. Building them below `COPY . /LocalAI`
+# threw that away: any Go-side edit invalidated the layer and recompiled C++ that
+# had not changed. Measured on 2026-07-30, that is a 100+ minute rebuild for the
+# larger engines.
+#
+# Copying only this backend's directory first keeps the compile in a layer that
+# survives any change elsewhere in the tree, so `cache-from: type=registry`
+# restores it. That covers the expensive cases directly: a shared-build-input or
+# backend.proto change, the weekly full-matrix cron and a tag push all rebuild
+# every backend while touching none of their directories. This is the mechanism
+# behind base-grpc-* applied one level down; unlike a --mount=type=cache it is a
+# real layer, which is what actually survives to the registry.
+#
+# The whole directory rather than just the Makefile: the CMake targets also need
+# CMakeLists.txt, and the file list differs per backend. The cost is that editing
+# this backend's Go sources also invalidates the engine layer.
+#
+# Backends whose Makefile has no `engine` target are unaffected: the guard skips
+# the prebuild and their engine still compiles in the `build` step below.
+COPY backend/go/${BACKEND}/ /LocalAI/backend/go/${BACKEND}/
+RUN cd /LocalAI/backend/go/${BACKEND} && \
+    if make -n engine >/dev/null 2>&1; then \
+        echo "==> prebuilding engine for ${BACKEND} (cacheable layer)" && \
+        make engine; \
+    else \
+        echo "==> ${BACKEND} has no engine target; it builds with the backend"; \
+    fi
+
+COPY . /LocalAI
+
+# The engine variants built above survive this COPY (they are build outputs, not
+# tracked files) and are newer than the pinned clone, so make treats them as up
+# to date and goes straight to the Go binary.
 RUN cd /LocalAI && make protogen-go && make -C /LocalAI/backend/go/${BACKEND} build
 
 FROM scratch

backend/README.md

@@ -56,6 +56,7 @@ The backend system provides language-specific Dockerfiles that handle the build
 - **stablediffusion-ggml**: Stable Diffusion in Go with GGML Cpp backend
 - **piper**: Text-to-speech synthesis Golang with C bindings using rhaspy/piper
 - **local-store**: Vector storage backend
+- **valkey-store**: Durable vector storage backend backed by Valkey Search (FT.*)
 
 #### C++ Backends (`cpp/`)
 - **llama-cpp**: Llama.cpp integration

backend/backend.proto

@@ -16,6 +16,7 @@ service Backend {
   rpc Embedding(PredictOptions) returns (EmbeddingResult) {}
   rpc GenerateImage(GenerateImageRequest) returns (Result) {}
   rpc GenerateVideo(GenerateVideoRequest) returns (Result) {}
+  rpc Generate3D(Generate3DRequest) returns (Result) {}
   rpc AudioTranscription(TranscriptRequest) returns (TranscriptResult) {}
   rpc AudioTranscriptionStream(TranscriptRequest) returns (stream TranscriptStreamResponse) {}
   // AudioTranscriptionLive is the bidirectional live-microphone ASR RPC. The
@@ -34,6 +35,7 @@ service Backend {
   rpc TTSStream(TTSRequest) returns (stream Reply) {}
   rpc SoundGeneration(SoundGenerationRequest) returns (Result) {}
   rpc TokenizeString(PredictOptions) returns (TokenizationResponse) {}
+  rpc Detokenize(DetokenizeRequest) returns (DetokenizeResponse) {}
   rpc Status(HealthMessage) returns (StatusResponse) {}
   rpc Detect(DetectOptions) returns (DetectResponse) {}
   // SoundDetection runs an audio-tagging / sound-event-classification model
@@ -658,6 +660,20 @@ message GenerateVideoRequest {
   string ModelIdentity = 15;
 }
 
+message Generate3DRequest {
+  string src = 1;          // Path to the staged conditioning image (3D generation is image-conditioned)
+  string dst = 2;          // Output path for the generated binary glTF (.glb) asset
+  int32 seed = 3;          // <=0 lets the backend pick a random seed
+  int32 step = 4;          // Flow sampling steps; <=0 uses the backend default
+  float cfg_scale = 5;     // Classifier-free guidance scale; <=0 uses the backend default
+  int32 texture_steps = 6; // Texture flow sampling steps; <=0 uses the backend default
+  string quality = 7;      // Mesh pipeline: ""|"auto"|"coarse"|"512"|"1024"
+  string background = 8;   // Conditioning-image background handling: ""|"auto"|"keep"|"black"|"white"
+  // Backend-specific per-request generation parameters. Values are strings
+  // and are validated/coerced by the selected backend.
+  map<string, string> params = 9;
+}
+
 message TTSRequest {
   string text = 1;
   string model = 2;
@@ -781,6 +797,14 @@ message TokenizationResponse {
   repeated int32 tokens = 2;
 }
 
+message DetokenizeRequest {
+  repeated int32 tokens = 1;
+}
+
+message DetokenizeResponse {
+  string content = 1;
+}
+
 message MemoryUsageData {
   uint64 total = 1;
   map<string, uint64> breakdown = 2;
@@ -1107,11 +1131,28 @@ message AudioTransformRequest {
   string ModelIdentity = 5;
 }
 
+// One named output of a transform that produces several from a single run.
+// Source separation is the case that needs it: htdemucs yields drums, bass,
+// other and vocals from one pass over the input.
+message AudioTransformStem {
+  string name = 1;                   // the model's own stem id, e.g. "vocals"
+  string dst = 2;                    // path of the file written for that stem
+}
+
 message AudioTransformResult {
   string dst = 1;
   int32  sample_rate = 2;
   int32  samples = 3;
   bool   reference_provided = 4;
+  // Every named output the run produced, in the model's own order, including
+  // the one copied into dst. Empty for a transform with a single output.
+  //
+  // It exists because dst carries one file while separation produces several,
+  // and running the model once per stem would cost four full separations of
+  // the same audio. The backend runs once, writes each stem beside dst, and
+  // names them here; without this field the other stems are on disk but no
+  // caller can find them, which is the same as not having produced them.
+  repeated AudioTransformStem stems = 5;
 }
 
 // Bidirectional streaming audio transform. The first message MUST carry a

backend/cpp/audio-cpp/.gitignore

@@ -0,0 +1,8 @@
+audio.cpp/
+build/
+package/
+grpc-server
+backend.pb.cc
+backend.pb.h
+backend.grpc.pb.cc
+backend.grpc.pb.h

backend/cpp/audio-cpp/CMakeLists.txt

@@ -0,0 +1,331 @@
+cmake_minimum_required(VERSION 3.20)
+project(audio-cpp-grpc-server LANGUAGES C CXX)
+
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(TARGET grpc-server)
+
+set(AUDIO_CPP_DIR "${CMAKE_CURRENT_SOURCE_DIR}/audio.cpp"
+    CACHE PATH "Path to the pinned audio.cpp checkout")
+option(AUDIO_CPP_GRPC_BUILD_TESTS "Build engine-linked ctest binaries" OFF)
+
+if(NOT EXISTS "${AUDIO_CPP_DIR}/CMakeLists.txt")
+    message(FATAL_ERROR
+        "AUDIO_CPP_DIR does not contain an audio.cpp checkout: ${AUDIO_CPP_DIR}. "
+        "Run 'make audio.cpp' first.")
+endif()
+
+if(APPLE)
+    # Homebrew installs protobuf/grpc under a non-default prefix.
+    if(CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "arm64")
+        set(HOMEBREW_DEFAULT_PREFIX "/opt/homebrew")
+    else()
+        set(HOMEBREW_DEFAULT_PREFIX "/usr/local")
+    endif()
+    link_directories("${HOMEBREW_DEFAULT_PREFIX}/lib")
+    include_directories("${HOMEBREW_DEFAULT_PREFIX}/include")
+endif()
+
+find_package(Threads REQUIRED)
+find_package(Protobuf CONFIG QUIET)
+if(NOT Protobuf_FOUND)
+    find_package(Protobuf REQUIRED)
+endif()
+find_package(gRPC CONFIG QUIET)
+if(NOT gRPC_FOUND)
+    # Reached only on distros whose grpc++ packaging ships no CMake config.
+    # Ubuntu's libgrpc-dev does ship one, so this is dead code on LocalAI's own
+    # build distro. Kept for the distros that do not.
+    find_library(GRPCPP_LIB grpc++ REQUIRED)
+    find_library(GRPCPP_REFLECTION_LIB grpc++_reflection REQUIRED)
+    add_library(gRPC::grpc++ INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++ PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_LIB}")
+    add_library(gRPC::grpc++_reflection INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++_reflection PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_REFLECTION_LIB}")
+endif()
+
+find_program(_PROTOC NAMES protoc REQUIRED)
+find_program(_GRPC_CPP_PLUGIN NAMES grpc_cpp_plugin REQUIRED)
+
+get_filename_component(HW_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/../../backend.proto" ABSOLUTE)
+get_filename_component(HW_PROTO_PATH "${HW_PROTO}" PATH)
+
+set(HW_PROTO_SRCS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.cc")
+set(HW_PROTO_HDRS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.h")
+set(HW_GRPC_SRCS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.cc")
+set(HW_GRPC_HDRS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.h")
+
+add_custom_command(
+    OUTPUT "${HW_PROTO_SRCS}" "${HW_PROTO_HDRS}" "${HW_GRPC_SRCS}" "${HW_GRPC_HDRS}"
+    COMMAND ${_PROTOC}
+    ARGS --grpc_out "${CMAKE_CURRENT_BINARY_DIR}"
+         --cpp_out  "${CMAKE_CURRENT_BINARY_DIR}"
+         -I "${HW_PROTO_PATH}"
+         --plugin=protoc-gen-grpc="${_GRPC_CPP_PLUGIN}"
+         "${HW_PROTO}"
+    DEPENDS "${HW_PROTO}")
+
+add_library(hw_grpc_proto STATIC
+    ${HW_GRPC_SRCS} ${HW_GRPC_HDRS}
+    ${HW_PROTO_SRCS} ${HW_PROTO_HDRS})
+target_include_directories(hw_grpc_proto PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
+# Required on macOS: without these the Homebrew protobuf/grpc include dirs never
+# reach this target and google/protobuf/runtime_version.h is not found.
+target_link_libraries(hw_grpc_proto PUBLIC protobuf::libprotobuf gRPC::grpc++)
+
+# TWO PROTOBUF RUNTIMES IN ONE BINARY, AND THE ONE THAT WON WAS THE WRONG ONE.
+#
+# engine_runtime links sentencepiece, whose default SPM_PROTOBUF_PROVIDER
+# ("internal") builds the protobuf-lite 3.14.0 sources vendored under
+# external/sentencepiece/third_party/protobuf-lite. Our generated backend.pb.cc
+# is compiled against the toolchain's protobuf 3.21.12 headers and links
+# libprotobuf.so 3.21.12. Both used to end up in the executable: 476
+# google::protobuf:: symbols from that archive, 278 of them also defined by
+# libprotobuf.so.
+#
+# The binding is decided at STATIC LINK time. Once ld pulls a sentencepiece
+# member in for sentencepiece's own code, that member's protobuf definitions are
+# in the executable and references from libhw_grpc_proto.a bind to them. Do NOT
+# reach for -Wl,--exclude-libs: it flips those symbols to LOCAL in .dynsym and
+# the breakage is unchanged, because no visibility flag revisits a static
+# binding already made.
+#
+# What broke, measured rather than assumed:
+# google::protobuf::internal::ParseContext::ParseMessage(MessageLite*, const char*)
+# is what every generated _InternalParse calls for a SUBMESSAGE field and for
+# nothing else. Bound to the 3.14 definition it fails, so a flat message parsed
+# and every nested one did not: a TranscriptResult carrying segments serialized
+# to correct bytes that the same process could not read back, and
+# TranscriptLiveRequest, a oneof of submessages, could not have been parsed at
+# all. 3.21 generated code was also running 3.14 arena, ArenaStringPtr and
+# ExtensionSet code, which is an ABI mismatch rather than a missing feature, so
+# "not observed to bite yet" was never a reason to leave it.
+#
+# "package" makes sentencepiece use the protobuf found above, which is the one
+# the generated code was built against. It must be set before add_subdirectory,
+# since that is when sentencepiece's own cache entry is created.
+#
+# WHAT THAT BUYS IS ONE PROTOBUF RUNTIME, not an executable free of protobuf
+# symbols, and the difference matters to whoever checks this next. Measured with
+# nm -C --defined-only on the linked grpc-server, 2515 google::protobuf::
+# symbols are still DEFINED in it, and that is what should be there: they are
+# generated code, sentencepiece::ModelProto's own _InternalParse and
+# CheckTypeAndMergeFrom among them, which name protobuf types in their
+# signatures and are compiled into every user of a .proto. Expecting zero would
+# send a reader looking for a regression that is not one.
+#
+# The claim that decides whether the ABI mismatch above is gone is the RUNTIME
+# one, and it holds: google::protobuf::internal::ParseContext::ParseMessage is
+# UNDEFINED in the executable, so every generated _InternalParse resolves it to
+# libprotobuf.so at load instead of to a vendored 3.14 copy. No vendored
+# protobuf-lite archive is pulled in at all, and citrinet_asr, which parses a
+# SentencePiece ModelProto at load, tokenizes correctly as a result.
+set(SPM_PROTOBUF_PROVIDER "package" CACHE STRING
+    "Make sentencepiece use the found protobuf, not its vendored 3.14 copy" FORCE)
+
+# Upstream's global add_compile_options(-Wall -Wextra -Wpedantic -pedantic-errors)
+# is a directory property of the subdirectory and does not reach our targets.
+#
+# EXCLUDE_FROM_ALL is load-bearing, do not drop it: upstream's default target set
+# includes its CLI, server, converter and test binaries, none of which we ship.
+# Without it every build would compile all of them. The targets we do name in
+# target_link_libraries below are still built on demand, so nothing is lost.
+add_subdirectory("${AUDIO_CPP_DIR}" "${CMAKE_CURRENT_BINARY_DIR}/audio-cpp" EXCLUDE_FROM_ALL)
+
+add_executable(${TARGET}
+    grpc-server.cpp
+    model_options.cpp
+    capability_routing.cpp
+    family_gate.cpp
+    loaded_model.cpp
+    audio_io.cpp
+    audio_units.cpp
+    transcript_assembly.cpp
+    result_map.cpp
+    stem_selection.cpp
+    generation_request.cpp
+    stream_delta.cpp
+    wav_header.cpp
+    inference_lane.cpp
+    live_watchdog.cpp
+)
+
+# Two files carry a switch over an enum with no `default:` label, deliberately,
+# so that -Wswitch reports an enumerator nobody handled. -Wswitch is only a
+# warning by default, and a warning in a 600-file build log is a warning nobody
+# reads, so it is promoted to an error on exactly these two translation units.
+# Not project-wide: upstream's own sources are not held to this, and they are
+# where the churn is.
+#
+# loaded_model.cpp mirrors engine::runtime::VoiceTaskKind onto its own Task enum.
+# Its static_asserts catch an insertion or a reorder, but an enumerator APPENDED
+# after the last one shifts no value, so no assertion can see it. What does see
+# it is from_engine_task's switch over the engine enum. This is the difference
+# between a build failure and a backend that silently runs the wrong task.
+#
+# capability_routing.cpp's unsupported_surface() switches UnsupportedRpc onto the
+# row of unsupported_surfaces() that explains it. Left as a warning, a sixth
+# enumerator added without a row BUILDS AND SHIPS, and its trailing
+# `return surfaces[0];` then answers the new RPC with AudioEncode's codec reason:
+# a confident, specific and false statement about audio.cpp, on the wire, on the
+# one code path whose entire job is to be truthful about what this backend
+# cannot do. Verified rather than assumed: adding a sixth enumerator and building
+# the shipping target produced exit 0, a binary, and one warning. A compile-time
+# check is the better trade than the runtime fallback it replaced only if it is
+# fatal, so here it is fatal.
+if(NOT MSVC)
+    set_source_files_properties(loaded_model.cpp capability_routing.cpp
+        PROPERTIES COMPILE_OPTIONS "-Werror=switch")
+endif()
+
+target_include_directories(${TARGET} PRIVATE
+    "${AUDIO_CPP_DIR}/include"
+    "${CMAKE_CURRENT_SOURCE_DIR}")
+
+# The shipping binary is held to the same bar as the tests below. Upstream's own
+# add_compile_options is a property of its directory and never reached this
+# target, so until now "the build was clean" meant only that nothing was being
+# checked.
+if(NOT MSVC)
+    target_compile_options(${TARGET} PRIVATE -Wall -Wextra -Wpedantic)
+endif()
+
+target_link_libraries(${TARGET} PRIVATE
+    hw_grpc_proto
+    engine_runtime
+    ggml
+    gRPC::grpc++
+    gRPC::grpc++_reflection
+    protobuf::libprotobuf
+    Threads::Threads)
+
+# ENGINE_ENABLE_CPU_ALL_VARIANTS builds ggml backends as shared objects that sit
+# next to the binary in the package, so the binary must search its own directory.
+# BUILD_WITH_INSTALL_RPATH keeps the build-tree binary at exactly "$ORIGIN".
+# Upstream sets CMAKE_BUILD_WITH_INSTALL_RPATH in its own directory scope, which
+# does not reach ours, so without this CMake also appends its build-tree library
+# directory. That absolute build-host path would survive into the copied binary
+# and let a package.sh that forgot to bundle libggml*.so still pass on the build
+# machine while failing everywhere else.
+set_target_properties(${TARGET} PROPERTIES
+    BUILD_RPATH "$ORIGIN"
+    INSTALL_RPATH "$ORIGIN"
+    BUILD_WITH_INSTALL_RPATH TRUE)
+
+if(AUDIO_CPP_GRPC_BUILD_TESTS)
+    enable_testing()
+
+    # These are the units whose tests CANNOT run under
+    # backend/cpp/run-unit-tests.sh, because that script compiles each
+    # *_test.cpp standalone with no protobuf and no audio.cpp include path.
+    # They are named *_ctest.cpp so the script's glob does not pick them up and
+    # fail every backend's suite; everything that can be stdlib-only still is,
+    # and still lives in a *_test.cpp beside its unit.
+    add_executable(result_map_ctest
+        result_map_ctest.cpp
+        result_map.cpp
+        transcript_assembly.cpp
+        audio_units.cpp)
+    target_include_directories(result_map_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h. Every other target
+        # here inherits that directory from the ggml target it links; this one
+        # links no ggml, so it has to name it.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    # No engine_runtime: result_map touches only the plain structs in
+    # engine/framework/runtime/session.h, so the header is all it needs.
+    target_link_libraries(result_map_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(result_map_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME result_map COMMAND result_map_ctest)
+
+    # Same shape as result_map_ctest: generation_request touches only the plain
+    # structs in engine/framework/runtime/session.h plus the generated protobuf
+    # messages, so the headers are all it needs and no engine_runtime is linked.
+    add_executable(generation_request_ctest
+        generation_request_ctest.cpp
+        generation_request.cpp)
+    target_include_directories(generation_request_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h, and this target links
+        # no ggml, so it has to name the include directory itself.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    target_link_libraries(generation_request_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(generation_request_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME generation_request COMMAND generation_request_ctest)
+
+    add_executable(audio_io_ctest
+        audio_io_ctest.cpp
+        audio_io.cpp)
+    target_include_directories(audio_io_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(audio_io_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(audio_io_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    set_target_properties(audio_io_ctest PROPERTIES
+        BUILD_RPATH "$ORIGIN"
+        INSTALL_RPATH "$ORIGIN"
+        BUILD_WITH_INSTALL_RPATH TRUE)
+    add_test(NAME audio_io COMMAND audio_io_ctest)
+
+    # The streaming drivers live in loaded_model.cpp, which links the engine, so
+    # this cannot be a standalone *_test.cpp. It builds no model and reads no
+    # file: LoadedModel::Session is a plain struct holding a pointer to an
+    # engine interface, so the drivers are exercised against fake sessions.
+    add_executable(streaming_driver_ctest
+        streaming_driver_ctest.cpp
+        loaded_model.cpp
+        capability_routing.cpp
+        family_gate.cpp
+        model_options.cpp
+        inference_lane.cpp)
+    target_include_directories(streaming_driver_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(streaming_driver_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(streaming_driver_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    # No "$ORIGIN" rpath override here, unlike the shipping target and unlike
+    # audio_io_ctest. loaded_model.cpp reaches make_default_registry, so this
+    # binary genuinely links libggml, and CMake's own build-tree rpath is what
+    # finds it: the ggml shared objects land in ${CMAKE_CURRENT_BINARY_DIR}/bin
+    # while the test binary sits one directory up. A build-host absolute path in
+    # a test binary is harmless, since package.sh ships only grpc-server, and
+    # forcing "$ORIGIN" here means ctest cannot start the binary at all.
+    add_test(NAME streaming_driver COMMAND streaming_driver_ctest)
+
+    # Asserts that the upstream ABSENCES capability_routing.cpp's refusal
+    # messages rest on are still absences, by querying make_default_registry()
+    # rather than by re-reading upstream. This is what makes an AUDIO_CPP_VERSION
+    # bump that adds a codec task kind, an spk family or a streaming converter
+    # fail the build instead of leaving a false statement on the wire.
+    #
+    # It links engine_runtime purely to run that query, which is why it lives
+    # here rather than with the standalone *_test.cpp files, and it needs no
+    # "$ORIGIN" rpath override for the same reason streaming_driver_ctest does
+    # not: see the note above.
+    add_executable(upstream_absence_ctest upstream_absence_ctest.cpp)
+    target_include_directories(upstream_absence_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(upstream_absence_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(upstream_absence_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME upstream_absence COMMAND upstream_absence_ctest)
+endif()

backend/cpp/audio-cpp/Makefile

@@ -0,0 +1,172 @@
+# audio.cpp backend Makefile.
+#
+# Upstream pin lives below in the AUDIO_CPP_VERSION variable, so
+# .github/bump_deps.sh can find and update it, matching the llama-cpp / ds4
+# convention. That script seds every line matching the variable name followed by
+# an assignment, so this comment deliberately spells the name on its own: a
+# comment repeating the full assignment token gets rewritten and mangled by the
+# first auto-bump (backend/cpp/ds4/Makefile shows the damage). The clone
+# recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
+# rebuild and so the bump bot can see the pin.
+
+AUDIO_CPP_VERSION?=545e29a6f2fde24298cb3b0f07baab4352987ac9
+AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
+
+CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+BUILD_DIR := build
+
+BUILD_TYPE ?=
+NATIVE ?= false
+JOBS ?= $(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
+
+UNAME_S := $(shell uname -s)
+
+# AUDIOCPP_DEPLOYMENT_BUILD compiles the model_specs/*.json catalog into
+# engine_runtime, so the shipped package needs no model_specs directory and a
+# safetensors model tree still resolves its family spec.
+CMAKE_ARGS ?= -DCMAKE_BUILD_TYPE=Release -DAUDIOCPP_DEPLOYMENT_BUILD=ON
+
+# CMAKE_CUDA_ARCHITECTURES must be set explicitly for a cublas build, and this
+# is not a tuning knob: upstream's CMakeLists sets CUDA_ARCHITECTURES to
+# `native` on the engine_runtime target whenever the root-scope variable is
+# unset (audio.cpp/CMakeLists.txt, the `if (CMAKE_CUDA_ARCHITECTURES)` branch
+# next to the istft/torch_random .cu sources), and docs/build/linux.md says so
+# outright: "Leave CMAKE_CUDA_ARCHITECTURES unset to build for the GPUs present
+# at build time (native)". No CI runner has a GPU, so `native` has nothing to
+# enumerate. ggml's own default (external/ggml/src/ggml-cuda/CMakeLists.txt)
+# does not rescue this: it list(APPEND)s in the ggml subdirectory scope, which
+# never reaches the root scope where the engine_runtime property is decided.
+#
+# The values below are ggml's list for the matching toolkit, copied rather than
+# invented, so the two targets compile for exactly the same set:
+#   - CUDA 13 drops the Maxwell/Pascal/Volta virtual archs (50/61/70).
+#   - 121a-real needs CUDA >= 12.9, so the CUDA 12 list (built against 12.8)
+#     stops at 120a-real.
+#   - `a`-suffixed archs are used rather than ggml's rejected 120f-virtual: the
+#     `f` suffix needs CMake >= 3.31.8, and Ubuntu Noble ships 3.28.3. The
+#     3.28 validator (Modules/Internal/CMakeCUDAArchitecturesValidate.cmake)
+#     accepts `[0-9]+a?(-real|-virtual)?`.
+#
+# Setting it here also pins ggml's copy, since its default is guarded by
+# `if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES)`. CUDA_MAJOR_VERSION is the CI
+# build-arg, forwarded by Dockerfile.audio-cpp.
+#
+# An EMPTY major maps to `native`, NOT to the CUDA 12 list. Only CI declares a
+# major; a developer running `BUILD_TYPE=cublas make` locally declares none, and
+# the CUDA 12 list contains 120a-real, which needs nvcc >= 12.8. Falling through
+# to it turned every local build on a CUDA 12.0-12.7 host into a compile error,
+# where upstream's documented behaviour ("Leave CMAKE_CUDA_ARCHITECTURES unset
+# to build for the GPUs present at build time") worked. `native` restores that.
+# It does require a GPU to enumerate, so the escape hatch for a GPU-less local
+# cross-build is to set CUDA_ARCHITECTURES on the command line, which the ?=
+# assignments below leave untouched.
+CUDA_MAJOR_VERSION ?=
+ifeq ($(CUDA_MAJOR_VERSION),13)
+    CUDA_ARCHITECTURES ?= 75-virtual;80-virtual;86-real;89-real;120a-real;121a-real
+else ifeq ($(CUDA_MAJOR_VERSION),12)
+    CUDA_ARCHITECTURES ?= 50-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-real;89-real;120a-real
+else ifeq ($(CUDA_MAJOR_VERSION),)
+    CUDA_ARCHITECTURES ?= native
+else ifeq ($(BUILD_TYPE),cublas)
+    # Gated on cublas because the variable means nothing to any other build, so a
+    # stray CUDA_MAJOR_VERSION in the environment must not break `make clean` or
+    # a CPU build. It does still error for `BUILD_TYPE=cublas make clean`, which
+    # is the right trade: that invocation is asking about a CUDA build tree.
+    $(error CUDA_MAJOR_VERSION=$(CUDA_MAJOR_VERSION) has no architecture list here (12 and 13 do). Leave it empty for a native build, or pass CUDA_ARCHITECTURES explicitly.)
+endif
+
+ifeq ($(BUILD_TYPE),cublas)
+    CMAKE_ARGS += -DENGINE_ENABLE_CUDA=ON "-DCMAKE_CUDA_ARCHITECTURES=$(CUDA_ARCHITECTURES)"
+else ifeq ($(BUILD_TYPE),vulkan)
+    CMAKE_ARGS += -DENGINE_ENABLE_VULKAN=ON
+else ifeq ($(UNAME_S),Darwin)
+    # Metal. ggml embeds the shader library by default (GGML_METAL_EMBED_LIBRARY
+    # defaults to GGML_METAL), so the package needs no .metallib beside the
+    # binary. Darwin builds go through scripts/build/audio-cpp-darwin.sh.
+    CMAKE_ARGS += -DENGINE_ENABLE_METAL=ON
+    # AppleClang ships no OpenMP runtime and Homebrew's libomp is keg-only, so
+    # neither libomp.dylib nor omp.h is symlinked into /opt/homebrew and CMake's
+    # FindOpenMP cannot find them on its own (the workflow's `brew link libomp`
+    # is a no-op for a keg-only formula, and its failure is swallowed).
+    # audio.cpp calls find_package(OpenMP REQUIRED COMPONENTS CXX) whenever
+    # ENGINE_ENABLE_OPENMP is ON, so with no hint the macOS build dies at
+    # configure time before compiling anything. OpenMP_ROOT is honoured by the
+    # find_library/find_path calls inside FindOpenMP under CMP0074, which is NEW
+    # here because audio.cpp requires CMake 3.20.
+    #
+    # If the keg is absent, turn OpenMP off rather than fail: the tree's only
+    # <omp.h> include is guarded by #ifdef _OPENMP and a #pragma omp without
+    # -fopenmp is simply ignored, so an OpenMP-less build is CORRECT. It is not
+    # cheap, though: 108 `#pragma omp` directives across ~30 files (roformer,
+    # demucs, chatterbox, moss, supertonic, seed_vc, framework/audio/dsp) are
+    # compiled out, and clang says nothing about an ignored omp pragma unless
+    # -Wsource-uses-openmp is on. A green package that is quietly single-threaded
+    # in every host DSP loop gets blamed on Metal, not on packaging, so the
+    # fallback announces itself.
+    ifeq ($(origin LIBOMP_PREFIX),undefined)
+        LIBOMP_PREFIX := $(shell brew --prefix libomp 2>/dev/null)
+    endif
+    # Nested ifneq rather than $(and ...): $(and) needs GNU make 3.81, and while
+    # that is what Apple ships, an older make expands it to empty and would take
+    # the OpenMP-OFF branch with no way to tell that from a genuinely missing
+    # keg. Two plain conditionals cannot fail that way.
+    LIBOMP_USABLE :=
+    ifneq ($(wildcard $(LIBOMP_PREFIX)/lib/libomp.dylib),)
+        ifneq ($(wildcard $(LIBOMP_PREFIX)/include/omp.h),)
+            LIBOMP_USABLE := yes
+        endif
+    endif
+    ifeq ($(LIBOMP_USABLE),yes)
+        CMAKE_ARGS += "-DOpenMP_ROOT=$(LIBOMP_PREFIX)"
+    else
+        $(warning audio-cpp: libomp not found at '$(LIBOMP_PREFIX)'; building without OpenMP (single-threaded host DSP). Install it with `brew install libomp`, or set LIBOMP_PREFIX.)
+        CMAKE_ARGS += -DENGINE_ENABLE_OPENMP=OFF
+    endif
+else
+    # Portable Linux CPU. Upstream wires this to GGML_BACKEND_DL +
+    # GGML_CPU_ALL_VARIANTS + $ORIGIN rpath, so one build serves every CPU
+    # tier instead of an AVX-tier image fan-out.
+    CMAKE_ARGS += -DENGINE_ENABLE_CPU_ALL_VARIANTS=ON
+endif
+
+ifneq ($(NATIVE),true)
+    CMAKE_ARGS += -DENGINE_ENABLE_NATIVE_CPU=OFF
+endif
+
+.PHONY: all grpc-server package test test-engine clean purge
+all: grpc-server
+
+# Clone the upstream source at the pinned commit. The directory is the target
+# so make only re-clones when it is missing. After bumping AUDIO_CPP_VERSION,
+# run 'make purge && make' to refetch.
+audio.cpp:
+	mkdir -p audio.cpp
+	cd audio.cpp && \
+	git init -q && \
+	git remote add origin $(AUDIO_CPP_REPO) && \
+	git fetch --depth 1 origin $(AUDIO_CPP_VERSION) && \
+	git checkout FETCH_HEAD
+
+grpc-server: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS)
+	cp $(BUILD_DIR)/grpc-server grpc-server
+
+package: grpc-server
+	bash package.sh
+
+test:
+	@echo "audio-cpp: standalone unit tests run from the repo root via 'make test-backend-cpp'"
+
+# Engine-linked tests. Needs the upstream checkout and a full engine build.
+test-engine: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) -DAUDIO_CPP_GRPC_BUILD_TESTS=ON $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS) && ctest --output-on-failure --no-tests=error
+
+clean:
+	rm -rf $(BUILD_DIR) grpc-server package
+
+purge: clean
+	rm -rf audio.cpp

backend/cpp/audio-cpp/audio_io.cpp

@@ -0,0 +1,124 @@
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include "engine/framework/audio/conversion.h"
+#include "engine/framework/audio/wav_reader.h"
+#include "engine/framework/audio/wav_writer.h"
+
+#include <filesystem>
+#include <string>
+#include <utility>
+
+namespace audiocpp_backend {
+
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no input audio path was supplied");
+    }
+    std::error_code ec;
+    const bool present = std::filesystem::exists(std::filesystem::path(path), ec);
+    if (ec) {
+        // exists() returning false with ec set does NOT mean the file is
+        // absent, it means the question could not be answered: most often a
+        // parent directory is not searchable. Reporting that as "does not
+        // exist" sends the operator after the file when the fault is the
+        // permissions on the directory above it.
+        throw ConfigError("audio-cpp: cannot stat input audio " + path + ": " +
+                          ec.message());
+    }
+    if (!present) {
+        throw ConfigError("audio-cpp: input audio does not exist: " + path);
+    }
+    engine::audio::WavData wav;
+    try {
+        wav = engine::audio::read_wav_f32(std::filesystem::path(path));
+    } catch (const std::exception &err) {
+        throw ConfigError("audio-cpp: cannot read " + path +
+                          " as WAV: " + err.what());
+    }
+    if (wav.sample_rate <= 0) {
+        throw ConfigError("audio-cpp: " + path +
+                          " declares a non-positive sample rate; every "
+                          "timestamp derived from it would be zero");
+    }
+    // AudioBuffer's own default is 1, and a reader that reports 0 channels
+    // still gave us an interleaving of one. Normalised before the conversion
+    // below rather than after, because mixdown_interleaved_to_mono_average
+    // throws on a non-positive channel count.
+    if (wav.channels <= 0) {
+        wav.channels = 1;
+    }
+
+    engine::runtime::AudioBuffer buffer;
+    if (target_sample_rate <= 0) {
+        buffer.sample_rate = wav.sample_rate;
+        buffer.channels = wav.channels;
+        buffer.samples = std::move(wav.samples);
+        return buffer;
+    }
+
+    buffer.sample_rate = target_sample_rate;
+    buffer.channels = 1;
+    try {
+        // A no-op copy when the rates already match, so the common 16 kHz
+        // upload pays only the mono mixdown it would have paid inside the
+        // family anyway.
+        buffer.samples =
+            engine::audio::convert_wav_to_mono_linear_resampled(wav, target_sample_rate);
+    } catch (const std::exception &err) {
+        // ConfigError, so this is INVALID_ARGUMENT rather than INTERNAL. What
+        // reaches here is a malformed input: a sample count that is not a whole
+        // number of frames is the realistic one, and it is the uploader's file
+        // that is truncated, not this backend that is broken.
+        throw ConfigError("audio-cpp: cannot resample " + path + " from " +
+                          std::to_string(wav.sample_rate) + " Hz to " +
+                          std::to_string(target_sample_rate) +
+                          " Hz: " + err.what());
+    }
+    return buffer;
+}
+
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no output path was supplied");
+    }
+    const std::filesystem::path destination(path);
+    if (destination.has_parent_path()) {
+        // Best effort: a failure here shows up as a write failure below, with a
+        // message naming the file the caller actually asked for.
+        std::error_code ec;
+        std::filesystem::create_directories(destination.parent_path(), ec);
+    }
+    try {
+        engine::audio::write_pcm16_wav(destination, audio.sample_rate,
+                                       audio.channels > 0 ? audio.channels : 1,
+                                       audio.samples);
+    } catch (const std::exception &err) {
+        // NOT a ConfigError, and the distinction is not cosmetic. The
+        // destination is chosen by LocalAI rather than by the caller: it is a
+        // unique name inside GeneratedContentDir. A failure to write it is a
+        // full disk, a permission fault on the server's own directory, or a bad
+        // mount, none of which the caller can fix or is to blame for. As a
+        // ConfigError this surfaced as INVALID_ARGUMENT, which tells a client
+        // its request was wrong and not to retry; a plain runtime_error maps to
+        // INTERNAL, which is both true and retryable. The empty path above
+        // stays INVALID_ARGUMENT, because that one really is a malformed
+        // request.
+        throw std::runtime_error("audio-cpp: cannot write " + path + ": " +
+                                 err.what());
+    }
+}
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = 1;
+    buffer.samples = std::move(samples);
+    return buffer;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io.h

@@ -0,0 +1,61 @@
+#pragma once
+
+// Thin wrappers over the framework's public audio IO. Engine-linked, so this
+// unit is built and tested through the CMake target rather than by
+// backend/cpp/run-unit-tests.sh. The pure part of the arithmetic these
+// wrappers feed lives in audio_units, which is stdlib-only and does have a
+// standalone test.
+
+#include "engine/framework/runtime/session.h"
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Reads a WAV file. Throws ConfigError when the file is missing, is not
+// readable as WAV, or declares a non-positive sample rate: all three are
+// user-fixable input problems rather than backend faults.
+//
+// A declared sample rate of zero is refused rather than passed on, because
+// every downstream conversion in audio_units answers 0 for a non-positive rate.
+// Accepting it would turn a corrupt header into a response full of zero
+// timestamps, which reads as a real answer.
+//
+// `target_sample_rate` is the rate the CALLER needs, in Hz:
+//
+//   0 (or negative)  keep the file's own rate and channel count.
+//   positive         downmix to mono and resample to that rate. Resampling is
+//                    skipped when the file already declares it, so passing the
+//                    rate a route needs costs nothing on the common input.
+//
+// It is a parameter, and not a constant inside this function, because the
+// routes that read audio do not agree on an answer. Speech routes want 16 kHz
+// mono; source separation does not, and folding a 44.1 kHz stereo input to
+// 16 kHz mono for demucs or roformer would destroy the very thing they separate
+// (both refuse a rate other than their own outright). Making the caller name
+// the rate keeps that decision where the route is known.
+//
+// Downmixing along with the resample is not an extra liberty: every family a
+// positive rate is used for (silero_vad, sortformer_diar and every ASR family)
+// begins by calling the same mixdown_interleaved_to_mono_average on whatever it
+// is given. Doing it once here produces the identical samples and halves the
+// buffer that is then moved through the request.
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate);
+
+// Writes 16-bit PCM WAV, creating parent directories.
+//
+// Throws ConfigError, i.e. INVALID_ARGUMENT, ONLY for an empty path, which is a
+// malformed request. Every other failure throws a plain runtime_error, i.e.
+// INTERNAL: the destination is LocalAI's own generated-content directory and
+// not anything the caller named, so a full disk or a permission fault there is
+// a server fault and is worth retrying, which is the opposite of what
+// INVALID_ARGUMENT tells a client.
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio);
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io_ctest.cpp

@@ -0,0 +1,217 @@
+// Tests for audio_io's reading contract, and in particular for the resampling
+// that keeps a 44.1 or 48 kHz upload from reaching a family that only accepts
+// 16 kHz.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE: see the note at the top of
+// result_map_ctest.cpp. This file links the audio.cpp engine, so it is built
+// and run by ctest, not by backend/cpp/run-unit-tests.sh.
+//
+//   make -C backend/cpp/audio-cpp test-engine
+
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include <algorithm>
+#include <cmath>
+#include <cstdio>
+#include <filesystem>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// A one-second tone, interleaved across `channels`. Real audio rather than
+// silence so a resample that dropped its input would be visible as a flat
+// buffer, not just as a different length.
+static engine::runtime::AudioBuffer tone(int sample_rate, int channels,
+                                         float seconds) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = channels;
+    const auto frames =
+        static_cast<size_t>(static_cast<double>(sample_rate) * seconds);
+    buffer.samples.reserve(frames * static_cast<size_t>(channels));
+    for (size_t frame = 0; frame < frames; ++frame) {
+        const float value = 0.5f * std::sin(2.0f * 3.14159265f * 220.0f *
+                                            static_cast<float>(frame) /
+                                            static_cast<float>(sample_rate));
+        for (int channel = 0; channel < channels; ++channel) {
+            buffer.samples.push_back(value);
+        }
+    }
+    return buffer;
+}
+
+static float peak(const std::vector<float> &samples) {
+    float highest = 0.0f;
+    for (const float sample : samples) {
+        highest = std::max(highest, std::abs(sample));
+    }
+    return highest;
+}
+
+static std::filesystem::path scratch_dir() {
+    const auto dir = std::filesystem::temp_directory_path() / "audiocpp-io-ctest";
+    std::filesystem::create_directories(dir);
+    return dir;
+}
+
+// The I2 fixture. Before the resample this returned a 44.1 kHz buffer, which
+// silero_vad and sortformer_diar both reject with a plain runtime_error, which
+// the server maps to INTERNAL. A 44.1 kHz WAV is an ordinary upload.
+static void test_441k_stereo_is_read_as_16k_mono() {
+    const auto path = scratch_dir() / "input-44100-stereo.wav";
+    write_audio_file(path.string(), tone(44100, 2, 1.0f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "44.1 kHz input is resampled to 16 kHz");
+    check(audio.channels == 1, "stereo input is downmixed to mono");
+    // Linear resampling lands within a sample or two of the exact ratio.
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 15990 && frames < 16010,
+          "one second in stays one second out");
+    check(peak(audio.samples) > 0.2f,
+          "the resampled buffer still carries the signal");
+}
+
+static void test_48k_is_read_as_16k() {
+    const auto path = scratch_dir() / "input-48000-mono.wav";
+    write_audio_file(path.string(), tone(48000, 1, 0.5f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "48 kHz input is resampled to 16 kHz");
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 7990 && frames < 8010, "half a second in, half a second out");
+}
+
+// The common case: the upload is already 16 kHz mono, and nothing is resampled.
+static void test_16k_mono_passes_through_unchanged() {
+    const auto path = scratch_dir() / "input-16000-mono.wav";
+    const auto source = tone(16000, 1, 1.0f);
+    write_audio_file(path.string(), source);
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "16 kHz stays 16 kHz");
+    check(audio.channels == 1, "mono stays mono");
+    check(audio.samples.size() == source.samples.size(),
+          "a matching rate resamples nothing");
+}
+
+// Rate 0 means "give me the file as it is", which is what a source separation
+// route needs: demucs and roformer refuse anything but their own 44.1 kHz and
+// work on stereo, so the reader must not force them to 16 kHz mono.
+static void test_zero_target_keeps_the_native_format() {
+    const auto path = scratch_dir() / "input-native.wav";
+    write_audio_file(path.string(), tone(44100, 2, 0.25f));
+
+    const auto audio = read_audio_file(path.string(), 0);
+    check(audio.sample_rate == 44100, "a zero target keeps the file's rate");
+    check(audio.channels == 2, "a zero target keeps the file's channels");
+}
+
+static void test_missing_file_is_a_config_error() {
+    bool threw_config_error = false;
+    try {
+        read_audio_file((scratch_dir() / "does-not-exist.wav").string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+        // Any other type maps to INTERNAL, which is what this asserts against.
+    }
+    check(threw_config_error, "a missing input file is INVALID_ARGUMENT, not INTERNAL");
+}
+
+static void test_unreadable_file_is_a_config_error() {
+    const auto path = scratch_dir() / "not-a-wav.wav";
+    {
+        FILE *file = fopen(path.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("this is not a RIFF header", file);
+            fclose(file);
+        }
+    }
+    bool threw_config_error = false;
+    try {
+        read_audio_file(path.string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "a non-WAV input is INVALID_ARGUMENT, not INTERNAL");
+}
+
+// The write side of the same distinction. The destination is LocalAI's own
+// generated-content directory, not a caller-supplied path, so a failure to
+// write it is a server fault: INTERNAL, which a client may retry, and not
+// INVALID_ARGUMENT, which tells it the request itself was wrong.
+static void test_write_failure_is_not_a_config_error() {
+    // A regular file where a directory has to be. ENOTDIR defeats root as well
+    // as an ordinary user, unlike a chmod, which CI running as root would walk
+    // straight through.
+    const auto blocker = scratch_dir() / "blocking-file";
+    {
+        FILE *file = fopen(blocker.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("not a directory", file);
+            fclose(file);
+        }
+    }
+    const auto path = blocker / "nested" / "out.wav";
+
+    bool threw_config_error = false;
+    bool threw_something = false;
+    try {
+        write_audio_file(path.string(), tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+        threw_something = true;
+    } catch (const std::exception &) {
+        threw_something = true;
+    }
+    check(threw_something, "an unwritable destination is reported at all");
+    check(!threw_config_error,
+          "a failed write is INTERNAL, not INVALID_ARGUMENT: the caller did not "
+          "choose the destination and cannot fix it");
+    check(!std::filesystem::exists(path), "and nothing was written");
+}
+
+static void test_empty_output_path_is_a_config_error() {
+    // The one write failure that IS the caller's: no path at all.
+    bool threw_config_error = false;
+    try {
+        write_audio_file("", tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "an empty output path stays INVALID_ARGUMENT");
+}
+
+int main() {
+    test_441k_stereo_is_read_as_16k_mono();
+    test_48k_is_read_as_16k();
+    test_16k_mono_passes_through_unchanged();
+    test_zero_target_keeps_the_native_format();
+    test_missing_file_is_a_config_error();
+    test_unreadable_file_is_a_config_error();
+    test_write_failure_is_not_a_config_error();
+    test_empty_output_path_is_a_config_error();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_io checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/audio_units.cpp

@@ -0,0 +1,139 @@
+#include "audio_units.h"
+
+#include <algorithm>
+#include <cmath>
+#include <limits>
+
+namespace audiocpp_backend {
+
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels) {
+    const std::size_t lanes = channels > 0 ? static_cast<std::size_t>(channels)
+                                           : static_cast<std::size_t>(1);
+    // Truncating division is deliberate: a trailing partial frame is not a
+    // position every channel reached, so counting it would overstate the length.
+    return static_cast<std::int64_t>(sample_count / lanes);
+}
+
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0;
+    }
+    // Split into whole seconds plus a remainder so the intermediate product
+    // cannot overflow on long recordings, and so rates like 44100 stay exact.
+    // The remainder division truncates deliberately: that matches Go's
+    // time.Duration conventions and keeps successive sample indices monotonic.
+    const std::int64_t rate = static_cast<std::int64_t>(sample_rate);
+    const std::int64_t whole_seconds = samples / rate;
+    const std::int64_t remainder = samples % rate;
+    return whole_seconds * 1000000000LL + (remainder * 1000000000LL) / rate;
+}
+
+float samples_to_seconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0.0f;
+    }
+    return static_cast<float>(static_cast<double>(samples) /
+                              static_cast<double>(sample_rate));
+}
+
+std::int64_t seconds_to_samples(double seconds, int sample_rate) {
+    // !(seconds > 0.0) rather than seconds <= 0.0: every comparison against NaN
+    // is false, so the <= form lets NaN reach the cast below, which is undefined
+    // behaviour and lands on INT64_MIN in practice. This is the one entry point
+    // fed by untrusted-shaped input (a float-seconds timestamp off the wire, or
+    // a boundary from a model that diverged), and a hugely negative sample index
+    // used later as an offset or a length is a wild pointer rather than merely a
+    // wrong timestamp.
+    if (sample_rate <= 0 || !(seconds > 0.0)) {
+        return 0;
+    }
+    const double scaled = seconds * static_cast<double>(sample_rate);
+    // Bound before the cast for the same reason: converting a double at or above
+    // 2^63 (infinity included) is undefined behaviour, so saturate instead.
+    const double limit =
+        static_cast<double>(std::numeric_limits<std::int64_t>::max());
+    if (scaled >= limit) {
+        return std::numeric_limits<std::int64_t>::max();
+    }
+    // Round rather than truncate: these functions exist to cross the float
+    // seconds boundary the VAD and diarize messages use, so a value that came
+    // from samples_to_seconds converts back to the sample it started as.
+    // Truncation lost one sample about half the time, starting at n=1.
+    //
+    // That round trip is exact only below roughly 2^23 samples. Past that the
+    // float samples_to_seconds returns can no longer resolve adjacent indices
+    // and the trip fails whatever the rounding. Both the first failing INDEX
+    // and the duration it stands for depend on the rate, so they are listed per
+    // rate rather than folded into one range; measured:
+    //
+    //   16 kHz    16384001 samples   17.1 min
+    //   44.1 kHz  11289602 samples    4.3 min
+    //   48 kHz    12288002 samples    4.3 min
+    //   96 kHz    12288002 samples    2.1 min
+    //
+    // The shortest recording this bites is therefore a couple of minutes of
+    // 96 kHz audio. It is a property of the float seconds API itself, not of
+    // the rounding here, and it is why nothing should use these to carry a
+    // sample-accurate position in a long recording.
+    return static_cast<std::int64_t>(std::llround(scaled));
+}
+
+std::vector<float> s16le_to_f32(const std::string &bytes) {
+    std::vector<float> samples;
+    const size_t count = bytes.size() / 2;
+    samples.reserve(count);
+    for (size_t i = 0; i < count; ++i) {
+        const auto low = static_cast<unsigned char>(bytes[i * 2]);
+        const auto high = static_cast<unsigned char>(bytes[i * 2 + 1]);
+        const auto raw = static_cast<std::int16_t>(
+            static_cast<std::uint16_t>(low) |
+            (static_cast<std::uint16_t>(high) << 8));
+        // 32768 on decode against 32767 on encode is deliberate, not a typo.
+        // 32768 is what keeps INT16_MIN at exactly -1.0 and every other code
+        // inside the [-1, 1] range this header promises; dividing by 32767
+        // would decode INT16_MIN to -1.00003. See f32_to_s16le for the other
+        // half of the pair. The cost is that a round trip shrinks a sample by
+        // 32767/32768, well under one LSB.
+        samples.push_back(static_cast<float>(raw) / 32768.0f);
+    }
+    return samples;
+}
+
+std::string f32_to_s16le(const std::vector<float> &samples) {
+    std::string bytes;
+    bytes.reserve(samples.size() * 2);
+    for (const float sample : samples) {
+        // NaN maps to silence. A NaN sample rendered as a full-scale click is
+        // worse audio than a dropped one, and this unit converts audio that may
+        // have originated off the wire.
+        //
+        // This guard also removes what used to be a spelling hazard in the
+        // clamp below. std::min and std::max return their first argument when
+        // the comparison is false, and every comparison against NaN is false,
+        // so before this branch existed the choice of spelling silently decided
+        // whether a NaN reached std::lround, whose result is unspecified for
+        // NaN. These three leaked it, the last being the idiomatic C++17 way to
+        // write a clamp and so the likeliest future edit:
+        //   std::min(std::max(sample, -1.0f), 1.0f)
+        //   std::max(std::min(sample, 1.0f), -1.0f)
+        //   std::clamp(sample, -1.0f, 1.0f)
+        // The order is no longer load-bearing now that the guard runs first,
+        // but the history is why the guard is here, so do not drop it.
+        if (std::isnan(sample)) {
+            bytes.push_back(0);
+            bytes.push_back(0);
+            continue;
+        }
+        const float clamped = std::max(-1.0f, std::min(1.0f, sample));
+        // 32767 rather than 32768 so +1.0 saturates at INT16_MAX instead of
+        // overflowing to INT16_MIN. See s16le_to_f32 for why decode differs.
+        const auto value =
+            static_cast<std::int16_t>(std::lround(clamped * 32767.0f));
+        const auto raw = static_cast<std::uint16_t>(value);
+        bytes.push_back(static_cast<char>(raw & 0xFF));
+        bytes.push_back(static_cast<char>((raw >> 8) & 0xFF));
+    }
+    return bytes;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units.h

@@ -0,0 +1,50 @@
+#pragma once
+
+// Time and sample-format conversion between audio.cpp's runtime types (sample
+// indices, float PCM) and LocalAI's proto types. Standard library only.
+//
+// LocalAI uses three different time units:
+//   TranscriptSegment / TranscriptWord start,end : int64 nanoseconds
+//   VADSegment start,end                         : float seconds
+//   DiarizeSegment start,end                     : float seconds
+
+#include <cstddef>
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Frames in an interleaved buffer of `sample_count` floats laid out across
+// `channels` channels. A frame is one per-channel position, which is the unit
+// every duration and every span boundary in this backend is expressed in, so a
+// stereo buffer must not report twice its real length: feeding sample_count
+// straight to samples_to_seconds makes a 3 second stereo clip come back as 6.
+//
+// A non-positive channel count is treated as mono, matching
+// engine::runtime::AudioBuffer's own default of 1 and keeping a reader that
+// reports 0 channels from dividing by zero.
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels);
+
+// Returns 0 when sample_rate is not positive rather than dividing by zero.
+// Uses integer arithmetic so 44.1 kHz does not lose precision.
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate);
+
+float samples_to_seconds(std::int64_t samples, int sample_rate);
+
+// Rounds to nearest. Negative seconds and NaN both yield 0, and a value too
+// large to convert saturates at INT64_MAX rather than overflowing. Round trips
+// with samples_to_seconds only below roughly 2^23 samples, past which the float
+// seconds can no longer resolve adjacent sample indices.
+std::int64_t seconds_to_samples(double seconds, int sample_rate);
+
+// Decodes little-endian signed 16-bit PCM. A trailing odd byte is dropped.
+std::vector<float> s16le_to_f32(const std::string &bytes);
+
+// Encodes to little-endian signed 16-bit PCM, clamping to [-1, 1] first so an
+// overshooting sample saturates instead of wrapping to the opposite sign.
+// A NaN sample encodes to 0, on the grounds that silence beats a full-scale
+// click.
+std::string f32_to_s16le(const std::vector<float> &samples);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units_test.cpp

@@ -0,0 +1,226 @@
+// Unit tests for audio_units. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+
+#include "audio_units.cpp"
+
+#include <cfenv>
+#include <cmath>
+#include <cstdio>
+#include <limits>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static bool close_to(float a, float b, float tol) { return std::fabs(a - b) <= tol; }
+
+using namespace audiocpp_backend;
+
+static void test_nanoseconds() {
+    // LocalAI TranscriptSegment/TranscriptWord times are nanoseconds
+    // (Go reads them as time.Duration).
+    check(samples_to_nanoseconds(16000, 16000) == 1000000000LL, "1s at 16k is 1e9 ns");
+    check(samples_to_nanoseconds(8000, 16000) == 500000000LL, "0.5s at 16k");
+    check(samples_to_nanoseconds(0, 16000) == 0, "zero samples is zero ns");
+    check(samples_to_nanoseconds(1000, 0) == 0, "zero sample rate yields zero, not UB");
+    // 44.1 kHz must not lose precision to float arithmetic.
+    check(samples_to_nanoseconds(44100, 44100) == 1000000000LL, "1s at 44.1k");
+    check(samples_to_nanoseconds(22050, 44100) == 500000000LL, "0.5s at 44.1k");
+    // The cases above all land on values a float happens to hold exactly, so
+    // they do not actually rule float arithmetic out. These do:
+    // a fraction that does not divide evenly, and a duration whose magnitude
+    // exceeds a float's 24-bit mantissa at nanosecond resolution.
+    check(samples_to_nanoseconds(44099, 44100) == 999977324LL,
+          "44.1k fraction is exact, not rounded through a float");
+    check(samples_to_nanoseconds(44100LL * 3600, 44100) == 3600000000000LL,
+          "one hour at 44.1k is exact to the nanosecond");
+    // A naive samples * 1e9 would overflow int64 here; the split into whole
+    // seconds plus a remainder is what keeps this correct.
+    check(samples_to_nanoseconds(44100LL * 360000, 44100) == 360000000000000LL,
+          "100 hours at 44.1k does not overflow");
+    // Double arithmetic is close enough to pass everything above, but still
+    // truncates this one a nanosecond short. Integer division does not.
+    check(samples_to_nanoseconds(4004, 8000) == 500500000LL,
+          "0.5005s at 8k is exact to the nanosecond");
+    // Truncation, not rounding: this matches Go's time.Duration conventions and
+    // keeps successive sample indices monotonic. The exact value here is
+    // 22675.7...; rounding to nearest would give 22676.
+    check(samples_to_nanoseconds(1, 44100) == 22675LL,
+          "a sub-nanosecond fraction truncates rather than rounding up");
+}
+
+static void test_seconds() {
+    check(close_to(samples_to_seconds(24000, 24000), 1.0f, 1e-6f), "1s at 24k");
+    check(close_to(samples_to_seconds(12000, 24000), 0.5f, 1e-6f), "0.5s at 24k");
+    check(close_to(samples_to_seconds(100, 0), 0.0f, 1e-6f), "zero sample rate is 0s");
+    check(seconds_to_samples(1.0, 16000) == 16000, "1s to samples at 16k");
+    check(seconds_to_samples(0.5, 16000) == 8000, "0.5s to samples at 16k");
+    check(seconds_to_samples(1.0, 0) == 0, "zero sample rate yields zero samples");
+    check(seconds_to_samples(-1.0, 16000) == 0, "negative seconds clamps to zero");
+
+    // seconds_to_samples is the one entry point fed by untrusted-shaped input:
+    // a float-seconds timestamp off the wire, or a VAD boundary from a model
+    // that diverged. A hugely negative sample index used later as an offset or
+    // a length is a wild pointer, not merely a wrong timestamp.
+    const double nan_seconds = std::numeric_limits<double>::quiet_NaN();
+    const double inf_seconds = std::numeric_limits<double>::infinity();
+    const std::int64_t max_samples = std::numeric_limits<std::int64_t>::max();
+    check(seconds_to_samples(nan_seconds, 16000) == 0, "NaN seconds yields zero");
+    check(seconds_to_samples(inf_seconds, 16000) == max_samples,
+          "infinite seconds saturates instead of overflowing");
+    check(seconds_to_samples(1e30, 16000) == max_samples,
+          "out of range seconds saturates instead of overflowing");
+    check(seconds_to_samples(-inf_seconds, 16000) == 0,
+          "negative infinity clamps to zero");
+
+    // Crossing the float-seconds boundary and back is the expected round trip
+    // for the VAD and diarize messages, so it must not lose a sample.
+    // Truncation loses one about half the time, starting at n=1.
+    check(seconds_to_samples(samples_to_seconds(1, 44100), 44100) == 1,
+          "one sample survives the seconds round trip at 44.1k");
+    check(seconds_to_samples(samples_to_seconds(1, 16000), 16000) == 1,
+          "one sample survives the seconds round trip at 16k");
+    check(seconds_to_samples(samples_to_seconds(4001, 8000), 8000) == 4001,
+          "4001 samples survive the seconds round trip at 8k");
+}
+
+static void test_s16le_round_trip() {
+    const std::vector<float> original = {0.0f, 0.5f, -0.5f, 1.0f, -1.0f};
+    const std::string encoded = f32_to_s16le(original);
+    check(encoded.size() == original.size() * 2, "two bytes per sample");
+
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == original.size(), "round trip keeps the sample count");
+    for (size_t i = 0; i < original.size(); ++i) {
+        // 16-bit quantisation: one LSB is ~3.05e-5. Guard the index so a short
+        // result reports a named failure instead of aborting the whole suite.
+        check(i < decoded.size() && close_to(decoded[i], original[i], 1e-4f),
+              "round trip preserves sample " + std::to_string(i));
+    }
+}
+
+static void test_s16le_endianness() {
+    // 0.5 encodes to 16384 = 0x4000, little endian is 0x00 0x40.
+    const std::string encoded = f32_to_s16le({0.5f});
+    check(encoded.size() == 2, "one sample is two bytes");
+    check(static_cast<unsigned char>(encoded[0]) == 0x00, "low byte first");
+    check(static_cast<unsigned char>(encoded[1]) == 0x40, "high byte second");
+}
+
+static void test_s16le_clamping() {
+    // Values outside [-1, 1] must clamp, not wrap around to the opposite sign.
+    const std::string encoded = f32_to_s16le({2.0f, -2.0f});
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == 2, "two samples survive clamping");
+    check(decoded.size() > 0 && decoded[0] > 0.99f,
+          "positive overshoot clamps to full scale");
+    check(decoded.size() > 1 && decoded[1] < -0.99f,
+          "negative overshoot clamps to full scale");
+}
+
+static void test_s16le_decode_range() {
+    // INT16_MIN is the one value that pins the decode scale. Dividing by 32767
+    // instead of 32768 would decode it to -1.00003, outside the [-1, 1] range
+    // the header promises, and every other test would still pass.
+    const std::vector<float> decoded = s16le_to_f32(std::string("\x00\x80", 2));
+    check(decoded.size() == 1, "INT16_MIN decodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == -1.0f,
+          "INT16_MIN decodes to exactly -1.0, not past full scale");
+}
+
+static void test_s16le_nan_input() {
+    // A NaN sample must not reach std::lround, whose result is unspecified for
+    // NaN. Asserting a range is not enough to pin this: the three outcomes the
+    // plausible clamp spellings produce (full scale, negative full scale, zero)
+    // are all finite and all inside [-1, 1], so a range check passes for every
+    // one of them. Only an exact value distinguishes them.
+    // NaN maps to silence, not to full scale: a NaN sample rendered as a
+    // full-scale click is worse audio than a dropped one, and this unit
+    // converts audio that may have originated off the wire.
+    //
+    // volatile so the NaN cannot be constant-folded, which would let the
+    // compiler evaluate the conversion at compile time and raise no
+    // floating-point exception at run time for the check below to observe.
+    volatile float nan_source = std::numeric_limits<float>::quiet_NaN();
+    const std::vector<float> input = {nan_source};
+
+    std::feclearexcept(FE_ALL_EXCEPT);
+    const std::string encoded = f32_to_s16le(input);
+    const bool raised_invalid = std::fetestexcept(FE_INVALID) != 0;
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+
+    check(decoded.size() == 1, "a NaN sample still encodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == 0.0f,
+          "a NaN sample encodes to exactly zero, not to a full-scale click");
+    // Independent of the value: a quiet NaN raises invalid-operation as soon as
+    // it reaches any ordered comparison, which is what std::min and std::max
+    // use, so this fails unless the NaN is diverted before the clamp runs at
+    // all. That is what stops the explicit guard from being dropped in favour
+    // of a clamp spelling that happens to yield zero.
+    check(!raised_invalid,
+          "encoding a NaN sample raises no invalid-operation exception");
+}
+
+static void test_s16le_odd_length() {
+    // A truncated frame must drop the dangling byte rather than read past it.
+    const std::string odd(5, '\0');
+    check(s16le_to_f32(odd).size() == 2, "odd byte count drops the trailing byte");
+    check(s16le_to_f32(std::string()).empty(), "empty input yields no samples");
+}
+
+static void test_interleaved_frame_count() {
+    // Mono is a pass-through, which is the only case the VAD path exercises.
+    check(interleaved_frame_count(16000, 1) == 16000, "mono frames equal samples");
+    // The case that matters: a stereo buffer holds two floats per position, so a
+    // one second 16 kHz stereo clip is 32000 floats and still one second. Handing
+    // the raw float count to samples_to_seconds reports two seconds instead.
+    check(interleaved_frame_count(32000, 2) == 16000,
+          "stereo frames are half the samples");
+    check(samples_to_seconds(interleaved_frame_count(32000, 2), 16000) == 1.0f,
+          "a one second stereo clip measures one second, not two");
+    check(interleaved_frame_count(48000, 3) == 16000,
+          "three channels divide by three");
+    // engine::runtime::AudioBuffer defaults channels to 1, but a reader is free
+    // to report 0, and dividing by that is undefined rather than merely wrong.
+    check(interleaved_frame_count(1000, 0) == 1000,
+          "zero channels is treated as mono");
+    check(interleaved_frame_count(1000, -2) == 1000,
+          "a negative channel count is treated as mono");
+    // A dangling partial frame is not a position every channel reached.
+    check(interleaved_frame_count(3, 2) == 1,
+          "a trailing partial frame is not counted");
+    check(interleaved_frame_count(0, 2) == 0, "an empty buffer has no frames");
+    // Past 2^32 floats, so a size_t narrowed to 32 bits on the way in, or a
+    // signed 32-bit intermediate, shows up here rather than in a multi-hour
+    // recording nobody tests with.
+    check(interleaved_frame_count(static_cast<std::size_t>(9000000000ULL), 2) ==
+              4500000000LL,
+          "a buffer beyond 2^32 floats counts frames without truncating");
+}
+
+int main() {
+    test_interleaved_frame_count();
+    test_nanoseconds();
+    test_seconds();
+    test_s16le_round_trip();
+    test_s16le_endianness();
+    test_s16le_clamping();
+    test_s16le_decode_range();
+    test_s16le_nan_input();
+    test_s16le_odd_length();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_units checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/capability_routing.cpp

@@ -0,0 +1,411 @@
+#include "capability_routing.h"
+
+#include <algorithm>
+
+namespace audiocpp_backend {
+namespace {
+
+struct NamedTask {
+    Task task;
+    const char *name;
+};
+
+// Short names are exactly the strings audio.cpp prints and parses in
+// framework/runtime/session.cpp, so a name pinned here survives conversion at
+// the engine boundary and a name copied out of audio.cpp is accepted here. All
+// thirteen have an upstream name; only "spk" is absent from the --task table in
+// docs/usage.md.
+const NamedTask kTaskNames[] = {
+    {Task::Vad, "vad"},
+    {Task::Asr, "asr"},
+    {Task::Diarization, "diar"},
+    {Task::SourceSeparation, "sep"},
+    {Task::AudioGeneration, "gen"},
+    {Task::Tts, "tts"},
+    {Task::VoiceCloning, "clon"},
+    {Task::VoiceConversion, "vc"},
+    {Task::SpeechToSpeech, "s2s"},
+    {Task::Alignment, "align"},
+    {Task::VoiceDesign, "vdes"},
+    {Task::SpeakerRecognition, "spk"},
+    {Task::Svc, "svc"},
+};
+
+// Accepted on input but never emitted. "spkrec" was this backend's own earlier
+// name for the kind; upstream only ever knew "spk".
+const NamedTask kTaskAliases[] = {
+    {Task::SpeakerRecognition, "spkrec"},
+};
+
+// First match wins, which is safe because a Capabilities value holds at most one
+// entry per task: it mirrors upstream runtime::TaskCapability (model.h), which
+// pairs one kind with a modes vector, and no loader's supported_tasks list
+// repeats a kind.
+bool family_supports(const Capabilities &caps, Task task, Mode mode) {
+    for (const auto &capability : caps.tasks) {
+        if (capability.task != task) {
+            continue;
+        }
+        return std::find(capability.modes.begin(), capability.modes.end(), mode) !=
+               capability.modes.end();
+    }
+    return false;
+}
+
+// Mode preference per RPC. Only AudioTranscriptionStream has a fallback: a
+// server-streaming transcription can be satisfied by an offline run that emits
+// one delta then the final result. Live transcription cannot, because it is
+// bidirectional and must consume audio incrementally.
+std::vector<Mode> mode_candidates(Rpc rpc) {
+    switch (rpc) {
+    case Rpc::TtsStream:
+    case Rpc::AudioTranscriptionLive:
+        return {Mode::Streaming};
+    case Rpc::AudioTranscriptionStream:
+        return {Mode::Streaming, Mode::Offline};
+    default:
+        return {Mode::Offline};
+    }
+}
+
+std::vector<Task> task_candidates(Rpc rpc, const RequestShape &shape) {
+    switch (rpc) {
+    case Rpc::Tts:
+    case Rpc::TtsStream:
+        // A supplied speaker clip is the strongest signal: the caller named the
+        // voice they want. Free-form instructions come next. Both fall back to
+        // plain Tts so a family without the specialised task still answers.
+        if (shape.has_voice_reference) {
+            return {Task::VoiceCloning, Task::Tts, Task::VoiceDesign};
+        }
+        if (shape.has_instructions) {
+            return {Task::VoiceDesign, Task::Tts, Task::VoiceCloning};
+        }
+        return {Task::Tts, Task::VoiceCloning, Task::VoiceDesign};
+    case Rpc::AudioTranscription:
+    case Rpc::AudioTranscriptionStream:
+    case Rpc::AudioTranscriptionLive:
+        // Asr first: `prompt` is also whisper-style decoding context, so its
+        // presence must not hijack a real ASR family into forced alignment.
+        if (shape.has_prompt_text) {
+            return {Task::Asr, Task::Alignment};
+        }
+        return {Task::Asr};
+    case Rpc::Vad:
+        return {Task::Vad};
+    case Rpc::Diarize:
+        return {Task::Diarization};
+    case Rpc::SoundGeneration:
+        return {Task::AudioGeneration};
+    case Rpc::AudioTransform:
+        // Svc is listed for completeness but is unreachable by auto-routing, by
+        // design: the only families advertising it (seed_vc, vevo2) also
+        // advertise VoiceConversion, which always wins, and no request signal
+        // means "this input is singing". Singing voice conversion therefore
+        // requires an explicit task:svc pin.
+        return {Task::SourceSeparation, Task::VoiceConversion, Task::Svc,
+                Task::SpeechToSpeech};
+    }
+    return {};
+}
+
+// The reasons behind unsupported_surfaces(), spelled once because AudioEncode
+// and AudioDecode share theirs. Each is phrased in terms of what upstream does
+// and does not have, so a reader can check it against the pinned checkout
+// rather than take it on trust. Every one of them was checked against
+// audio.cpp e800d435d130dc776baf6f3e6129bb62b1495c89, and one of the four
+// claims this backend was planned against did not survive that check: see
+// kTransformStreamReason.
+//
+// A latent upstream inconsistency worth knowing about but deliberately NOT put
+// on the wire, because it would mislead: model_spec/schema.cpp's task-string
+// whitelist does accept "codec" (and "dialogue"), while
+// model_spec/metadata.cpp's parse_task_kind has no branch for either and
+// throws "unknown model spec task". So a spec declaring "codec" validates and
+// then fails to load. That is a hole in upstream's own validation, not a codec
+// task this backend could reach.
+const char *const kCodecReason =
+    "audio.cpp's VoiceTaskKind has no codec entry, so no family can be asked to "
+    "turn PCM into codec frames or back; miocodec carries a Codec tag in "
+    "upstream's README but its loader advertises only vc and s2s";
+// NOT "streaming exists for tts and asr only", which is what this backend was
+// planned to say and is false: silero_vad advertises vad with RunMode::Streaming
+// (src/models/silero_vad/session.cpp). The claim that actually holds is the
+// narrower one below, about the four tasks AudioTransform routes to.
+// The trailing clause is not padding. The premise is an absence, and an absence
+// does not on its own make the RPC impossible: an offline sep family could be
+// buffered and emitted as a stream, which is what several LocalAI backends do.
+// Stopping at "nothing advertises streaming" would imply an impossibility the
+// evidence does not support. What is true, and what the caller needs, is that
+// this backend declines to dress an offline call up as a streaming one.
+const char *const kTransformStreamReason =
+    "no audio.cpp family advertises streaming for any task AudioTransform routes "
+    "to (sep, vc, svc, s2s); upstream advertises RunMode::Streaming for tts, asr "
+    "and vad only, and no conversion or separation family even implements its "
+    "IStreamingVoiceTaskSession interface, so a streaming transform here would be "
+    "a buffered offline call in disguise, which this backend does not pretend to "
+    "offer";
+// "clip-to-clip processing against a target voice", NOT "voice conversion". The
+// latter is true of miocodec and FALSE of vevo2, whose s2s route is `editing`
+// and only `editing`: src/models/vevo2/session.cpp's default_route_for_task maps
+// SpeechToSpeech to Editing and route_matches_task accepts nothing else, and
+// docs/models/vevo2.md defines that route as "Edit source speech into new target
+// text while using the target voice", requiring --target-text. It rewrites what
+// was said. vevo2's actual voice conversion is its separate vc task, which is
+// why upstream's README tags the family "TTS, Music, VC, Edit". The conclusion
+// is unaffected: neither family converses.
+const char *const kAudioToAudioReason =
+    "LocalAI's contract here is OpenAI-Realtime shaped, an audio conversation "
+    "emitting audio, transcript and tool-call deltas from a system prompt and a "
+    "tool list; audio.cpp's s2s is offline clip-to-clip processing against a "
+    "target voice, declared only by miocodec (voice conversion) and vevo2 "
+    "(speech editing), with no conversation, system prompt or tool loop";
+const char *const kVoiceEmbedReason =
+    "no audio.cpp family advertises the spk (SpeakerRecognition) task, so "
+    "nothing in the engine can produce a speaker embedding; the task kind "
+    "itself exists upstream, and TitaNet and ECAPA-TDNN exist as internal "
+    "conditioning encoders, but neither is registered as a loadable family";
+
+// The tasks an RPC is ever willing to route to, independent of request shape.
+//
+// DERIVED from task_candidates rather than restated, so a task added to an
+// RPC's candidate list cannot become inadmissible as a pin by omission. Setting
+// every shape flag yields each RPC's widest list: the per-flag branches only
+// reorder the same three tasks for Tts, and only ADD Alignment for
+// transcription, so the union is what comes back.
+std::vector<Task> admissible_tasks(Rpc rpc) {
+    RequestShape widest;
+    widest.has_voice_reference = true;
+    widest.has_instructions = true;
+    widest.has_prompt_text = true;
+    return task_candidates(rpc, widest);
+}
+
+std::string join_task_names(const std::vector<Task> &tasks) {
+    std::string out;
+    for (const Task task : tasks) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += task_name(task);
+    }
+    if (out.empty()) {
+        out = "nothing";
+    }
+    return out;
+}
+
+std::string join_attempts(const std::vector<Task> &tasks,
+                          const std::vector<Mode> &modes) {
+    std::string out;
+    for (const Task task : tasks) {
+        for (const Mode mode : modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += task_name(task);
+            out += "/";
+            out += mode_name(mode);
+        }
+    }
+    return out;
+}
+
+} // namespace
+
+const char *task_name(Task task) {
+    for (const auto &entry : kTaskNames) {
+        if (entry.task == task) {
+            return entry.name;
+        }
+    }
+    return "unknown";
+}
+
+const char *mode_name(Mode mode) {
+    return mode == Mode::Streaming ? "streaming" : "offline";
+}
+
+const char *rpc_name(Rpc rpc) {
+    switch (rpc) {
+    case Rpc::Tts:
+        return "TTS";
+    case Rpc::TtsStream:
+        return "TTSStream";
+    case Rpc::AudioTranscription:
+        return "AudioTranscription";
+    case Rpc::AudioTranscriptionStream:
+        return "AudioTranscriptionStream";
+    case Rpc::AudioTranscriptionLive:
+        return "AudioTranscriptionLive";
+    case Rpc::Vad:
+        return "VAD";
+    case Rpc::Diarize:
+        return "Diarize";
+    case Rpc::SoundGeneration:
+        return "SoundGeneration";
+    case Rpc::AudioTransform:
+        return "AudioTransform";
+    }
+    return "unknown";
+}
+
+bool parse_task_name(const std::string &value, Task &out) {
+    for (const auto &entry : kTaskNames) {
+        if (value == entry.name) {
+            out = entry.task;
+            return true;
+        }
+    }
+    for (const auto &entry : kTaskAliases) {
+        if (value == entry.name) {
+            out = entry.task;
+            return true;
+        }
+    }
+    return false;
+}
+
+std::string describe_capabilities(const Capabilities &caps) {
+    std::string out;
+    for (const auto &capability : caps.tasks) {
+        for (const Mode mode : capability.modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += task_name(capability.task);
+            out += "/";
+            out += mode_name(mode);
+        }
+    }
+    if (out.empty()) {
+        out = "nothing";
+    }
+    return out;
+}
+
+const std::vector<UnsupportedSurface> &unsupported_surfaces() {
+    // Ordered as UnsupportedRpc declares them. unsupported_surface() names each
+    // index in a switch rather than casting the enum, so the order is checked at
+    // compile time rather than trusted.
+    static const std::vector<UnsupportedSurface> kSurfaces = {
+        {"AudioEncode", kCodecReason},
+        {"AudioDecode", kCodecReason},
+        {"AudioTransformStream", kTransformStreamReason},
+        {"AudioToAudioStream", kAudioToAudioReason},
+        {"VoiceEmbed", kVoiceEmbedReason},
+    };
+    return kSurfaces;
+}
+
+// A switch with NO default label, deliberately. -Wswitch is on under -Wall, so a
+// sixth UnsupportedRpc added without a case here is a BUILD diagnostic, which is
+// the only place this class of mistake can be caught for free: a positional
+// static_cast<size_t>(rpc) would compile fine and read past the end of the table
+// at run time, on the one code path whose entire job is to be diagnosable. The
+// table stays a table because the tests iterate it.
+//
+// The trailing return is unreachable through the enum and exists only for a
+// caller that hands over a value outside it, which is already undefined
+// behaviour by the time it arrives.
+const UnsupportedSurface &unsupported_surface(UnsupportedRpc rpc) {
+    const std::vector<UnsupportedSurface> &surfaces = unsupported_surfaces();
+    switch (rpc) {
+    case UnsupportedRpc::AudioEncode:
+        return surfaces[0];
+    case UnsupportedRpc::AudioDecode:
+        return surfaces[1];
+    case UnsupportedRpc::AudioTransformStream:
+        return surfaces[2];
+    case UnsupportedRpc::AudioToAudioStream:
+        return surfaces[3];
+    case UnsupportedRpc::VoiceEmbed:
+        return surfaces[4];
+    }
+    return surfaces[0];
+}
+
+std::string unsupported_surface_message(const Capabilities &caps, const char *rpc,
+                                        const char *reason) {
+    return std::string("audio-cpp: the ") + rpc +
+           " RPC is not available through this backend because " + reason +
+           ". Loaded family '" + caps.family +
+           "' supports: " + describe_capabilities(caps);
+}
+
+std::string unsupported_surface_message(const char *rpc, const char *reason) {
+    return std::string("audio-cpp: the ") + rpc +
+           " RPC is not available through this backend because " + reason +
+           ". No model is loaded, so there is no family to list; loading one "
+           "would not change this answer";
+}
+
+Route resolve_route(Rpc rpc, const RequestShape &shape,
+                    const Capabilities &caps) {
+    Route route;
+
+    std::vector<Task> tasks;
+    if (!shape.pinned_task.empty()) {
+        Task pinned = Task::Tts;
+        if (!parse_task_name(shape.pinned_task, pinned)) {
+            route.error = "audio-cpp: unknown task option '" + shape.pinned_task +
+                          "'. Known tasks: gen, tts, clon, vc, svc, s2s, asr, "
+                          "align, vad, diar, sep, vdes, spk";
+            return route;
+        }
+        // A pin is honoured exactly, but ONLY on an RPC that could have routed
+        // to it anyway. It used to replace the candidate list wholesale for
+        // every RPC, and because the model's `task:` option is copied into the
+        // shape by all nine handlers, one pin bled across all nine surfaces and
+        // produced wrong 200s rather than errors: nemotron with task:asr made
+        // Vad return 200 with zero segments after a full ASR decode, so 14
+        // seconds of speech was reported as silence, and silero_vad with
+        // task:vad made AudioTranscription return 200 with empty text and four
+        // segments whose spans were VAD segments, which the srt/vtt/lrc writers
+        // then rendered as a well formed subtitle file of four timed EMPTY
+        // cues. Refusing is what the docs already promise: "if the family
+        // cannot serve it, the request is refused rather than rerouted".
+        //
+        // Every legitimate pin survives, because a pin only ever names the task
+        // its own RPC already routes to: svc is in AudioTransform's candidates,
+        // tts/clon/vdes in TTS's, asr in transcription's, vad and diar in
+        // theirs.
+        const std::vector<Task> admissible = admissible_tasks(rpc);
+        if (std::find(admissible.begin(), admissible.end(), pinned) ==
+            admissible.end()) {
+            route.error = std::string("audio-cpp: this model pins task '") +
+                          task_name(pinned) + "', which the " + rpc_name(rpc) +
+                          " RPC never routes to (it routes to " +
+                          join_task_names(admissible) +
+                          "). Remove the task option to reach this RPC, or call "
+                          "the RPC the pinned task serves";
+            return route;
+        }
+        tasks = {pinned};
+    } else {
+        tasks = task_candidates(rpc, shape);
+    }
+
+    const std::vector<Mode> modes = mode_candidates(rpc);
+
+    // Task-major: prefer the right task in a fallback mode over the wrong task
+    // in the preferred mode.
+    for (const Task task : tasks) {
+        for (const Mode mode : modes) {
+            if (family_supports(caps, task, mode)) {
+                route.ok = true;
+                route.task = task;
+                route.mode = mode;
+                return route;
+            }
+        }
+    }
+
+    route.error = std::string("audio-cpp: family '") + caps.family +
+                  "' cannot serve the " + rpc_name(rpc) + " RPC (tried " +
+                  join_attempts(tasks, modes) + "); it supports: " +
+                  describe_capabilities(caps);
+    return route;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/capability_routing.h

@@ -0,0 +1,131 @@
+#pragma once
+
+// Decides which audio.cpp (task, mode) pair serves a given LocalAI RPC, or
+// produces the capability error when none can. Standard library only, so this
+// unit is tested without an audio.cpp checkout; loaded_model.cpp converts
+// to and from engine::runtime types at the boundary.
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Mirrors engine::runtime::VoiceTaskKind, same members and same order.
+enum class Task {
+    Vad,
+    Asr,
+    Diarization,
+    SourceSeparation,
+    AudioGeneration,
+    Tts,
+    VoiceCloning,
+    VoiceConversion,
+    SpeechToSpeech,
+    Alignment,
+    VoiceDesign,
+    SpeakerRecognition,
+    Svc,
+};
+
+// Mirrors engine::runtime::RunMode.
+enum class Mode { Offline, Streaming };
+
+struct TaskCapability {
+    Task task = Task::Vad;
+    std::vector<Mode> modes;
+};
+
+struct Capabilities {
+    std::string family;
+    std::vector<TaskCapability> tasks;
+};
+
+// The LocalAI RPCs this backend serves. The ones it cannot serve at all are in
+// UnsupportedRpc below rather than here: they never reach routing, because no
+// family could satisfy them.
+enum class Rpc {
+    Tts,
+    TtsStream,
+    AudioTranscription,
+    AudioTranscriptionStream,
+    AudioTranscriptionLive,
+    Vad,
+    Diarize,
+    SoundGeneration,
+    AudioTransform,
+};
+
+struct RequestShape {
+    // A speaker reference clip was supplied (TTSRequest.voice resolved to audio).
+    bool has_voice_reference = false;
+    // TTSRequest.instructions is set.
+    bool has_instructions = false;
+    // TranscriptRequest.prompt is set.
+    bool has_prompt_text = false;
+    // The model's `task:` option, empty when unset. Overrides routing.
+    std::string pinned_task;
+};
+
+struct Route {
+    bool ok = false;
+    Task task = Task::Tts;
+    Mode mode = Mode::Offline;
+    // Set when ok is false. Suitable verbatim as an UNIMPLEMENTED message.
+    std::string error;
+};
+
+Route resolve_route(Rpc rpc, const RequestShape &shape, const Capabilities &caps);
+
+// Canonical audio.cpp short names: gen, tts, clon, vc, svc, s2s, asr, align,
+// vad, diar, sep, vdes, spk. parse_task_name additionally accepts "spkrec" as
+// a legacy alias; task_name only ever emits "spk".
+const char *task_name(Task task);
+const char *mode_name(Mode mode);
+const char *rpc_name(Rpc rpc);
+bool parse_task_name(const std::string &value, Task &out);
+
+// "asr/offline, asr/streaming", for error messages.
+std::string describe_capabilities(const Capabilities &caps);
+
+// The RPCs in LocalAI's backend contract that audio.cpp has no counterpart for,
+// as opposed to the ones in Rpc above, which a particular family may or may not
+// be able to serve. Nothing routes to these: the refusal is a property of the
+// engine, not of the loaded model, so loading a different family cannot change
+// it.
+//
+// This is deliberately NOT deferred work. Each entry names the upstream
+// limitation that keeps it out of Rpc, and each becomes an ordinary routing
+// entry the day upstream lifts that limitation.
+enum class UnsupportedRpc {
+    AudioEncode,
+    AudioDecode,
+    AudioTransformStream,
+    AudioToAudioStream,
+    VoiceEmbed,
+};
+
+struct UnsupportedSurface {
+    // The RPC's name as backend.proto spells it, for the message.
+    const char *rpc;
+    // Why audio.cpp cannot serve it, in terms of what upstream does and does
+    // not have. Stated so a caller can tell "not built yet" from "not possible".
+    const char *reason;
+};
+
+// The table behind the five refusals. Exposed whole so a test can assert every
+// entry rather than the one somebody remembered to cover, and so the reasons
+// are data in one place instead of string literals hand-copied into handlers.
+const std::vector<UnsupportedSurface> &unsupported_surfaces();
+const UnsupportedSurface &unsupported_surface(UnsupportedRpc rpc);
+
+// Message for an RPC this backend cannot serve at all, as opposed to one this
+// particular family cannot serve. `reason` states the upstream limitation.
+std::string unsupported_surface_message(const Capabilities &caps, const char *rpc,
+                                        const char *reason);
+
+// Same, for the no-model-loaded case. Says so explicitly rather than naming an
+// empty family, and says that loading one would not help, because the caller's
+// obvious next move otherwise is to load a model and try again.
+std::string unsupported_surface_message(const char *rpc, const char *reason);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/capability_routing_test.cpp

@@ -0,0 +1,600 @@
+// Unit tests for capability_routing. Standard library only. The harness
+// compiles this as a single translation unit, so the implementation is
+// included directly rather than linked.
+
+#include "capability_routing.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// Mirrors what supertonic advertises: TTS offline and streaming.
+static Capabilities supertonic() {
+    return Capabilities{"supertonic",
+                        {{Task::Tts, {Mode::Offline, Mode::Streaming}}}};
+}
+
+// Mirrors chatterbox: TTS, cloning and voice conversion, offline only.
+static Capabilities chatterbox() {
+    return Capabilities{"chatterbox",
+                        {{Task::Tts, {Mode::Offline}},
+                         {Task::VoiceCloning, {Mode::Offline}},
+                         {Task::VoiceConversion, {Mode::Offline}}}};
+}
+
+// Mirrors nemotron_asr: ASR offline and streaming.
+static Capabilities nemotron() {
+    return Capabilities{"nemotron_asr",
+                        {{Task::Asr, {Mode::Offline, Mode::Streaming}}}};
+}
+
+// Mirrors qwen3_asr: ASR offline only.
+static Capabilities qwen3_asr() {
+    return Capabilities{"qwen3_asr", {{Task::Asr, {Mode::Offline}}}};
+}
+
+// Mirrors qwen3_forced_aligner: alignment only.
+static Capabilities aligner() {
+    return Capabilities{"qwen3_forced_aligner",
+                        {{Task::Alignment, {Mode::Offline}}}};
+}
+
+// Mirrors htdemucs: separation only.
+static Capabilities htdemucs() {
+    return Capabilities{"htdemucs", {{Task::SourceSeparation, {Mode::Offline}}}};
+}
+
+static void test_plain_tts() {
+    const auto r = resolve_route(Rpc::Tts, RequestShape{}, chatterbox());
+    check(r.ok, "plain TTS routes");
+    check(r.task == Task::Tts, "plain TTS picks Tts, not VoiceCloning");
+    check(r.mode == Mode::Offline, "TTS runs offline");
+}
+
+static void test_tts_with_voice_reference_prefers_cloning() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    const auto r = resolve_route(Rpc::Tts, shape, chatterbox());
+    check(r.ok, "TTS with a voice reference routes");
+    check(r.task == Task::VoiceCloning, "voice reference prefers VoiceCloning");
+}
+
+// supertonic has no VoiceCloning: a voice reference must fall back to Tts
+// rather than failing the request.
+static void test_tts_voice_reference_falls_back_to_tts() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    const auto r = resolve_route(Rpc::Tts, shape, supertonic());
+    check(r.ok, "voice reference on a clone-less family still routes");
+    check(r.task == Task::Tts, "falls back to Tts");
+}
+
+static void test_tts_instructions_prefer_voice_design() {
+    RequestShape shape;
+    shape.has_instructions = true;
+    Capabilities caps{"qwen3_tts",
+                      {{Task::Tts, {Mode::Offline}},
+                       {Task::VoiceDesign, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::Tts, shape, caps);
+    check(r.ok, "TTS with instructions routes");
+    check(r.task == Task::VoiceDesign, "instructions prefer VoiceDesign");
+}
+
+// A voice reference is a stronger signal than free-form instructions: cloning
+// a specific voice is what the user asked for.
+static void test_voice_reference_beats_instructions() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    shape.has_instructions = true;
+    Capabilities caps{"omnivoice",
+                      {{Task::Tts, {Mode::Offline}},
+                       {Task::VoiceCloning, {Mode::Offline}},
+                       {Task::VoiceDesign, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::Tts, shape, caps);
+    check(r.ok, "both signals present routes");
+    check(r.task == Task::VoiceCloning, "voice reference outranks instructions");
+}
+
+static void test_tts_stream_requires_streaming() {
+    const auto ok = resolve_route(Rpc::TtsStream, RequestShape{}, supertonic());
+    check(ok.ok, "streaming TTS routes on supertonic");
+    check(ok.mode == Mode::Streaming, "TTSStream runs in streaming mode");
+
+    const auto bad = resolve_route(Rpc::TtsStream, RequestShape{}, chatterbox());
+    check(!bad.ok, "streaming TTS is refused on an offline-only family");
+    check(bad.error.find("chatterbox") != std::string::npos,
+          "error names the family");
+    check(bad.error.find("tts/offline") != std::string::npos,
+          "error lists what the family does support");
+    check(bad.error.find("TTSStream") != std::string::npos,
+          "error names the RPC that was refused");
+    check(bad.error.find("tts/streaming") != std::string::npos,
+          "error lists the (task, mode) pairs that were tried");
+}
+
+static void test_transcription_stream_falls_back_to_offline() {
+    const auto streaming =
+        resolve_route(Rpc::AudioTranscriptionStream, RequestShape{}, nemotron());
+    check(streaming.ok && streaming.mode == Mode::Streaming,
+          "streaming ASR uses streaming mode when offered");
+
+    const auto offline =
+        resolve_route(Rpc::AudioTranscriptionStream, RequestShape{}, qwen3_asr());
+    check(offline.ok, "streaming ASR falls back on an offline-only family");
+    check(offline.mode == Mode::Offline, "fallback mode is offline");
+    check(offline.task == Task::Asr, "fallback task is still Asr");
+}
+
+// Task preference dominates mode preference: it is better to run the right
+// task in a fallback mode than the wrong task in the preferred mode. This is
+// the only RPC where the two orderings can disagree, because it is the only
+// one with more than one acceptable mode.
+static void test_task_preference_beats_mode_preference() {
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    Capabilities mixed{"mixed_asr_aligner",
+                       {{Task::Asr, {Mode::Offline}},
+                        {Task::Alignment, {Mode::Streaming}}}};
+    const auto r = resolve_route(Rpc::AudioTranscriptionStream, shape, mixed);
+    check(r.ok, "mixed family routes");
+    check(r.task == Task::Asr,
+          "the preferred task wins even in its fallback mode");
+    check(r.mode == Mode::Offline,
+          "the fallback mode is accepted to keep the preferred task");
+}
+
+// Live transcription is bidirectional and cannot be faked from an offline run.
+static void test_live_transcription_has_no_offline_fallback() {
+    const auto r =
+        resolve_route(Rpc::AudioTranscriptionLive, RequestShape{}, qwen3_asr());
+    check(!r.ok, "live transcription is refused on an offline-only family");
+}
+
+static void test_alignment_needs_prompt_text() {
+    const auto without =
+        resolve_route(Rpc::AudioTranscription, RequestShape{}, aligner());
+    check(!without.ok, "aligner without a transcript is refused");
+
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    const auto with = resolve_route(Rpc::AudioTranscription, shape, aligner());
+    check(with.ok, "aligner with a transcript routes");
+    check(with.task == Task::Alignment, "routes to Alignment");
+}
+
+// A real ASR family must not be hijacked to Alignment just because the caller
+// passed a prompt: `prompt` is also whisper-style decoding context.
+static void test_prompt_does_not_hijack_asr() {
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    const auto r = resolve_route(Rpc::AudioTranscription, shape, nemotron());
+    check(r.ok, "ASR with a prompt routes");
+
+    // nemotron advertises Asr alone, so the assertion has to be made against a
+    // family that advertises both: otherwise "Asr is preferred" only restates
+    // that Asr is the only option, and reversing the preference order passes.
+    Capabilities both{"asr_with_aligner",
+                      {{Task::Asr, {Mode::Offline}},
+                       {Task::Alignment, {Mode::Offline}}}};
+    const auto pref = resolve_route(Rpc::AudioTranscription, shape, both);
+    check(pref.ok, "a family offering both routes");
+    check(pref.task == Task::Asr, "Asr is preferred over Alignment");
+}
+
+static void test_audio_transform_prefers_separation() {
+    const auto sep =
+        resolve_route(Rpc::AudioTransform, RequestShape{}, htdemucs());
+    check(sep.ok && sep.task == Task::SourceSeparation, "separation routes");
+
+    // htdemucs advertises separation alone, so the check above cannot fail on
+    // ordering. This family advertises both, which is what pins the preference.
+    Capabilities sep_and_vc{"sep_and_vc",
+                            {{Task::SourceSeparation, {Mode::Offline}},
+                             {Task::VoiceConversion, {Mode::Offline}}}};
+    const auto pref =
+        resolve_route(Rpc::AudioTransform, RequestShape{}, sep_and_vc);
+    check(pref.ok && pref.task == Task::SourceSeparation,
+          "separation is preferred over voice conversion");
+
+    Capabilities miocodec{"miocodec",
+                          {{Task::VoiceConversion, {Mode::Offline}},
+                           {Task::SpeechToSpeech, {Mode::Offline}}}};
+    const auto vc = resolve_route(Rpc::AudioTransform, RequestShape{}, miocodec);
+    check(vc.ok && vc.task == Task::VoiceConversion,
+          "voice conversion is preferred over speech-to-speech");
+}
+
+static void test_pinned_task_overrides_routing() {
+    RequestShape shape;
+    shape.pinned_task = "s2s";
+    Capabilities miocodec{"miocodec",
+                          {{Task::VoiceConversion, {Mode::Offline}},
+                           {Task::SpeechToSpeech, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::AudioTransform, shape, miocodec);
+    check(r.ok && r.task == Task::SpeechToSpeech, "pinned task wins");
+
+    RequestShape bad;
+    bad.pinned_task = "not-a-task";
+    const auto e = resolve_route(Rpc::AudioTransform, bad, miocodec);
+    check(!e.ok, "an unknown pinned task is an error");
+    check(e.error.find("not-a-task") != std::string::npos,
+          "error names the bad task");
+
+    // A pinned task the family does not offer must fail, not silently reroute.
+    RequestShape unsupported;
+    unsupported.pinned_task = "sep";
+    const auto u = resolve_route(Rpc::AudioTransform, unsupported, miocodec);
+    check(!u.ok, "a pinned but unsupported task is refused");
+}
+
+// A pin lives on the MODEL, and every one of the nine handlers copies it into
+// the shape, so a pin set for one RPC arrives at all of them. It used to
+// replace the candidate list wholesale, which turned the other eight into wrong
+// 200s rather than errors: nemotron pinned to asr made Vad answer with zero
+// segments after a full ASR decode, and silero_vad pinned to vad made
+// AudioTranscription answer with empty text and four segments whose spans were
+// VAD segments, which the srt/vtt/lrc writers rendered as timed EMPTY cues.
+static void test_pin_must_be_admissible_for_the_rpc() {
+    Capabilities nemotron_asr{"nemotron_asr",
+                              {{Task::Asr, {Mode::Offline, Mode::Streaming}},
+                               {Task::Vad, {Mode::Offline}}}};
+
+    // The pin is legitimate on the RPC it was meant for.
+    RequestShape asr_pin;
+    asr_pin.pinned_task = "asr";
+    const auto transcription =
+        resolve_route(Rpc::AudioTranscription, asr_pin, nemotron_asr);
+    check(transcription.ok && transcription.task == Task::Asr,
+          "an admissible pin is still honoured exactly");
+
+    // ...and refused on one that never routes to it, EVEN THOUGH the family
+    // advertises the pinned task. That is the whole point: family support is
+    // not the question, RPC admissibility is.
+    const auto vad = resolve_route(Rpc::Vad, asr_pin, nemotron_asr);
+    check(!vad.ok, "an inadmissible pin is refused rather than served");
+    check(vad.error.find("asr") != std::string::npos,
+          "the refusal names the pinned task");
+    check(vad.error.find(rpc_name(Rpc::Vad)) != std::string::npos,
+          "the refusal names the RPC that cannot serve it");
+
+    Capabilities silero{"silero_vad", {{Task::Vad, {Mode::Offline}}}};
+    RequestShape vad_pin;
+    vad_pin.pinned_task = "vad";
+    const auto vad_ok = resolve_route(Rpc::Vad, vad_pin, silero);
+    check(vad_ok.ok && vad_ok.task == Task::Vad, "vad is admissible on Vad");
+    const auto transcribe_vad =
+        resolve_route(Rpc::AudioTranscription, vad_pin, silero);
+    check(!transcribe_vad.ok,
+          "a vad pin cannot make a transcription request return empty cues");
+
+    // Every pin a shipped configuration could sensibly set stays reachable on
+    // the RPC that serves it. This is the list the fix was checked against.
+    struct AdmissibleCase {
+        Rpc rpc;
+        const char *task;
+    };
+    const AdmissibleCase kAdmissible[] = {
+        {Rpc::AudioTransform, "svc"},   {Rpc::AudioTransform, "sep"},
+        {Rpc::AudioTransform, "vc"},    {Rpc::AudioTransform, "s2s"},
+        {Rpc::Tts, "tts"},              {Rpc::Tts, "clon"},
+        {Rpc::Tts, "vdes"},             {Rpc::TtsStream, "tts"},
+        {Rpc::AudioTranscription, "asr"},
+        {Rpc::AudioTranscription, "align"},
+        {Rpc::AudioTranscriptionStream, "asr"},
+        {Rpc::AudioTranscriptionLive, "asr"},
+        {Rpc::Vad, "vad"},              {Rpc::Diarize, "diar"},
+        {Rpc::SoundGeneration, "gen"},
+    };
+    for (const auto &entry : kAdmissible) {
+        Task task = Task::Tts;
+        check(parse_task_name(entry.task, task),
+              std::string("known task name: ") + entry.task);
+        // A family that advertises the pinned task offline and nothing else, so
+        // the ONLY thing that can refuse the route is the admissibility check.
+        Capabilities only{"probe",
+                          {{task, {Mode::Offline, Mode::Streaming}}}};
+        RequestShape pin;
+        pin.pinned_task = entry.task;
+        const auto route = resolve_route(entry.rpc, pin, only);
+        check(route.ok && route.task == task,
+              std::string("pin '") + entry.task + "' stays admissible on " +
+                  rpc_name(entry.rpc));
+    }
+
+    // And the pins that must NOT cross over, one per RPC pair that was
+    // observed producing a wrong 200.
+    const AdmissibleCase kInadmissible[] = {
+        {Rpc::Vad, "asr"},          {Rpc::Diarize, "asr"},
+        {Rpc::AudioTranscription, "vad"}, {Rpc::AudioTranscription, "diar"},
+        {Rpc::Tts, "asr"},          {Rpc::Vad, "tts"},
+        {Rpc::SoundGeneration, "tts"},    {Rpc::AudioTransform, "asr"},
+    };
+    for (const auto &entry : kInadmissible) {
+        Task task = Task::Tts;
+        check(parse_task_name(entry.task, task),
+              std::string("known task name: ") + entry.task);
+        Capabilities only{"probe",
+                          {{task, {Mode::Offline, Mode::Streaming}}}};
+        RequestShape pin;
+        pin.pinned_task = entry.task;
+        const auto route = resolve_route(entry.rpc, pin, only);
+        check(!route.ok,
+              std::string("pin '") + entry.task + "' is refused on " +
+                  rpc_name(entry.rpc));
+    }
+}
+
+static void test_vad_and_diarize() {
+    Capabilities silero{"silero_vad", {{Task::Vad, {Mode::Offline, Mode::Streaming}}}};
+    const auto v = resolve_route(Rpc::Vad, RequestShape{}, silero);
+    check(v.ok && v.task == Task::Vad && v.mode == Mode::Offline, "VAD routes offline");
+
+    const auto d = resolve_route(Rpc::Diarize, RequestShape{}, silero);
+    check(!d.ok, "diarization is refused on a VAD-only family");
+
+    Capabilities sortformer{"sortformer_diar", {{Task::Diarization, {Mode::Offline}}}};
+    const auto ok = resolve_route(Rpc::Diarize, RequestShape{}, sortformer);
+    check(ok.ok && ok.task == Task::Diarization, "diarization routes");
+}
+
+static void test_sound_generation() {
+    Capabilities stable{"stable_audio", {{Task::AudioGeneration, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::SoundGeneration, RequestShape{}, stable);
+    check(r.ok && r.task == Task::AudioGeneration, "sound generation routes");
+}
+
+static void test_names_round_trip() {
+    const Task all[] = {Task::Vad, Task::Asr, Task::Diarization,
+                        Task::SourceSeparation, Task::AudioGeneration, Task::Tts,
+                        Task::VoiceCloning, Task::VoiceConversion,
+                        Task::SpeechToSpeech, Task::Alignment, Task::VoiceDesign,
+                        Task::SpeakerRecognition, Task::Svc};
+    for (const Task t : all) {
+        Task parsed = Task::Vad;
+        const bool ok = parse_task_name(task_name(t), parsed);
+        check(ok && parsed == t,
+              std::string("task name round-trips: ") + task_name(t));
+    }
+    check(std::string(mode_name(Mode::Offline)) == "offline", "offline name");
+    check(std::string(mode_name(Mode::Streaming)) == "streaming", "streaming name");
+
+    // The emitted name must be the one audio.cpp itself prints and parses
+    // (framework/runtime/session.cpp), because `task:` is user-facing: a name
+    // copied out of audio.cpp has to be accepted here, and a name pinned here
+    // has to survive conversion at the engine boundary.
+    check(std::string(task_name(Task::SpeakerRecognition)) == "spk",
+          "speaker recognition emits upstream's name 'spk'");
+
+    Task pinned = Task::Vad;
+    check(parse_task_name("spk", pinned) && pinned == Task::SpeakerRecognition,
+          "'spk' parses to SpeakerRecognition");
+
+    // Accepted as a legacy alias so configs written against the earlier name
+    // keep working, but never emitted.
+    Task alias = Task::Vad;
+    check(parse_task_name("spkrec", alias) && alias == Task::SpeakerRecognition,
+          "'spkrec' is still accepted as an alias");
+}
+
+static void test_describe_capabilities() {
+    const std::string described = describe_capabilities(nemotron());
+    check(described.find("asr/offline") != std::string::npos,
+          "description lists asr/offline");
+    check(described.find("asr/streaming") != std::string::npos,
+          "description lists asr/streaming");
+}
+
+static void test_empty_capabilities() {
+    const auto r = resolve_route(Rpc::Tts, RequestShape{}, Capabilities{"mystery", {}});
+    check(!r.ok, "a family advertising nothing is refused");
+    check(r.error.find("mystery") != std::string::npos, "error names the family");
+}
+
+// The table is indexed by UnsupportedRpc's underlying value, so a reordering of
+// either list silently pairs an RPC with another's reason. Nothing else would
+// catch that: both sides still compile and every message still reads plausibly.
+static void test_unsupported_surface_table_matches_the_enum() {
+    check(unsupported_surfaces().size() == 5,
+          "all five unsupported surfaces are tabulated");
+    check(std::string(unsupported_surface(UnsupportedRpc::AudioEncode).rpc) ==
+              "AudioEncode",
+          "UnsupportedRpc::AudioEncode indexes AudioEncode");
+    check(std::string(unsupported_surface(UnsupportedRpc::AudioDecode).rpc) ==
+              "AudioDecode",
+          "UnsupportedRpc::AudioDecode indexes AudioDecode");
+    check(std::string(
+              unsupported_surface(UnsupportedRpc::AudioTransformStream).rpc) ==
+              "AudioTransformStream",
+          "UnsupportedRpc::AudioTransformStream indexes AudioTransformStream");
+    check(std::string(
+              unsupported_surface(UnsupportedRpc::AudioToAudioStream).rpc) ==
+              "AudioToAudioStream",
+          "UnsupportedRpc::AudioToAudioStream indexes AudioToAudioStream");
+    check(std::string(unsupported_surface(UnsupportedRpc::VoiceEmbed).rpc) ==
+              "VoiceEmbed",
+          "UnsupportedRpc::VoiceEmbed indexes VoiceEmbed");
+
+    // Two entries may share a reason (the codec pair does), but two entries
+    // naming the same RPC would mean one of the five is unreachable.
+    for (size_t i = 0; i < unsupported_surfaces().size(); ++i) {
+        for (size_t j = i + 1; j < unsupported_surfaces().size(); ++j) {
+            check(std::string(unsupported_surfaces()[i].rpc) !=
+                      unsupported_surfaces()[j].rpc,
+                  std::string("no duplicate RPC name at ") + std::to_string(i) +
+                      "/" + std::to_string(j));
+        }
+    }
+}
+
+// There is no out-of-range test for unsupported_surface(). It switches over the
+// enumerators with no default label, so a sixth UnsupportedRpc without a case is
+// a -Wswitch diagnostic at build time and cannot reach a run-time check at all.
+
+// Every entry, not just the one somebody remembered to cover. A refusal that
+// drops the family, the RPC or the reason is a refusal the caller cannot act
+// on, which is the entire point of this surface existing.
+static void test_every_unsupported_surface_message_is_diagnosable() {
+    for (const auto &surface : unsupported_surfaces()) {
+        const std::string label = std::string(" [") + surface.rpc + "]";
+        const std::string loaded =
+            unsupported_surface_message(nemotron(), surface.rpc, surface.reason);
+
+        check(loaded.find(surface.rpc) != std::string::npos,
+              "message names the RPC" + label);
+        check(std::string(surface.reason).size() > 20 &&
+                  loaded.find(surface.reason) != std::string::npos,
+              "message gives a substantive upstream reason" + label);
+        check(loaded.find("nemotron_asr") != std::string::npos,
+              "message names the loaded family" + label);
+        check(loaded.find("asr/offline") != std::string::npos &&
+                  loaded.find("asr/streaming") != std::string::npos,
+              "message lists what the family does support" + label);
+
+        // The no-model form keeps the two facts that do not depend on a model
+        // and drops only the one that does, so the caller still learns why.
+        const std::string unloaded =
+            unsupported_surface_message(surface.rpc, surface.reason);
+        check(unloaded.find(surface.rpc) != std::string::npos,
+              "no-model message names the RPC" + label);
+        check(unloaded.find(surface.reason) != std::string::npos,
+              "no-model message gives the upstream reason" + label);
+        check(unloaded.find("nemotron_asr") == std::string::npos,
+              "no-model message names no family" + label);
+        // Without this the caller's obvious next move is to load a model and
+        // retry, which cannot work: the refusal is a property of the engine.
+        check(unloaded.find("would not change this answer") != std::string::npos,
+              "no-model message says loading a model would not help" + label);
+    }
+}
+
+// The reasons are the load-bearing half of this feature and each was checked
+// against the pinned upstream checkout. Pinning the distinguishing phrase here
+// means a later edit that guts one into a generic "not supported" fails rather
+// than passes quietly.
+static void test_unsupported_reasons_name_the_upstream_limitation() {
+    const auto &encode = unsupported_surface(UnsupportedRpc::AudioEncode);
+    const auto &decode = unsupported_surface(UnsupportedRpc::AudioDecode);
+    check(std::string(encode.reason).find("VoiceTaskKind") != std::string::npos &&
+              std::string(encode.reason).find("codec") != std::string::npos,
+          "the AudioEncode reason names the missing VoiceTaskKind entry");
+    // miocodec is the family a reader will reach for first, because upstream's
+    // README tags it Codec. Naming it and its actual advertised tasks is what
+    // stops the next person re-deriving the same dead end.
+    check(std::string(encode.reason).find("miocodec") != std::string::npos,
+          "the AudioEncode reason disposes of miocodec's README Codec tag");
+    check(std::string(encode.reason) == decode.reason,
+          "AudioEncode and AudioDecode refuse for the same reason");
+
+    const auto &transform =
+        unsupported_surface(UnsupportedRpc::AudioTransformStream);
+    // The reason must be scoped to the tasks AudioTransform routes to. The
+    // broader claim, "upstream streams tts and asr only", is FALSE: silero_vad
+    // advertises vad with RunMode::Streaming. A refusal resting on a false
+    // premise is worse than a bare UNIMPLEMENTED, because it will be believed.
+    check(std::string(transform.reason).find("sep, vc, svc, s2s") !=
+              std::string::npos,
+          "the AudioTransformStream reason is scoped to the routed tasks");
+    check(std::string(transform.reason).find("tts, asr and vad") !=
+              std::string::npos,
+          "the AudioTransformStream reason counts vad among the streaming tasks");
+    check(std::string(transform.reason).find("tts and asr only") ==
+              std::string::npos,
+          "the AudioTransformStream reason does not repeat the refuted claim");
+    // An absence is not an impossibility. A sep family could be buffered and
+    // emitted as a stream, so the reason has to say this backend declines to
+    // rather than cannot, or it overreaches on a true premise.
+    check(std::string(transform.reason).find("buffered offline call in disguise") !=
+              std::string::npos,
+          "the AudioTransformStream reason does not overclaim impossibility");
+
+    const auto &s2s = unsupported_surface(UnsupportedRpc::AudioToAudioStream);
+    check(std::string(s2s.reason).find("Realtime") != std::string::npos &&
+              std::string(s2s.reason).find("clip-to-clip") != std::string::npos,
+          "the AudioToAudioStream reason contrasts the two contracts");
+    // Naming both s2s families, and what each of them actually does, makes the
+    // claim checkable. It must NOT say s2s is voice conversion full stop: that
+    // is true of miocodec and false of vevo2, whose s2s route is `editing` and
+    // rewrites the spoken content against a target voice.
+    check(std::string(s2s.reason).find("miocodec (voice conversion)") !=
+                  std::string::npos &&
+              std::string(s2s.reason).find("vevo2 (speech editing)") !=
+                  std::string::npos,
+          "the AudioToAudioStream reason names each s2s family's actual task");
+    check(std::string(s2s.reason).find("s2s is offline voice conversion") ==
+              std::string::npos,
+          "the AudioToAudioStream reason does not miscast vevo2 as conversion");
+
+    const auto &embed = unsupported_surface(UnsupportedRpc::VoiceEmbed);
+    check(std::string(embed.reason).find("spk") != std::string::npos,
+          "the VoiceEmbed reason names the task no family advertises");
+    // spk IS a VoiceTaskKind upstream; what is missing is any family that
+    // advertises it. Saying the kind does not exist would be false, and would
+    // send a reader looking in the wrong place.
+    check(std::string(embed.reason).find("no audio.cpp family") !=
+              std::string::npos,
+          "the VoiceEmbed reason blames the families, not the enum");
+    // The speaker encoders DO exist upstream, as conditioning modules inside
+    // TTS and VC families. Not saying so invites "but audio.cpp ships TitaNet".
+    check(std::string(embed.reason).find("TitaNet") != std::string::npos,
+          "the VoiceEmbed reason disposes of the internal speaker encoders");
+    Task parsed = Task::Vad;
+    check(parse_task_name("spk", parsed) && parsed == Task::SpeakerRecognition,
+          "spk is a real task kind, so the reason must not claim otherwise");
+}
+
+// A family advertising nothing still gets a message that reads, rather than one
+// trailing off after "supports: ".
+static void test_unsupported_surface_message_with_empty_capabilities() {
+    const auto &embed = unsupported_surface(UnsupportedRpc::VoiceEmbed);
+    const std::string message = unsupported_surface_message(
+        Capabilities{"mystery", {}}, embed.rpc, embed.reason);
+    check(message.find("mystery") != std::string::npos,
+          "empty-capability message still names the family");
+    check(message.find("supports: nothing") != std::string::npos,
+          "empty-capability message says the family supports nothing");
+}
+
+int main() {
+    test_plain_tts();
+    test_tts_with_voice_reference_prefers_cloning();
+    test_tts_voice_reference_falls_back_to_tts();
+    test_tts_instructions_prefer_voice_design();
+    test_voice_reference_beats_instructions();
+    test_tts_stream_requires_streaming();
+    test_transcription_stream_falls_back_to_offline();
+    test_task_preference_beats_mode_preference();
+    test_live_transcription_has_no_offline_fallback();
+    test_alignment_needs_prompt_text();
+    test_prompt_does_not_hijack_asr();
+    test_audio_transform_prefers_separation();
+    test_pinned_task_overrides_routing();
+    test_pin_must_be_admissible_for_the_rpc();
+    test_vad_and_diarize();
+    test_sound_generation();
+    test_names_round_trip();
+    test_describe_capabilities();
+    test_empty_capabilities();
+    test_unsupported_surface_table_matches_the_enum();
+    test_every_unsupported_surface_message_is_diagnosable();
+    test_unsupported_reasons_name_the_upstream_limitation();
+    test_unsupported_surface_message_with_empty_capabilities();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all capability_routing checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/family_gate.cpp

@@ -0,0 +1,163 @@
+#include "family_gate.h"
+
+#include <cctype>
+#include <cstddef>
+
+namespace audiocpp_backend {
+namespace {
+
+// PRECONDITION: `suffix` must already be lowercase. Both sides are folded, so
+// this reads as symmetric, but only `value` can carry case in practice and a
+// caller passing ".GGUF" would still work today for that reason alone. Do not
+// rely on it: the fold on the suffix side is the only thing standing between
+// this and a helper that answers false for every input, and it is not covered
+// by any test, because with a lowercase suffix no input can distinguish it.
+bool ends_with_ci(const std::string &value, const std::string &suffix) {
+    if (value.size() <= suffix.size()) {
+        return false; // a bare ".gguf" is an extension, not a model file
+    }
+    const size_t offset = value.size() - suffix.size();
+    for (size_t i = 0; i < suffix.size(); ++i) {
+        const auto lhs = static_cast<unsigned char>(value[offset + i]);
+        const auto rhs = static_cast<unsigned char>(suffix[i]);
+        if (std::tolower(lhs) != std::tolower(rhs)) {
+            return false;
+        }
+    }
+    return true;
+}
+
+} // namespace
+
+bool path_looks_like_gguf(const std::string &path) {
+    return ends_with_ci(path, ".gguf");
+}
+
+FamilyDecision decide_family(bool path_is_gguf, const std::string &embedded_family,
+                             const std::string &configured_family) {
+    FamilyDecision decision;
+
+    if (!configured_family.empty()) {
+        decision.ok = true;
+        decision.family = configured_family;
+        return decision;
+    }
+
+    if (path_is_gguf) {
+        if (!embedded_family.empty()) {
+            decision.ok = true;
+            decision.family = embedded_family;
+            return decision;
+        }
+        decision.error =
+            "audio-cpp: this GGUF carries no 'audiocpp.model_spec.family' "
+            "metadata key, so it is not an audio.cpp model. Convert it with "
+            "audiocpp_gguf, or name the family explicitly with the model option "
+            "'family:<name>'";
+        return decision;
+    }
+
+    decision.error =
+        "audio-cpp: a model path that is not a standalone audio.cpp GGUF needs "
+        "an explicit 'family:<name>' model option, because the audio.cpp family "
+        "cannot be inferred from a safetensors or package directory";
+    return decision;
+}
+
+namespace {
+
+// Families that ABORT THE PROCESS on a weight dtype they cannot handle, and the
+// dtypes they can. See the header for why this is a list of crashes rather than
+// a list of preferences.
+//
+// supertonic: upstream's docs/gguf.md:90 records its 16-bit GGUF column as
+// "---", i.e. NOT TESTED, and its q8_0 as "No (unsupported weight dtype)". Only
+// the `orig` package is marked Pass, and its 698 weight tensors are f32 while
+// its 72 index and shape constants are i64. The f16 abort is a LOCAL
+// OBSERVATION rather than an upstream claim, and it is attributed rather than
+// assumed: it is identical through the unary TTS RPC and through TTSStream, so
+// it is the packaging and not the streaming path. q8_0 was never run here and is
+// refused on upstream's "unsupported weight dtype" alone, which is the weaker of
+// the two claims. See the header for why keeping them apart matters.
+//
+// TO REMOVE AN ENTRY: bump AUDIO_CPP_VERSION past a fix, load a package in the
+// refused dtype, and synthesise. If audio comes out, delete the entry. No test
+// can do that for you, which is exactly why it is written here: the test beside
+// this file pins WHAT the table says, not whether upstream has moved on. Do not
+// widen an entry without running that, because what it prevents is a process
+// death rather than a wrong answer.
+struct DtypeAllowList {
+    // NULL TERMINATED, and the terminator occupies one of these slots: both
+    // loops below stop at the first nullptr and have no other bound, so an entry
+    // that named three dtypes would leave them reading past the end of the
+    // array. That is undefined behaviour rather than a wrong answer, and it is
+    // one keystroke away from any edit that widens an entry, so the terminator
+    // is asserted at compile time below rather than trusted.
+    static constexpr std::size_t kSlots = 3;
+
+    const char *family;
+    const char *allowed[kSlots];
+};
+
+constexpr DtypeAllowList kDtypeAllowLists[] = {
+    {"supertonic", {"f32", "i64", nullptr}},
+};
+
+constexpr bool allow_lists_are_terminated() {
+    for (const auto &entry : kDtypeAllowLists) {
+        if (entry.allowed[DtypeAllowList::kSlots - 1] != nullptr) {
+            return false;
+        }
+    }
+    return true;
+}
+
+static_assert(allow_lists_are_terminated(),
+              "every DtypeAllowList must leave its last slot null: the lookups "
+              "below stop at the first nullptr and would otherwise read past "
+              "the end of the array");
+
+const DtypeAllowList *find_allow_list(const std::string &family) {
+    for (const auto &entry : kDtypeAllowLists) {
+        if (family == entry.family) {
+            return &entry;
+        }
+    }
+    return nullptr;
+}
+
+} // namespace
+
+bool family_has_weight_dtype_allow_list(const std::string &family) {
+    return find_allow_list(family) != nullptr;
+}
+
+bool weight_dtype_is_supported(const std::string &family, const std::string &dtype) {
+    const DtypeAllowList *list = find_allow_list(family);
+    if (list == nullptr) {
+        return true;
+    }
+    for (const char *const *name = list->allowed; *name != nullptr; ++name) {
+        if (dtype == *name) {
+            return true;
+        }
+    }
+    return false;
+}
+
+std::string supported_weight_dtypes(const std::string &family) {
+    const DtypeAllowList *list = find_allow_list(family);
+    if (list == nullptr) {
+        return {};
+    }
+    std::string out;
+    for (const char *const *name = list->allowed; *name != nullptr; ++name) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += *name;
+    }
+    return out;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/family_gate.h

@@ -0,0 +1,86 @@
+#pragma once
+
+// Decides which audio.cpp family a model path belongs to, and refuses paths
+// this backend must not claim. Standard library only.
+//
+// This is the guard against issue #9287. A model config with no explicit
+// backend makes LocalAI probe every installed backend and bind to the first
+// Load that succeeds, so accepting an arbitrary GGUF here would capture
+// unrelated LLMs. audio.cpp GGUFs carry an audiocpp.model_spec.family metadata
+// key; llama.cpp GGUFs do not.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+// True when the path ends in ".gguf", case insensitively, and has a stem.
+bool path_looks_like_gguf(const std::string &path);
+
+struct FamilyDecision {
+    bool ok = false;
+    std::string family;
+    // Set when ok is false. Suitable verbatim as an INVALID_ARGUMENT message.
+    std::string error;
+};
+
+// Precedence:
+//   1. an explicit `family:` option, so a user can override wrong metadata;
+//   2. for a GGUF, the family embedded in audiocpp.model_spec.family;
+//   3. otherwise refuse.
+// A directory path never consults embedded metadata: there is no single GGUF
+// to read it from.
+FamilyDecision decide_family(bool path_is_gguf, const std::string &embedded_family,
+                             const std::string &configured_family);
+
+// True when `family` can run weights stored as `dtype`, where dtype is the
+// string a TensorMetadata carries ("f32", "f16", "q8_0", "i64", ...).
+//
+// This is a LIST OF FAMILIES THAT CRASH THE PROCESS, not a list of families that
+// perform badly. It exists because the failure is not an exception: loading the
+// supertonic f16 GGUF package reaches ggml_concat with one f16 operand and one
+// f32 one, GGML_ASSERT(a->type == b->type) fails (external/ggml/src/ggml.c:2595)
+// and ggml_abort takes the backend down with SIGABRT on the FIRST request.
+// Nothing upstream of the load can catch that, so an operator sees a model that
+// loaded successfully and a backend that dies on every request with no status
+// and no message.
+//
+// EVIDENCE, per dtype, because the two are not equally attested:
+//   - f16 was OBSERVED to abort here, identically through the unary TTS RPC and
+//     through TTSStream, so it is the packaging and not the streaming path.
+//     Upstream's docs/gguf.md:90 has supertonic's 16-bit column as "---", which
+//     its own legend (:53) defines as not tested, so upstream neither confirms
+//     nor contradicts it.
+//   - q8_0 was NOT run here. Upstream records it as "No (unsupported weight
+//     dtype)" in the same row, which is a weaker claim than the f16 abort: it
+//     says the format is unusable, not that it takes the process down.
+// Both are refused, because the allow list is what the family CAN run (f32 for
+// weights, i64 for the shape and index constants) rather than a list of the
+// dtypes that fail, and a format upstream calls unusable has no business being
+// loaded either way.
+//
+// A family with no entry is unrestricted, which is every family but one.
+//
+// Split out of loaded_model.cpp, where the caller lives, so that the policy is
+// stdlib-only and can be held by a test: the caller needs a real GGUF on disk
+// and an engine, and neither is available to a unit test. What the test pins is
+// that the table says what it is meant to say, so widening it is a deliberate
+// act rather than a typo. It CANNOT pin the removal criterion, which is
+// "upstream fixed it": no test can know that without downloading the package and
+// synthesising, so that step stays a documented manual one at the table itself.
+bool weight_dtype_is_supported(const std::string &family, const std::string &dtype);
+
+// True when `family` has an entry in the table at all, which is the question a
+// caller deciding whether to OPEN THE FILE has to ask. Distinct from
+// "supported_weight_dtypes(family) is empty": that string is also empty for an
+// entry with an empty allow list, and such an entry means "this family can run
+// nothing", which weight_dtype_is_supported already answers by refusing every
+// dtype. Deciding from the string would skip the check on precisely the entry
+// that most needs it.
+bool family_has_weight_dtype_allow_list(const std::string &family);
+
+// The dtypes `family` is restricted to, as "f32, i64", or empty when it is not
+// restricted at all. For the refusal message, so the operator is told what to
+// look for rather than only what is wrong.
+std::string supported_weight_dtypes(const std::string &family);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/family_gate_test.cpp

@@ -0,0 +1,179 @@
+// Unit tests for family_gate. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+//
+// This unit is the guard against issue #9287: when a model config has no
+// explicit backend, LocalAI probes every installed backend and binds to the
+// first Load that succeeds. Accepting an arbitrary GGUF here would capture
+// unrelated LLMs.
+
+#include "family_gate.cpp"
+
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+using namespace audiocpp_backend;
+
+static void test_gguf_suffix_detection() {
+    check(path_looks_like_gguf("/models/chatterbox-q8_0.gguf"), "plain .gguf");
+    check(path_looks_like_gguf("/models/CHATTERBOX.GGUF"), "uppercase .GGUF");
+    check(path_looks_like_gguf("/models/x.GgUf"), "mixed case .GgUf");
+    check(path_looks_like_gguf("a.gguf"), "a one character stem is still a stem");
+    check(!path_looks_like_gguf("/models/chatterbox"), "extensionless directory");
+    check(!path_looks_like_gguf("/models/model.safetensors"), "safetensors");
+    check(!path_looks_like_gguf("/models/gguf"), "a name that is merely 'gguf'");
+    check(!path_looks_like_gguf("/models/GGUF"), "an uppercase name that is merely 'GGUF'");
+    check(!path_looks_like_gguf(""), "empty path");
+    check(!path_looks_like_gguf(".gguf"), "a bare extension is not a model file");
+    // The suffix has to be at the end. A prefix or infix match would let
+    // ".gguf.tmp" download artefacts and ".ggufx" siblings through.
+    check(!path_looks_like_gguf("/models/model.gguf.tmp"), ".gguf in the middle");
+    check(!path_looks_like_gguf("/models/model.ggufx"), "a longer extension");
+    // Every character of the suffix has to match, including the last one.
+    check(!path_looks_like_gguf("/models/model.ggug"), "a near miss in the final character");
+    check(!path_looks_like_gguf("/models/model_gguf"), "a near miss in the first character");
+    check(!path_looks_like_gguf("/models/.gguf-notes"), "a leading .gguf");
+}
+
+static void test_explicit_family_always_wins() {
+    // Explicit configuration beats metadata, so a user can force a family when
+    // upstream metadata is wrong or absent.
+    const auto gguf = decide_family(true, "chatterbox", "omnivoice");
+    check(gguf.ok && gguf.family == "omnivoice", "explicit family overrides GGUF metadata");
+    check(gguf.error.empty(), "an accepted decision carries no error text");
+
+    const auto dir = decide_family(false, "", "qwen3_tts");
+    check(dir.ok && dir.family == "qwen3_tts", "explicit family satisfies a directory path");
+
+    // A GGUF with no embedded spec is still loadable when the user names the
+    // family: the option is an override, not a tie-break that needs metadata to
+    // break against.
+    const auto bare = decide_family(true, "", "supertonic");
+    check(bare.ok && bare.family == "supertonic",
+          "explicit family rescues a GGUF that carries no spec");
+}
+
+static void test_gguf_metadata_supplies_the_family() {
+    const auto d = decide_family(true, "nemotron_asr", "");
+    check(d.ok, "an audio.cpp GGUF loads with no family option");
+    check(d.family == "nemotron_asr", "family comes from the embedded spec");
+    check(d.error.empty(), "an accepted GGUF carries no error text");
+}
+
+// THE GATE. A llama.cpp GGUF has no audiocpp.model_spec.family key.
+static void test_foreign_gguf_is_refused() {
+    const auto d = decide_family(true, "", "");
+    check(!d.ok, "a GGUF with no audio.cpp spec is refused");
+    check(d.family.empty(), "no family is guessed");
+    check(!d.error.empty(), "a refusal always says why");
+    check(d.error.find("audiocpp.model_spec.family") != std::string::npos,
+          "error names the missing metadata key so the cause is diagnosable");
+    check(d.error.find("family:") != std::string::npos,
+          "error names the option that would override it");
+}
+
+static void test_directory_without_family_is_refused() {
+    const auto d = decide_family(false, "", "");
+    check(!d.ok, "a non-GGUF path with no family option is refused");
+    check(d.family.empty(), "a refused directory guesses no family");
+    check(d.error.find("family:") != std::string::npos,
+          "error names the required option");
+}
+
+// A directory path never consults embedded metadata, because there is no single
+// GGUF to read it from.
+static void test_directory_ignores_embedded_family() {
+    const auto d = decide_family(false, "chatterbox", "");
+    check(!d.ok, "a directory is refused even when an embedded family is supplied");
+    check(d.family.empty(), "a refused directory does not adopt the embedded family");
+    // If the GGUF branch ever leaked into the directory branch this message
+    // would start blaming a metadata key that a directory has no place to carry.
+    check(d.error.find("audiocpp.model_spec.family") == std::string::npos,
+          "a directory refusal does not blame GGUF metadata it could not have");
+}
+
+// Pins the weight-dtype allow list. Not a style preference: an entry here is a
+// family that ABORTS THE PROCESS on the first request when handed the wrong
+// dtype, so the model loads and then every request kills the backend with no
+// status and no message.
+//
+// What this test can and cannot do, stated so the next reader does not expect
+// more of it: it pins WHAT THE TABLE SAYS, so widening an entry is a deliberate
+// act rather than a typo, and it pins that an unlisted family is unrestricted.
+// It CANNOT pin the removal criterion, which is "upstream fixed it": knowing
+// that needs the package downloaded and a synthesis run, so it stays a manual
+// step documented at the table in family_gate.cpp.
+static void test_weight_dtype_allow_list() {
+    // The entry that exists, and the exact reason it exists.
+    check(!weight_dtype_is_supported("supertonic", "f16"),
+          "supertonic refuses f16, the package that aborts the process");
+    check(!weight_dtype_is_supported("supertonic", "q8_0"),
+          "supertonic refuses q8_0, which upstream records as unsupported");
+    check(!weight_dtype_is_supported("supertonic", "bf16"),
+          "supertonic refuses bf16, which is untested rather than known good");
+    check(weight_dtype_is_supported("supertonic", "f32"),
+          "supertonic accepts f32, which is what the orig package stores");
+    check(weight_dtype_is_supported("supertonic", "i64"),
+          "supertonic accepts i64: the orig package carries 72 such tensors and "
+          "refusing them would refuse the artifact that works");
+
+    // Every other family is unrestricted, and must stay that way: this guard is
+    // for process death, not for quality.
+    check(weight_dtype_is_supported("nemotron_asr", "q8_0"),
+          "an unlisted family is not restricted");
+    check(weight_dtype_is_supported("citrinet_asr", "f16"),
+          "an unlisted family is not restricted by another family's entry");
+    check(weight_dtype_is_supported("", "anything"),
+          "an empty family name is not restricted");
+
+    // The message the operator reads has to name the remedy, so the refusal is
+    // actionable rather than only correct.
+    check_eq(supported_weight_dtypes("supertonic"), "f32, i64",
+             "the refusal can name what to look for");
+    check_eq(supported_weight_dtypes("nemotron_asr"), "",
+             "an unlisted family reports no restriction");
+
+    // What the caller actually decides on, and it is a DIFFERENT question from
+    // "is the description empty": an entry with an empty allow list would
+    // describe itself as "" while refusing every dtype, so a caller that skipped
+    // the file read on the empty string would skip the check on the one entry
+    // that refuses everything.
+    check(family_has_weight_dtype_allow_list("supertonic"),
+          "a listed family has an allow list");
+    check(!family_has_weight_dtype_allow_list("nemotron_asr"),
+          "an unlisted family has none, which is what lets the caller skip "
+          "opening the file at all");
+    check(!family_has_weight_dtype_allow_list(""),
+          "an empty family name has no allow list");
+}
+
+int main() {
+    test_gguf_suffix_detection();
+    test_explicit_family_always_wins();
+    test_gguf_metadata_supplies_the_family();
+    test_foreign_gguf_is_refused();
+    test_directory_without_family_is_refused();
+    test_directory_ignores_embedded_family();
+    test_weight_dtype_allow_list();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all family_gate checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/generation_request.cpp

@@ -0,0 +1,280 @@
+#include "generation_request.h"
+
+#include <filesystem>
+#include <string>
+#include <system_error>
+#include <utility>
+
+namespace audiocpp_backend {
+namespace {
+
+const char *bool_option(bool value) { return value ? "true" : "false"; }
+
+} // namespace
+
+bool voice_is_reference_file(const std::string &voice) {
+    if (voice.empty()) {
+        return false;
+    }
+    std::error_code ec;
+    return std::filesystem::is_regular_file(std::filesystem::path(voice), ec);
+}
+
+RequestShape build_tts_shape(const backend::TTSRequest &request) {
+    RequestShape shape;
+    shape.has_voice_reference = voice_is_reference_file(request.voice());
+    // !empty() as well as has_instructions(), and it must match the guard in
+    // build_tts_request: a request whose instructions are an empty string
+    // carries no style condition, so telling routing to prefer VoiceDesign for
+    // it would route to a task with nothing to design from.
+    shape.has_instructions =
+        request.has_instructions() && !request.instructions().empty();
+    return shape;
+}
+
+engine::runtime::TaskRequest
+build_tts_request(const backend::TTSRequest &request,
+                  std::optional<engine::runtime::AudioBuffer> reference_audio) {
+    engine::runtime::TaskRequest task;
+
+    // The Transcript, not an option, is where every TTS family reads its
+    // language: chatterbox normalises request.text_input->language into its
+    // voice-clone config, qwen3_tts reads it as out.language, ace_step turns it
+    // into vocal_language. Upstream's own HTTP server does the same and sets no
+    // language option at all (app/server/runtime.cpp build_speech_request).
+    engine::runtime::Transcript transcript;
+    transcript.text = request.text();
+    if (request.has_language()) {
+        transcript.language = request.language();
+    }
+    task.text_input = std::move(transcript);
+
+    engine::runtime::VoiceCondition condition;
+    bool condition_used = false;
+
+    if (reference_audio.has_value()) {
+        // A clip: VoiceReference::audio, at the file's own rate and channel
+        // count. See kVoiceReferenceSampleRate in grpc-server.cpp for why it is
+        // not folded first.
+        engine::runtime::VoiceReference reference;
+        reference.audio = std::move(*reference_audio);
+        condition.speaker = std::move(reference);
+        condition_used = true;
+    } else if (!request.voice().empty()) {
+        // A named preset. cached_voice_id is the channel that actually lands:
+        // supertonic (options.voice), pocket_tts (voice_config.preset_name),
+        // voxcpm2, vibevoice, fish_audio (its saved-reference lookup) and
+        // qwen3_tts CustomVoice all read request.voice->speaker->cached_voice_id,
+        // and upstream's own server puts a non-preset `voice` body field in
+        // exactly this slot (app/server/runtime.cpp build_speech_request).
+        engine::runtime::VoiceReference reference;
+        reference.cached_voice_id = request.voice();
+        condition.speaker = std::move(reference);
+        condition_used = true;
+        // Forward-tolerant alias only. A bare "voice" REQUEST OPTION is read by
+        // no family in the pinned upstream: grepping find_option for it returns
+        // nothing. It is sent so a family adopting the name later works with no
+        // change here, not because it does anything today.
+        task.options["voice"] = request.voice();
+    }
+
+    if (request.has_instructions() && !request.instructions().empty()) {
+        // "instruct" is the key upstream itself maps the OpenAI `instructions`
+        // body field onto (app/server/runtime.cpp: request.options["instruct"]
+        // = value->as_string()), and it is read: qwen3_tts VoiceDesign and
+        // CustomVoice both take find_option(options, {"instruct"}) first, and
+        // omnivoice reads it in resolve_instruct.
+        task.options["instruct"] = request.instructions();
+        // "caption" is irodori_tts's name for the same thing, read in its
+        // make_request and documented as the voice-design caption for the 600M
+        // VoiceDesign model (docs/tts.md). Without this, that family's voice
+        // design cannot be driven from this RPC at all.
+        task.options["caption"] = request.instructions();
+        // The proto field's own name, forwarded for the same forward-tolerant
+        // reason as "voice" above and with the same honest accounting: NO family
+        // in the pinned upstream reads a request option called "instructions".
+        task.options["instructions"] = request.instructions();
+
+        engine::runtime::StyleCondition style;
+        // "instruct", not "instructions". This tag IS read, and only under that
+        // spelling: omnivoice and qwen3_tts both fall back to
+        // request.voice->style->tags.find("instruct") when the option is absent.
+        // Spelling it "instructions" here would have made the whole
+        // StyleCondition dead weight.
+        style.tags["instruct"] = request.instructions();
+        // !empty(), matching the option emission below, and load-bearing rather
+        // than tidiness. core/backend/tts.go's newTTSRequest sets
+        // `Language: &language` UNCONDITIONALLY, so has_language() is true on
+        // every request LocalAI sends and carries "" whenever the caller named
+        // no language. An engaged-but-empty style language is WORSE than an
+        // absent one: supertonic reads text_input->language behind its own
+        // !empty() guard and then OVERRIDES it from style->language with no
+        // guard at all (supertonic/session.cpp), so "" would replace its "en"
+        // default and tokenizer_text.cpp would throw
+        // "invalid Supertonic language: " on every request that set
+        // instructions and no language.
+        if (request.has_language() && !request.language().empty()) {
+            style.language = request.language();
+        }
+        condition.style = std::move(style);
+        condition_used = true;
+    }
+
+    if (condition_used) {
+        task.voice = std::move(condition);
+    }
+
+    if (request.has_language() && !request.language().empty()) {
+        // Forward-tolerant alias, exactly as in build_transcription_request. The
+        // families that read a "language" request option are the ASR ones
+        // (nemotron_asr, hviske_asr, vibevoice_asr, higgs_audio_stt), none of
+        // which this RPC can route to; pocket_tts reads one but from its
+        // ModelLoadRequest at load time, not from here. The Transcript above is
+        // what actually carries the language to a TTS family.
+        task.options["language"] = request.language();
+    }
+
+    // LAST, so an explicit params entry wins over anything derived above. That
+    // matters for "caption": a caller who sets params[caption] has named the
+    // exact string they want, and it must not be overwritten by `instructions`.
+    for (const auto &param : request.params()) {
+        task.options[param.first] = param.second;
+    }
+    return task;
+}
+
+engine::runtime::TaskRequest
+build_sound_generation_request(const backend::SoundGenerationRequest &request,
+                               std::optional<engine::runtime::AudioBuffer> source_audio) {
+    engine::runtime::TaskRequest task;
+
+    engine::runtime::Transcript transcript;
+    transcript.text = request.text();
+    if (request.has_language()) {
+        transcript.language = request.language();
+    }
+    task.text_input = std::move(transcript);
+
+    // src is the input clip for the editing routes. ace_step's repaint, cover
+    // and edit routes need it; stable_audio uses it as init_audio or
+    // inpaint_audio; heartmula refuses it outright.
+    if (source_audio.has_value()) {
+        task.audio_input = std::move(*source_audio);
+    }
+
+    // WHAT LANDS AND WHAT DOES NOT. Three families advertise AudioGeneration in
+    // the pinned upstream: ace_step, heartmula and stable_audio. Every key below
+    // was grepped against find_option/parse_*_option in src/ and include/ rather
+    // than assumed, because a key nobody reads is not a feature and shipping one
+    // while implying it works is the mistake this comment exists to prevent.
+    //
+    // Unknown REQUEST options cannot turn a valid request into an error:
+    // families look theirs up by name and ignore the rest, and the unknown-key
+    // refusals upstream does have are on SESSION options, which arrive at load
+    // time. So a forward-tolerant alias is free; it is just not a feature.
+
+    if (request.has_duration()) {
+        // duration_seconds is the key that works, and it works everywhere:
+        // ace_step (request_parser.cpp), heartmula (session.cpp, which also
+        // refuses a non-positive value) and stable_audio (request.cpp) all read
+        // it. This is the SoundGeneration analogue of Task 9's return_timestamps.
+        task.options["duration_seconds"] = std::to_string(request.duration());
+        // The proto field's own name. Read by exactly one family, omnivoice, and
+        // omnivoice advertises Tts rather than AudioGeneration, so this RPC can
+        // never route to it: DEAD here, kept only as a forward-tolerant alias.
+        task.options["duration"] = std::to_string(request.duration());
+    }
+    if (request.has_temperature()) {
+        // Read by heartmula. ace_step's sampling temperature is a different,
+        // narrower knob it calls lm_temperature (it drives the caption/thinking
+        // LM, not the audio diffusion), so this is deliberately NOT mapped onto
+        // it; a caller who wants it sets it through the request options that
+        // reach ace_step by name. stable_audio has no temperature at all.
+        task.options["temperature"] = std::to_string(request.temperature());
+    }
+    if (request.has_sample()) {
+        // do_sample is read widely upstream, but only by TTS and ASR families
+        // (chatterbox, index_tts2, miotts, moss, qwen3_tts, vibevoice,
+        // hviske_asr, voxtral_realtime). NO AudioGeneration family reads it, so
+        // it is dead on this route.
+        task.options["do_sample"] = bool_option(request.sample());
+    }
+    if (request.has_src_divisor()) {
+        // Read by nobody, anywhere in the pinned upstream. Forwarded because the
+        // proto documents it as part of this request and a family adopting it
+        // then works unchanged.
+        task.options["src_divisor"] = std::to_string(request.src_divisor());
+    }
+    if (request.has_think()) {
+        // "thinking" is the key ace_step actually reads (request_parser.cpp),
+        // which is why it is sent alongside the proto's own "think". "think" on
+        // its own is read by nobody.
+        task.options["thinking"] = bool_option(request.think());
+        task.options["think"] = bool_option(request.think());
+    }
+    if (request.has_caption()) {
+        // Read only by irodori_tts, which advertises Tts/VoiceCloning/VoiceDesign
+        // and not AudioGeneration, so it is unreachable from this RPC: DEAD here.
+        task.options["caption"] = request.caption();
+    }
+    if (request.has_lyrics()) {
+        // Read by ace_step and heartmula.
+        task.options["lyrics"] = request.lyrics();
+    }
+    if (request.has_bpm()) {
+        // Read by ace_step.
+        task.options["bpm"] = std::to_string(request.bpm());
+    }
+    if (request.has_keyscale()) {
+        // Read by ace_step.
+        task.options["keyscale"] = request.keyscale();
+    }
+    if (request.has_timesignature()) {
+        // Read by ace_step.
+        task.options["timesignature"] = request.timesignature();
+    }
+    if (request.has_instrumental()) {
+        // Read by nobody: "instrumental" appears in the pinned upstream only as
+        // a roformer STEM NAME, never as a request option. A forward-tolerant
+        // alias and nothing more.
+        task.options["instrumental"] = bool_option(request.instrumental());
+    }
+    if (request.has_language() && !request.language().empty()) {
+        // Alias again: the Transcript above is what ace_step reads as
+        // vocal_language. No AudioGeneration family reads a "language" option.
+        task.options["language"] = request.language();
+    }
+    return task;
+}
+
+bool apply_transform_text_input(engine::runtime::TaskRequest &task) {
+    // Canonical first, alias second, and an empty value falls through to the
+    // next candidate rather than ending the search: a caller who sent
+    // target_text="" and text="the real one" meant the second one.
+    static const char *const kTextKeys[] = {"target_text", "text"};
+
+    std::string text;
+    for (const char *key : kTextKeys) {
+        const auto found = task.options.find(key);
+        if (found != task.options.end() && !found->second.empty()) {
+            text = found->second;
+            break;
+        }
+    }
+    if (text.empty()) {
+        return false;
+    }
+
+    engine::runtime::Transcript transcript;
+    transcript.text = std::move(text);
+    // Inside the has-text branch on purpose. See the header: a language on its
+    // own conditions nothing and must not manufacture a text_input.
+    const auto language = task.options.find("language");
+    if (language != task.options.end()) {
+        transcript.language = language->second;
+    }
+    task.text_input = std::move(transcript);
+    return true;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/generation_request.h

@@ -0,0 +1,108 @@
+#pragma once
+
+// Builds the engine::runtime::TaskRequest for the two audio-PRODUCING offline
+// RPCs, TTS and SoundGeneration, and answers the one filesystem question TTS
+// routing depends on.
+//
+// It is a unit of its own rather than a pair of statics in grpc-server.cpp so
+// that it can be tested: grpc-server.cpp has a main() and cannot be linked into
+// a test binary, and everything here is a pure function of its arguments once
+// the file read has been lifted out (which is why the reference clip arrives as
+// an already-read buffer rather than a path). TTSStream reuses build_tts_request
+// unchanged.
+//
+// Only the plain structs in engine/framework/runtime/session.h are touched, so
+// this compiles against the header without linking engine_runtime, the same way
+// result_map does.
+
+#include "backend.pb.h"
+#include "capability_routing.h"
+
+#include "engine/framework/runtime/session.h"
+
+#include <optional>
+#include <string>
+
+namespace audiocpp_backend {
+
+// True when TTSRequest.voice names an existing regular file, in which case it
+// is a speaker reference clip and routing prefers VoiceCloning; false when it is
+// a named preset (or empty).
+//
+// The overload is LocalAI's, not this backend's: `voice` is the OpenAI speech
+// field and different LocalAI backends have always read it both ways. Deciding
+// it from the filesystem needs no new option and matches how somebody actually
+// configures a cloning family, which is by pointing at a clip.
+//
+// A DIRECTORY is deliberately not a reference: is_regular_file, not exists. A
+// directory named as a voice cannot be read as a WAV, and treating it as a
+// reference would turn a preset typo into "cannot read /x as WAV" instead of
+// letting it travel as the preset name it looks like.
+//
+// The error_code overload is used so an unreadable parent directory answers
+// false rather than throwing. That is the right answer here: the name is then
+// passed on as a preset, and if it really was meant to be a clip the family
+// refuses a request it cannot serve, which is a better message than a
+// filesystem exception thrown while classifying a string.
+bool voice_is_reference_file(const std::string &voice);
+
+// Everything routing needs to know about a TTSRequest, in one place, so that TTS
+// and TTSStream cannot describe the same request differently.
+//
+// `pinned_task` is deliberately NOT filled here: it comes off the LoadedModel,
+// not off the request, and this unit links no engine. The caller must still
+// write `shape.pinned_task = model->pinned_task();` or the model's `task:`
+// option is dead. That is the one field a new handler can forget, so it is the
+// one field left visible at the call site rather than hidden behind this
+// helper.
+RequestShape build_tts_shape(const backend::TTSRequest &request);
+
+// `reference_audio` is the already-read speaker clip, present exactly when
+// voice_is_reference_file(request.voice()) was true. Passing it in rather than a
+// path keeps this function pure and lets the caller do the read where the
+// ordering rules (capability refusal first, then the lane) are enforced.
+//
+// It is taken BY VALUE and moved in: a reference clip is seconds of audio and
+// the caller has no use for it afterwards.
+engine::runtime::TaskRequest
+build_tts_request(const backend::TTSRequest &request,
+                  std::optional<engine::runtime::AudioBuffer> reference_audio);
+
+// `source_audio` is SoundGenerationRequest.src already read, present exactly
+// when the field was set and non-empty. Same reasoning as above.
+engine::runtime::TaskRequest
+build_sound_generation_request(const backend::SoundGenerationRequest &request,
+                               std::optional<engine::runtime::AudioBuffer> source_audio);
+
+// Lifts a text-conditioned transform route's text out of the request params
+// into TaskRequest.text_input, and reports whether it set one.
+//
+// WHY THIS EXISTS. AudioTransform is an audio-in / audio-out RPC and its proto
+// message has no text field, but not every task it routes to is audio-only.
+// vevo2's speech-to-speech and prosody routes read their text from
+// request.text_input (src/models/vevo2/session.cpp fills refs.target_text from
+// exactly there and nowhere else) and refuse the run without one: "Vevo2
+// text/prosody route requires text_input or target_text". The params map is the
+// only channel AudioTransform has that reaches the engine, so the text travels
+// through it and is unpacked here. Without this, s2s is not merely awkward to
+// reach through this RPC, it is unreachable.
+//
+// CALL IT AFTER the params have been copied into task.options, and note that it
+// does NOT erase the keys it reads. vevo2's loader advertises "target_text" in
+// its own documented request-option table, so a family that looks there keeps
+// finding it; the copy in text_input is what the session actually reads today.
+//
+// "target_text" is canonical and "text" is its alias, the same order vevo2's
+// option table declares them in. A request setting both gets target_text, so
+// the canonical spelling wins rather than whichever the map happened to store
+// first. An empty value is not a text: it means the caller sent the key with
+// nothing in it, and a family asked to vocalise "" should say so itself rather
+// than be handed an empty Transcript that looks deliberate.
+//
+// "language" rides along when a text was found, and only then. On its own it
+// conditions nothing, and setting text_input for it alone would turn a plain
+// separation request that happened to carry a language hint into a text-routed
+// one.
+bool apply_transform_text_input(engine::runtime::TaskRequest &task);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/generation_request_ctest.cpp

@@ -0,0 +1,575 @@
+// Tests for the TTS and SoundGeneration request builders, and for the
+// filesystem rule that decides whether TTSRequest.voice is a speaker reference
+// clip or a named preset.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE: see the note at the top of
+// result_map_ctest.cpp. This file needs the generated protobuf messages and the
+// audio.cpp include path, neither of which backend/cpp/run-unit-tests.sh
+// provides, so it is built and run by ctest:
+//
+//   make -C backend/cpp/audio-cpp test-engine
+//
+// The assertions on OPTION KEYS are the point of this file, not decoration.
+// Every one of them names a key that was grepped against the pinned upstream:
+// "instruct" is read, "instructions" is not; "duration_seconds" is read,
+// "duration" is not. A rename that looks harmless is exactly the change that
+// silently stops a family honouring the request, so the spellings are pinned
+// here rather than left to a comment.
+
+#include "generation_request.h"
+
+#include <cstdio>
+#include <filesystem>
+#include <fstream>
+#include <string>
+#include <unordered_map>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+static bool has_key(const std::unordered_map<std::string, std::string> &options,
+                    const std::string &key) {
+    return options.find(key) != options.end();
+}
+
+static std::string option_or(
+    const std::unordered_map<std::string, std::string> &options,
+    const std::string &key, const std::string &fallback) {
+    const auto it = options.find(key);
+    return it == options.end() ? fallback : it->second;
+}
+
+static engine::runtime::AudioBuffer clip(int sample_rate, int channels) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = channels;
+    // Distinguishable content, so a builder that swapped one buffer for another
+    // (or default-constructed one) is visible rather than merely a size change.
+    buffer.samples = {0.25f, -0.5f, 0.75f, -1.0f};
+    return buffer;
+}
+
+static void test_voice_is_reference_file() {
+    const auto dir = std::filesystem::temp_directory_path() /
+                     "audiocpp-generation-request-ctest";
+    std::filesystem::remove_all(dir);
+    std::filesystem::create_directories(dir);
+
+    const auto file = dir / "reference.wav";
+    {
+        std::ofstream out(file, std::ios::binary);
+        out << "not really a wav, but a regular file";
+    }
+    const auto subdir = dir / "a-directory";
+    std::filesystem::create_directories(subdir);
+
+    check(!voice_is_reference_file(""), "voice_is_reference_file: empty");
+    check(!voice_is_reference_file("alloy"),
+          "voice_is_reference_file: bare preset name");
+    check(!voice_is_reference_file((dir / "absent.wav").string()),
+          "voice_is_reference_file: missing path");
+    check(voice_is_reference_file(file.string()),
+          "voice_is_reference_file: existing regular file");
+    // A directory is NOT a reference. exists() would say yes here and the read
+    // would then fail with "cannot read <dir> as WAV", which sends the operator
+    // after a file problem instead of a preset typo.
+    check(!voice_is_reference_file(subdir.string()),
+          "voice_is_reference_file: directory is not a reference");
+
+    std::filesystem::remove_all(dir);
+}
+
+// build_tts_shape is what TTS and TTSStream both hand to routing, so a wrong
+// answer here silently changes which task a request runs as, with a 200 and no
+// diagnostic. Every field is asserted in both directions.
+static void test_tts_shape() {
+    const auto dir = std::filesystem::temp_directory_path() /
+                     "audiocpp-generation-request-ctest-shape";
+    std::filesystem::remove_all(dir);
+    std::filesystem::create_directories(dir);
+    const auto file = dir / "reference.wav";
+    {
+        std::ofstream out(file, std::ios::binary);
+        out << "a regular file";
+    }
+
+    {
+        backend::TTSRequest request;
+        request.set_text("hello");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference && !shape.has_instructions,
+              "shape: bare request has neither signal");
+        // Never filled here: it comes off the LoadedModel, and leaving it empty
+        // is what makes the caller's assignment visible at the call site.
+        check(shape.pinned_task.empty(), "shape: pinned_task is left to the caller");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(file.string());
+        const auto shape = build_tts_shape(request);
+        check(shape.has_voice_reference,
+              "shape: an existing file is a voice reference");
+        check(!shape.has_instructions, "shape: a clip is not an instruction");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice("alloy");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference,
+              "shape: a preset name is not a voice reference");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(dir.string());
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference,
+              "shape: a directory is not a voice reference");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_instructions("a calm older man");
+        const auto shape = build_tts_shape(request);
+        check(shape.has_instructions, "shape: instructions are seen");
+        check(!shape.has_voice_reference,
+              "shape: instructions do not imply a reference");
+    }
+    {
+        // The guard that has to match build_tts_request's. An empty
+        // instructions string builds no style condition, so telling routing to
+        // prefer VoiceDesign for it would route to a task with nothing to
+        // design from.
+        backend::TTSRequest request;
+        request.set_instructions("");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_instructions,
+              "shape: an empty instructions string is not an instruction");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(file.string());
+        request.set_instructions("a calm older man");
+        const auto shape = build_tts_shape(request);
+        check(shape.has_voice_reference && shape.has_instructions,
+              "shape: both signals are reported when both are set");
+    }
+
+    std::filesystem::remove_all(dir);
+}
+
+static void test_tts_plain() {
+    backend::TTSRequest request;
+    request.set_text("hello there");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.text_input.has_value() && task.text_input->text == "hello there",
+          "tts: text reaches the transcript");
+    // No voice and no instructions means NO voice condition at all. A builder
+    // that always emitted one would make every family think a speaker was
+    // named, and chatterbox in particular refuses a prepare whose voice
+    // condition carries neither audio nor anything else it can use.
+    check(!task.voice.has_value(), "tts: no voice condition when nothing is set");
+    check(task.options.empty(), "tts: no options when nothing is set");
+    check(!task.audio_input.has_value(), "tts: no audio input");
+}
+
+static void test_tts_named_preset() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("alloy");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.voice.has_value() && task.voice->speaker.has_value(),
+          "tts preset: speaker condition present");
+    check(task.voice->speaker->cached_voice_id.has_value() &&
+              *task.voice->speaker->cached_voice_id == "alloy",
+          "tts preset: lands in cached_voice_id");
+    // The clip slot must stay empty, or a cloning family would try to prepare
+    // conditionals from a default-constructed buffer.
+    check(!task.voice->speaker->audio.has_value(),
+          "tts preset: no reference audio");
+    check(!task.voice->style.has_value(), "tts preset: no style condition");
+    check(option_or(task.options, "voice", "") == "alloy",
+          "tts preset: forwarded as the voice option too");
+}
+
+static void test_tts_reference_clip() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("/tmp/reference.wav");
+
+    const auto task = build_tts_request(request, clip(44100, 2));
+
+    check(task.voice.has_value() && task.voice->speaker.has_value(),
+          "tts clip: speaker condition present");
+    check(task.voice->speaker->audio.has_value(),
+          "tts clip: reference audio present");
+    // Rate and channels survive untouched. This is the assertion that fails if
+    // anybody decides to fold the clip to 16 kHz mono on the way in.
+    check(task.voice->speaker->audio->sample_rate == 44100 &&
+              task.voice->speaker->audio->channels == 2 &&
+              task.voice->speaker->audio->samples.size() == 4,
+          "tts clip: rate, channels and samples pass through unchanged");
+    // A clip is NOT also a cached voice id, and the path must not travel as a
+    // preset name: a family reading cached_voice_id would then look up a voice
+    // called "/tmp/reference.wav".
+    check(!task.voice->speaker->cached_voice_id.has_value(),
+          "tts clip: no cached_voice_id");
+    check(!has_key(task.options, "voice"), "tts clip: no voice option");
+}
+
+static void test_tts_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("a calm older man, speaking slowly");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(option_or(task.options, "instruct", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: instruct option is the one qwen3_tts reads");
+    check(option_or(task.options, "caption", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: caption option is the one irodori_tts reads");
+    check(option_or(task.options, "instructions", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: proto field name forwarded as an alias");
+    check(task.voice.has_value() && task.voice->style.has_value(),
+          "tts instructions: style condition present");
+    // "instruct", not "instructions". omnivoice and qwen3_tts both look this tag
+    // up by that exact name and by no other.
+    check(option_or(task.voice->style->tags, "instruct", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: style tag is spelled instruct");
+    check(!has_key(task.voice->style->tags, "instructions"),
+          "tts instructions: style tag is NOT spelled instructions");
+    // Instructions alone must not invent a speaker: has_voice_reference is what
+    // routing keys VoiceCloning off, and a speaker here would make a
+    // clone-capable family expect a clip it never received.
+    check(!task.voice->speaker.has_value(),
+          "tts instructions: no speaker without a voice");
+}
+
+static void test_tts_empty_instructions_are_not_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    // has_instructions() is true here, because the field was set. An empty
+    // string is still no instruction, and forwarding it would set an empty
+    // instruct option that qwen3_tts would prefer over its style tag fallback.
+    check(!has_key(task.options, "instruct"),
+          "tts: an empty instructions string sets no instruct option");
+    check(!task.voice.has_value(),
+          "tts: an empty instructions string sets no voice condition");
+}
+
+// THE EXACT SHAPE LocalAI PUTS ON THE WIRE. core/backend/tts.go's newTTSRequest
+// sets Language: &language UNCONDITIONALLY, so has_language() is true on every
+// request that ever reaches this backend, carrying an empty string whenever the
+// caller named no language.
+//
+// An empty StyleCondition::language is not a harmless default. supertonic reads
+// text_input->language behind a !empty() guard and then OVERRIDES it from
+// style->language whenever that optional is engaged, with no guard at all
+// (supertonic/session.cpp generation_options_from_request), so an empty style
+// language replaces its "en" default (session.h) with "" and
+// tokenizer_text.cpp's preprocess throws "invalid Supertonic language: ".
+// Every /v1/audio/speech request carrying instructions and no language would be
+// an INTERNAL. A plain request never sees it, because the style condition only
+// exists when instructions are non-empty.
+static void test_tts_empty_language_is_not_a_language() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("a calm older man");
+    request.set_language("");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.voice.has_value() && task.voice->style.has_value(),
+          "tts empty language: the style condition still exists");
+    check(!task.voice->style->language.has_value(),
+          "tts empty language: style language is left unset, not set to empty");
+    // The option emission has always guarded on !empty(); this pins the two to
+    // the same rule so they cannot drift apart again.
+    check(!has_key(task.options, "language"),
+          "tts empty language: no language option");
+    check(task.text_input.has_value() && task.text_input->language.empty(),
+          "tts empty language: transcript language stays empty");
+}
+
+// The one shape routing treats specially: a clip outranks instructions, and the
+// VoiceCondition then has to carry BOTH, because the family that wins is chosen
+// on the clip but may still read the style tag.
+static void test_tts_clip_and_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("/tmp/reference.wav");
+    request.set_instructions("bright and fast");
+    request.set_language("en");
+
+    const auto task = build_tts_request(request, clip(22050, 1));
+
+    check(task.voice.has_value(), "tts clip+instructions: voice condition present");
+    check(task.voice->speaker.has_value() &&
+              task.voice->speaker->audio.has_value() &&
+              task.voice->speaker->audio->sample_rate == 22050,
+          "tts clip+instructions: speaker carries the clip");
+    check(!task.voice->speaker->cached_voice_id.has_value(),
+          "tts clip+instructions: the clip path is not also a preset id");
+    check(task.voice->style.has_value() &&
+              option_or(task.voice->style->tags, "instruct", "") == "bright and fast",
+          "tts clip+instructions: style carries the instruct tag");
+    check(task.voice->style->language.has_value() &&
+              *task.voice->style->language == "en",
+          "tts clip+instructions: a real language does reach the style condition");
+    check(option_or(task.options, "instruct", "") == "bright and fast",
+          "tts clip+instructions: instruct option still emitted");
+    check(!has_key(task.options, "voice"),
+          "tts clip+instructions: still no voice option for a clip");
+}
+
+static void test_tts_language_and_params() {
+    backend::TTSRequest request;
+    request.set_text("ciao");
+    request.set_language("it");
+    request.set_instructions("warm");
+    (*request.mutable_params())["exaggeration"] = "0.7";
+    // An explicit param must win over the value derived from instructions.
+    (*request.mutable_params())["caption"] = "explicitly chosen caption";
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.text_input.has_value() && task.text_input->language == "it",
+          "tts language: reaches the transcript");
+    check(option_or(task.options, "language", "") == "it",
+          "tts language: forwarded as an option alias");
+    check(task.voice.has_value() && task.voice->style.has_value() &&
+              task.voice->style->language.has_value() &&
+              *task.voice->style->language == "it",
+          "tts language: reaches the style condition");
+    check(option_or(task.options, "exaggeration", "") == "0.7",
+          "tts params: passed through verbatim");
+    check(option_or(task.options, "caption", "") == "explicitly chosen caption",
+          "tts params: an explicit param overrides the derived caption");
+}
+
+static void test_sound_generation_minimal() {
+    backend::SoundGenerationRequest request;
+    request.set_text("a distant thunderstorm");
+
+    const auto task = build_sound_generation_request(request, std::nullopt);
+
+    check(task.text_input.has_value() &&
+              task.text_input->text == "a distant thunderstorm",
+          "sound: text reaches the transcript");
+    // Unset optionals must emit NOTHING. Emitting a zero for an unset duration
+    // would make heartmula refuse the request ("duration_seconds must be
+    // positive") on a request that never mentioned a duration.
+    check(task.options.empty(), "sound: unset optionals emit no options");
+    check(!task.audio_input.has_value(), "sound: no audio input without src");
+    check(!task.voice.has_value(), "sound: no voice condition");
+}
+
+static void test_sound_generation_full() {
+    backend::SoundGenerationRequest request;
+    request.set_text("a slow blues in E");
+    request.set_duration(30.0f);
+    request.set_temperature(0.8f);
+    request.set_sample(false);
+    request.set_src_divisor(4);
+    request.set_think(true);
+    request.set_caption("smoky bar recording");
+    request.set_lyrics("first line\nsecond line");
+    request.set_bpm(72);
+    request.set_keyscale("E minor");
+    request.set_language("en");
+    request.set_timesignature("4/4");
+    request.set_instrumental(true);
+
+    const auto task = build_sound_generation_request(request, clip(48000, 2));
+
+    // duration_seconds is the key every AudioGeneration family actually reads;
+    // "duration" rides along as an alias. Both spellings are pinned so a
+    // "cleanup" that keeps only the proto's own name is a test failure and not
+    // a silent loss of the duration.
+    check(option_or(task.options, "duration_seconds", "").rfind("30.", 0) == 0,
+          "sound: duration lands as duration_seconds");
+    check(option_or(task.options, "duration", "").rfind("30.", 0) == 0,
+          "sound: duration also forwarded under its own name");
+    check(option_or(task.options, "temperature", "").rfind("0.8", 0) == 0,
+          "sound: temperature forwarded");
+    // Set to FALSE, so this also proves the key is written whenever the field is
+    // present rather than only when the value is truthy.
+    check(option_or(task.options, "do_sample", "") == "false",
+          "sound: sample=false is forwarded as do_sample=false");
+    check(option_or(task.options, "src_divisor", "") == "4",
+          "sound: src_divisor forwarded");
+    check(option_or(task.options, "thinking", "") == "true",
+          "sound: think lands as thinking, the key ace_step reads");
+    check(option_or(task.options, "think", "") == "true",
+          "sound: think also forwarded under its own name");
+    check(option_or(task.options, "caption", "") == "smoky bar recording",
+          "sound: caption forwarded");
+    check(option_or(task.options, "lyrics", "") == "first line\nsecond line",
+          "sound: lyrics forwarded");
+    check(option_or(task.options, "bpm", "") == "72", "sound: bpm forwarded");
+    check(option_or(task.options, "keyscale", "") == "E minor",
+          "sound: keyscale forwarded");
+    check(option_or(task.options, "timesignature", "") == "4/4",
+          "sound: timesignature forwarded");
+    check(option_or(task.options, "instrumental", "") == "true",
+          "sound: instrumental forwarded");
+    check(option_or(task.options, "language", "") == "en",
+          "sound: language forwarded as an option alias");
+    check(task.text_input->language == "en",
+          "sound: language reaches the transcript, which is what ace_step reads");
+    check(task.audio_input.has_value() &&
+              task.audio_input->sample_rate == 48000 &&
+              task.audio_input->channels == 2 &&
+              task.audio_input->samples.size() == 4,
+          "sound: src passes through at its own rate and channel count");
+}
+
+
+// ---------------------------------------------------------------------------
+// apply_transform_text_input
+//
+// AudioTransform has no text field on the wire, so a text-conditioned route
+// (vevo2's speech-to-speech) can only be reached if the text travels as a
+// param and is unpacked into text_input. Every assertion below pins a spelling
+// or a precedence that a family actually depends on, not a shape that merely
+// looks tidy.
+
+static void test_transform_text_absent() {
+    engine::runtime::TaskRequest task;
+    task.options["stem"] = "vocals";
+    check(!apply_transform_text_input(task),
+          "transform text: reports false when no text key is present");
+    check(!task.text_input.has_value(),
+          "transform text: a request with no text keeps text_input unset");
+}
+
+static void test_transform_text_canonical_key() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "sing this line";
+    check(apply_transform_text_input(task), "transform text: target_text reports true");
+    check(task.text_input.has_value() && task.text_input->text == "sing this line",
+          "transform text: target_text becomes text_input.text");
+    check(has_key(task.options, "target_text"),
+          "transform text: target_text survives in options for families that read it there");
+}
+
+static void test_transform_text_alias_key() {
+    engine::runtime::TaskRequest task;
+    task.options["text"] = "say this instead";
+    check(apply_transform_text_input(task), "transform text: text alias reports true");
+    check(task.text_input.has_value() && task.text_input->text == "say this instead",
+          "transform text: the text alias becomes text_input.text");
+}
+
+static void test_transform_text_canonical_wins() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "canonical";
+    task.options["text"] = "alias";
+    check(apply_transform_text_input(task), "transform text: both keys reports true");
+    check(task.text_input.has_value() && task.text_input->text == "canonical",
+          "transform text: target_text wins over text, not whichever hashed first");
+}
+
+static void test_transform_text_empty_is_not_a_text() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "";
+    check(!apply_transform_text_input(task),
+          "transform text: an empty target_text reports false");
+    check(!task.text_input.has_value(),
+          "transform text: an empty target_text leaves text_input unset");
+}
+
+static void test_transform_text_empty_canonical_falls_through_to_alias() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "";
+    task.options["text"] = "the real one";
+    check(apply_transform_text_input(task),
+          "transform text: an empty canonical key does not mask a usable alias");
+    check(task.text_input.has_value() && task.text_input->text == "the real one",
+          "transform text: the alias is used when the canonical key is empty");
+}
+
+static void test_transform_text_language_rides_along() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "vocalise me";
+    task.options["language"] = "ja";
+    check(apply_transform_text_input(task), "transform text: text plus language reports true");
+    check(task.text_input.has_value() && task.text_input->language == "ja",
+          "transform text: language lands on the Transcript alongside the text");
+    check(has_key(task.options, "language"),
+          "transform text: language survives in options too");
+}
+
+static void test_transform_language_alone_is_not_a_text() {
+    engine::runtime::TaskRequest task;
+    task.options["language"] = "ja";
+    check(!apply_transform_text_input(task),
+          "transform text: a language with no text reports false");
+    check(!task.text_input.has_value(),
+          "transform text: a language alone must not route a separation request through text");
+}
+
+static void test_transform_text_preserves_other_inputs() {
+    engine::runtime::TaskRequest task;
+    engine::runtime::AudioBuffer audio;
+    audio.sample_rate = 44100;
+    audio.channels = 2;
+    audio.samples = {0.1f, 0.2f, 0.3f, 0.4f};
+    task.audio_input = audio;
+    task.options["target_text"] = "keep the audio";
+    check(apply_transform_text_input(task), "transform text: with audio present reports true");
+    check(task.audio_input.has_value() && task.audio_input->samples.size() == 4 &&
+              task.audio_input->sample_rate == 44100,
+          "transform text: the source audio is untouched");
+}
+
+int main() {
+    test_voice_is_reference_file();
+    test_tts_shape();
+    test_tts_plain();
+    test_tts_named_preset();
+    test_tts_reference_clip();
+    test_tts_instructions();
+    test_tts_empty_instructions_are_not_instructions();
+    test_tts_empty_language_is_not_a_language();
+    test_tts_clip_and_instructions();
+    test_tts_language_and_params();
+    test_sound_generation_minimal();
+    test_sound_generation_full();
+    test_transform_text_absent();
+    test_transform_text_canonical_key();
+    test_transform_text_alias_key();
+    test_transform_text_canonical_wins();
+    test_transform_text_empty_is_not_a_text();
+    test_transform_text_empty_canonical_falls_through_to_alias();
+    test_transform_text_language_rides_along();
+    test_transform_language_alone_is_not_a_text();
+    test_transform_text_preserves_other_inputs();
+
+    if (failures != 0) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/inference_lane.cpp

@@ -0,0 +1,114 @@
+#include "inference_lane.h"
+
+#include <algorithm>
+#include <chrono>
+
+namespace audiocpp_backend {
+
+std::int64_t monotonic_millis() {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+               std::chrono::steady_clock::now().time_since_epoch())
+        .count();
+}
+
+int resolve_wait_budget_ms(int policy_ceiling_ms, int request_hint_ms) {
+    // Both sides normalise to 0 for "not specified", which is also the value
+    // that means unbounded on the way out, so the unspecified cases fall out of
+    // the arithmetic instead of needing their own branches.
+    const int ceiling = policy_ceiling_ms > 0 ? policy_ceiling_ms : 0;
+    const int hint = request_hint_ms > 0 ? request_hint_ms : 0;
+
+    if (ceiling == 0) {
+        return hint;
+    }
+    if (hint == 0) {
+        return ceiling;
+    }
+    // The hint can tighten the ceiling but never loosen it.
+    return std::min(ceiling, hint);
+}
+
+bool run_exceeds_budget(bool lane_occupied, std::int64_t run_started_ms,
+                        std::int64_t now_ms, int budget_ms) {
+    if (!lane_occupied || budget_ms <= 0) {
+        return false;
+    }
+    return now_ms - run_started_ms > static_cast<std::int64_t>(budget_ms);
+}
+
+namespace {
+
+std::string busy_prefix(const std::string &model_label) {
+    return "inference lane for model '" + model_label + "' is busy: ";
+}
+
+// States the measurement and nothing else. A short-budget caller meeting a
+// legitimately long run lands here too, so this text must not declare the run
+// broken; the numbers let a reader decide that for themselves.
+std::string overrun_message(const std::string &model_label,
+                            std::int64_t run_age_ms, int budget_ms) {
+    return busy_prefix(model_label) + "the in-flight run has been running for " +
+           std::to_string(run_age_ms) + " ms, longer than this request's " +
+           std::to_string(budget_ms) + " ms wait budget";
+}
+
+std::string wait_exhausted_message(const std::string &model_label,
+                                   int budget_ms) {
+    return busy_prefix(model_label) + "timed out after " +
+           std::to_string(budget_ms) +
+           " ms waiting for the in-flight run to finish";
+}
+
+} // namespace
+
+void InferenceLane::occupy(int budget_ms) {
+    std::unique_lock<std::mutex> lock(state_mutex_);
+
+    // Checked once, on arrival: if the run already in the lane has outlived what
+    // this caller brought, no amount of waiting can help it, and queueing here
+    // is exactly how a wedged run swallows every handler thread.
+    const std::int64_t arrived_ms = monotonic_millis();
+    if (run_exceeds_budget(occupied_, run_started_ms_, arrived_ms, budget_ms)) {
+        throw LaneUnavailable(overrun_message(
+            model_label_, arrived_ms - run_started_ms_, budget_ms));
+    }
+
+    if (budget_ms > 0) {
+        if (!vacated_.wait_for(lock, std::chrono::milliseconds(budget_ms),
+                               [this] { return !occupied_; })) {
+            throw LaneUnavailable(
+                wait_exhausted_message(model_label_, budget_ms));
+        }
+    } else {
+        vacated_.wait(lock, [this] { return !occupied_; });
+    }
+
+    // Only now, holding both the mutex and the lane. Stamping any earlier would
+    // restart the age of the run for every caller behind this one and make a
+    // genuinely wedged holder look freshly started forever.
+    occupied_ = true;
+    run_started_ms_ = monotonic_millis();
+}
+
+void InferenceLane::vacate() {
+    {
+        std::lock_guard<std::mutex> lock(state_mutex_);
+        occupied_ = false;
+    }
+    // notify_all, not notify_one: a waiter that times out concurrently with a
+    // notification can consume it, and losing the only wakeup would park the
+    // remaining waiters for the rest of the run's lifetime. Waking all of them
+    // still admits exactly one, since the rest re-test occupancy under the mutex
+    // and go back to waiting, and ordering among waiters is not a requirement.
+    vacated_.notify_all();
+}
+
+LaneEntry::LaneEntry(InferenceLane &lane, int budget_ms) : lane_(lane) {
+    // If this throws, the object never existed, so ~LaneEntry does not run and
+    // cannot hand back a lane this caller never held.
+    lane_.occupy(budget_ms);
+}
+
+LaneEntry::~LaneEntry() { lane_.vacate(); }
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/inference_lane.h

@@ -0,0 +1,138 @@
+#pragma once
+
+// Serializes inference against the single audio.cpp model a backend process
+// owns. Standard library only.
+//
+// Why a plain mutex is not enough: an audio.cpp session is not reentrant, so
+// concurrent gRPC handlers have to take turns. But once a GPU call stops making
+// progress there is nothing a host thread can do to take it back, and an
+// unbounded queue behind such a run would absorb the handler threads one by one
+// until nothing is left to answer with. A caller therefore needs to be able to
+// walk away, and needs to be able to tell "the lane is busy with normal work and
+// I ran out of patience" apart from "the run in the lane has already outlived
+// the patience I brought".
+
+#include <condition_variable>
+#include <cstdint>
+#include <mutex>
+#include <stdexcept>
+#include <string>
+
+namespace audiocpp_backend {
+
+// Reading off a monotonic clock, in milliseconds. Monotonic on purpose: a wall
+// clock adjustment must never make an in-flight run look younger or older than
+// it is, because that reading decides whether callers give up.
+std::int64_t monotonic_millis();
+
+// Collapses the per-model configured ceiling and the optional per-request hint
+// into the wait budget a caller actually gets. Returns 0 for "wait
+// indefinitely".
+//
+// Either input may be non-positive, which means "not specified":
+//   - an unspecified hint yields the ceiling,
+//   - an unspecified ceiling means no policy limit, so the hint stands,
+//   - unspecified on both sides is unbounded.
+// A specified hint may only tighten the ceiling. A client asking for a longer
+// wait than the model's policy allows does not get it, because that would let a
+// request weaken an operator's choice.
+int resolve_wait_budget_ms(int policy_ceiling_ms, int request_hint_ms);
+
+// True when a caller carrying budget_ms should give up on arrival rather than
+// queue up. Deliberately a pure function of the lane's observable state so the
+// decision can be tested without threads or sleeping.
+//
+// Only occupancy makes a start timestamp meaningful: a lane nobody holds is
+// never overrunning, whatever timestamp the last holder left behind. An
+// unbounded caller (non-positive budget) has no budget to exceed. And the
+// comparison is strict, so a run whose age exactly equals the budget still has
+// its last millisecond.
+bool run_exceeds_budget(bool lane_occupied, std::int64_t run_started_ms,
+                        std::int64_t now_ms, int budget_ms);
+
+// Thrown when a caller cannot take the lane, in either of the two situations
+// resolve_wait_budget_ms allows for. The message distinguishes them; callers
+// that need to report a status code can treat them alike.
+class LaneUnavailable : public std::runtime_error {
+  public:
+    explicit LaneUnavailable(const std::string &reason)
+        : std::runtime_error(reason) {}
+};
+
+class LaneEntry;
+
+// One lane per loaded model. Shared by every handler thread; not copyable.
+class InferenceLane {
+  public:
+    explicit InferenceLane(std::string model_label)
+        : model_label_(std::move(model_label)) {}
+
+    InferenceLane(const InferenceLane &) = delete;
+    InferenceLane &operator=(const InferenceLane &) = delete;
+
+    const std::string &model_label() const { return model_label_; }
+
+  private:
+    // Occupancy is only reachable through LaneEntry, so there is no way to take
+    // the lane without also having something that gives it back.
+    friend class LaneEntry;
+
+    void occupy(int budget_ms);
+    void vacate();
+
+    const std::string model_label_;
+
+    std::mutex state_mutex_;
+    std::condition_variable vacated_;
+    bool occupied_ = false;
+    // Only meaningful while occupied_ is true.
+    std::int64_t run_started_ms_ = 0;
+};
+
+// Scoped occupancy of a lane. Construct it where the inference happens and it
+// is given back on every exit from that scope, including an exception and
+// including a caller that returns from the middle of a long stream. Throws
+// LaneUnavailable if the lane could not be taken, in which case there is no
+// object and nothing to release.
+//
+// Not reentrant, and it does not detect reentrancy: a second entry constructed
+// while the calling thread already holds the same lane waits for a lane only
+// that thread can release. With a positive budget that surfaces as
+// LaneUnavailable, but in unbounded mode the thread parks with no diagnostic at
+// all. Keep entries one per call: a handler that holds one across a stream must
+// not let a helper it calls construct another.
+class LaneEntry {
+  public:
+    // budget_ms <= 0 waits indefinitely. Pass the output of
+    // resolve_wait_budget_ms.
+    LaneEntry(InferenceLane &lane, int budget_ms);
+    ~LaneEntry();
+
+    LaneEntry(const LaneEntry &) = delete;
+    LaneEntry &operator=(const LaneEntry &) = delete;
+
+    // Deliberately immovable rather than carefully movable. A moved-from entry
+    // would have to stop releasing the lane while the lane still records it as
+    // occupied, and that hazard is not worth the convenience: the lane can only
+    // be recovered by whoever took it.
+    //
+    // To hold a lane for longer than one scope, construct the entry in place
+    // instead of moving one in. Two shapes work:
+    //   std::optional<LaneEntry> held;   // member or local
+    //   held.emplace(lane, budget_ms);   // takes the lane, held.reset() gives it back
+    //   auto held = std::make_unique<LaneEntry>(lane, budget_ms);  // also returnable
+    // Both outlive the acquiring scope and still release exactly once, when they
+    // are reset or destroyed. Prefer the optional for a member whose lifetime is
+    // the handler's; use the unique_ptr when the entry has to be returned, since
+    // an optional of an immovable type is itself immovable and cannot be. A
+    // factory may instead write `return LaneEntry(lane, budget_ms);`, which C++17
+    // guarantees to elide, whereas `LaneEntry entry(...); return entry;` does not
+    // compile, because that form is a move.
+    LaneEntry(LaneEntry &&) = delete;
+    LaneEntry &operator=(LaneEntry &&) = delete;
+
+  private:
+    InferenceLane &lane_;
+};
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/inference_lane_test.cpp

@@ -0,0 +1,618 @@
+// Unit tests for inference_lane. Standard library only. The harness compiles
+// this file as a single translation unit, so the implementation is included
+// directly rather than linked.
+//
+// Two kinds of test live here:
+//
+//   * The pure ones (budget negotiation, the overrun predicate) run with no
+//     threads and no sleeping. They carry the arithmetic, so they are the tests
+//     that must be exhaustive.
+//   * The threaded ones exercise the lane itself. Every one of them is bounded:
+//     contenders use a generous wait budget instead of the unbounded mode
+//     wherever the point of the test does not require unbounded, and a watchdog
+//     in main() puts a ceiling on the whole file. A broken implementation must
+//     go red, not hang, because a hung job costs CI far more than a red one.
+//
+// Wall-clock margins are called out individually. The rule applied throughout:
+// a margin is only allowed if a slow or loaded machine pushes the measurement
+// deeper into the passing region.
+
+#include "inference_lane.cpp"
+
+#include <atomic>
+#include <chrono>
+#include <condition_variable>
+#include <cstdio>
+#include <cstdlib>
+#include <mutex>
+#include <string>
+#include <thread>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using audiocpp_backend::InferenceLane;
+using audiocpp_backend::LaneEntry;
+using audiocpp_backend::LaneUnavailable;
+using audiocpp_backend::resolve_wait_budget_ms;
+using audiocpp_backend::run_exceeds_budget;
+
+// A one-shot, level-triggered signal with a bounded wait. Preferred over sleeps
+// for "the other thread got there" so the tests do not encode a guess about
+// scheduling.
+class Signal {
+  public:
+    void raise() {
+        {
+            std::lock_guard<std::mutex> lock(mutex_);
+            raised_ = true;
+        }
+        cv_.notify_all();
+    }
+
+    bool await(int timeout_ms) {
+        std::unique_lock<std::mutex> lock(mutex_);
+        return cv_.wait_for(lock, std::chrono::milliseconds(timeout_ms),
+                            [this] { return raised_; });
+    }
+
+  private:
+    std::mutex mutex_;
+    std::condition_variable cv_;
+    bool raised_ = false;
+};
+
+static std::int64_t elapsed_ms_since(
+    const std::chrono::steady_clock::time_point &start) {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+               std::chrono::steady_clock::now() - start)
+        .count();
+}
+
+static void nap(int ms) {
+    std::this_thread::sleep_for(std::chrono::milliseconds(ms));
+}
+
+static bool mentions(const std::string &haystack, const std::string &needle) {
+    return haystack.find(needle) != std::string::npos;
+}
+
+// Wording that separates the two failure modes. Kept here so a message reword
+// that erases the distinction breaks these tests loudly.
+static const char *const kTimedOutPhrase = "timed out after";
+static const char *const kStillRunningPhrase = "has been running for";
+
+// Attempts an entry and reports what happened, so the threaded tests can assert
+// on the message rather than only on the exception type.
+struct EntryOutcome {
+    bool acquired = false;
+    std::string message;
+    std::int64_t took_ms = 0;
+};
+
+static EntryOutcome try_entry(InferenceLane &lane, int budget_ms) {
+    EntryOutcome out;
+    const auto start = std::chrono::steady_clock::now();
+    try {
+        LaneEntry entry(lane, budget_ms);
+        out.acquired = true;
+    } catch (const LaneUnavailable &refused) {
+        out.message = refused.what();
+    }
+    out.took_ms = elapsed_ms_since(start);
+    return out;
+}
+
+// ---------------------------------------------------------------------------
+// B9: budget negotiation. Pure, no threads.
+// ---------------------------------------------------------------------------
+static void test_budget_negotiation() {
+    check(resolve_wait_budget_ms(0, 0) == 0,
+          "B9 no ceiling and no request hint is unbounded");
+    check(resolve_wait_budget_ms(-1, -1) == 0,
+          "B9 negative ceiling and negative hint is unbounded");
+
+    check(resolve_wait_budget_ms(5000, 0) == 5000,
+          "B9 absent hint yields the ceiling");
+    check(resolve_wait_budget_ms(5000, -250) == 5000,
+          "B9 negative hint yields the ceiling");
+
+    check(resolve_wait_budget_ms(5000, 1200) == 1200,
+          "B9 a shorter request hint is granted");
+    check(resolve_wait_budget_ms(5000, 1) == 1,
+          "B9 a much shorter request hint is granted");
+
+    check(resolve_wait_budget_ms(5000, 9000) == 5000,
+          "B9 a longer request hint cannot weaken the ceiling");
+    check(resolve_wait_budget_ms(5000, 5001) == 5000,
+          "B9 a hint one ms over the ceiling is clamped");
+    check(resolve_wait_budget_ms(5000, 5000) == 5000,
+          "B9 a hint equal to the ceiling is the ceiling");
+
+    check(resolve_wait_budget_ms(0, 1200) == 1200,
+          "B9 without a policy limit the request hint applies");
+    check(resolve_wait_budget_ms(-5, 1200) == 1200,
+          "B9 a negative ceiling is no policy limit");
+
+    check(resolve_wait_budget_ms(1, 0) == 1,
+          "B9 a one ms ceiling survives negotiation");
+}
+
+// ---------------------------------------------------------------------------
+// B5 and B6: the overrun predicate. Pure, no threads.
+// ---------------------------------------------------------------------------
+static void test_overrun_predicate() {
+    // B5: strictly longer.
+    check(run_exceeds_budget(true, 1000, 1100, 100) == false,
+          "B5 elapsed exactly equal to the budget is not an overrun");
+    check(run_exceeds_budget(true, 1000, 1101, 100) == true,
+          "B5 one ms past the budget is an overrun");
+    check(run_exceeds_budget(true, 1000, 1099, 100) == false,
+          "B5 one ms short of the budget is not an overrun");
+    check(run_exceeds_budget(true, 0, 1, 1) == false,
+          "B5 a one ms budget at one ms elapsed is not an overrun");
+    check(run_exceeds_budget(true, 0, 2, 1) == true,
+          "B5 a one ms budget at two ms elapsed is an overrun");
+
+    // B6: an unoccupied lane is never stuck, whatever the leftover timestamp
+    // says. This is the pure half of B6; the wiring half is threaded below.
+    check(run_exceeds_budget(false, 0, 10000000, 1) == false,
+          "B6 an idle lane with an ancient start timestamp is not an overrun");
+    check(run_exceeds_budget(false, 5, 5, 5) == false,
+          "B6 an idle lane is not an overrun at any elapsed value");
+
+    // An unbounded caller has no budget to exceed, so it never fails fast.
+    check(run_exceeds_budget(true, 0, 10000000, 0) == false,
+          "B2 an unbounded caller never sees an overrun");
+    check(run_exceeds_budget(true, 0, 10000000, -1) == false,
+          "B2 a negative budget never sees an overrun");
+}
+
+// ---------------------------------------------------------------------------
+// B1: mutual exclusion under real contention, and a release admits a waiter.
+// ---------------------------------------------------------------------------
+static void test_mutual_exclusion() {
+    InferenceLane lane("exclusion-model");
+
+    constexpr int kContenders = 4; // "at least three simultaneous contenders"
+    constexpr int kHoldMs = 15;
+    // Generous on purpose: the point of this test is exclusion, not timeouts.
+    // A larger budget only makes a healthy run more likely to pass, while still
+    // bounding a broken one at roughly five seconds instead of forever.
+    constexpr int kBudgetMs = 5000;
+
+    std::atomic<int> in_flight{0};
+    std::atomic<int> peak_in_flight{0};
+    std::atomic<int> completed{0};
+    std::atomic<int> refused{0};
+
+    Signal go;
+    std::vector<std::thread> contenders;
+    for (int i = 0; i < kContenders; i++) {
+        contenders.emplace_back([&] {
+            go.await(5000);
+            try {
+                LaneEntry entry(lane, kBudgetMs);
+                const int now_inside = in_flight.fetch_add(1) + 1;
+                int seen = peak_in_flight.load();
+                while (now_inside > seen &&
+                       !peak_in_flight.compare_exchange_weak(seen, now_inside)) {
+                    // retry with the refreshed value
+                }
+                nap(kHoldMs);
+                in_flight.fetch_sub(1);
+                completed.fetch_add(1);
+            } catch (const LaneUnavailable &) {
+                refused.fetch_add(1);
+            }
+        });
+    }
+    go.raise();
+    for (auto &t : contenders) {
+        t.join();
+    }
+
+    check(refused.load() == 0, "B1 no contender was refused within its budget");
+    check(completed.load() == kContenders,
+          "B1 every contender eventually got the lane");
+    check(peak_in_flight.load() == 1,
+          "B1 never more than one holder inside the lane at once");
+}
+
+// ---------------------------------------------------------------------------
+// B2: unbounded mode waits out a run longer than any bound would allow.
+// ---------------------------------------------------------------------------
+static void test_unbounded_waits_out_the_holder() {
+    InferenceLane lane("patient-model");
+    constexpr int kHoldMs = 300;
+
+    Signal held;
+    Signal patient_done;
+    std::int64_t patient_wait_ms = -1;
+    bool patient_acquired = false;
+
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        nap(kHoldMs);
+    });
+    check(held.await(5000), "B2 holder took the lane");
+
+    std::thread patient([&] {
+        const auto start = std::chrono::steady_clock::now();
+        try {
+            LaneEntry entry(lane, 0);
+            patient_acquired = true;
+        } catch (const LaneUnavailable &) {
+            patient_acquired = false;
+        }
+        patient_wait_ms = elapsed_ms_since(start);
+        patient_done.raise();
+    });
+
+    // Bound on the unbounded mode: a lane that never wakes its waiters goes red
+    // here instead of hanging in join(). The named failure is printed before the
+    // join, so even a hard hang leaves a diagnosis behind for the watchdog.
+    check(patient_done.await(10000), "B2 unbounded caller returned at all");
+    patient.join();
+    holder.join();
+
+    check(patient_acquired, "B2 unbounded caller acquired instead of failing");
+    // Margin: the holder holds for 300 ms, so the waiter must block for about
+    // that long. Asserting only half of it means a loaded machine, which makes
+    // the wait longer, drifts further into passing.
+    check(patient_wait_ms >= kHoldMs / 2,
+          "B2 unbounded caller actually waited for the in-flight run");
+}
+
+// ---------------------------------------------------------------------------
+// B3: a bounded caller that cannot get in gives up with the timeout wording.
+// ---------------------------------------------------------------------------
+static void test_bounded_wait_times_out() {
+    InferenceLane lane("impatient-model");
+    constexpr int kBudgetMs = 120;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B3 holder took the lane");
+
+    // The waiter arrives immediately, so the holder's elapsed time is far below
+    // the budget and the fail-fast path must not trigger here.
+    //
+    // Load-sensitive margin, and the tightest one in this file: what makes this
+    // the timeout path rather than the fail-fast path is the holder's age
+    // staying under 120 ms at the arrival check. All that sits between the
+    // holder's stamp and this call is one signal handover, microseconds against
+    // a 120 ms allowance, but unlike the other margins here load pushes this one
+    // toward failing rather than away from it. If it ever does flip, the symptom
+    // is the wording assertions below going red, not a hang, and the fix is a
+    // larger budget rather than a weaker assertion.
+    const EntryOutcome outcome = try_entry(lane, kBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!outcome.acquired, "B3 bounded caller did not acquire a held lane");
+    check(mentions(outcome.message, kTimedOutPhrase),
+          "B3 failure names the exhausted wait, not an overrunning run");
+    check(!mentions(outcome.message, kStillRunningPhrase),
+          "B3 failure is not worded as an overrun");
+    check(mentions(outcome.message, "impatient-model"),
+          "B3 failure names the model");
+    check(mentions(outcome.message, "120"),
+          "B3 failure reports the budget it waited out");
+    // Margin: wait_for cannot return before its deadline, so the true value is
+    // at least 120 ms and load only raises it. Asserting 100 leaves room for
+    // clock granularity while still catching an implementation that returns
+    // early without waiting.
+    check(outcome.took_ms >= 100, "B3 bounded caller waited out its budget");
+}
+
+// ---------------------------------------------------------------------------
+// B4: a caller whose budget is already exceeded fails at once.
+// ---------------------------------------------------------------------------
+static void test_fail_fast_against_a_long_run() {
+    InferenceLane lane("wedged-model");
+    constexpr int kBudgetMs = 200;
+    constexpr int kRunAgeMs = 400;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B4 holder took the lane");
+
+    // Margin: the arriving caller needs the holder's elapsed time to exceed
+    // 200 ms. Sleeping 400 ms means a loaded machine oversleeps and pushes the
+    // elapsed time further past the budget, never below it.
+    nap(kRunAgeMs);
+    const EntryOutcome outcome = try_entry(lane, kBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!outcome.acquired, "B4 caller did not acquire a long-running lane");
+    check(mentions(outcome.message, kStillRunningPhrase),
+          "B4 failure states the measured age of the in-flight run");
+    check(!mentions(outcome.message, kTimedOutPhrase),
+          "B4 failure is not worded as an exhausted wait");
+    check(mentions(outcome.message, "wedged-model"),
+          "B4 failure names the model");
+    // Margin: a fail-fast return takes microseconds. 150 ms of headroom under a
+    // 200 ms budget separates "returned at once" from "waited out the budget"
+    // by a wide enough gap that scheduler noise cannot close it. The message
+    // assertions above are the load-independent proof; this one pins the timing.
+    check(outcome.took_ms < 150, "B4 caller failed without waiting out its budget");
+}
+
+// ---------------------------------------------------------------------------
+// B6 wiring: an idle lane never looks stuck, however old the last run is.
+// ---------------------------------------------------------------------------
+static void test_idle_lane_is_never_stuck() {
+    InferenceLane lane("idle-model");
+
+    {
+        LaneEntry entry(lane, 0);
+    }
+    // Ages the leftover start timestamp well past the tiny budget used below.
+    // A longer sleep only makes a stale-timestamp bug more visible, so load
+    // helps this test rather than hurting it.
+    nap(80);
+
+    const EntryOutcome first = try_entry(lane, 20);
+    check(first.acquired, "B6 tiny budget still acquires an idle lane");
+
+    nap(80);
+    const EntryOutcome second = try_entry(lane, 1);
+    check(second.acquired, "B6 a one ms budget still acquires an idle lane");
+}
+
+// ---------------------------------------------------------------------------
+// B7: every ownership exit clears the busy state, including an exception
+// thrown from inside the guarded region.
+// ---------------------------------------------------------------------------
+static void test_release_on_exception() {
+    InferenceLane lane("throwing-model");
+
+    struct GuardedRegionFailure {};
+    bool propagated = false;
+    try {
+        LaneEntry entry(lane, 0);
+        throw GuardedRegionFailure{};
+    } catch (const GuardedRegionFailure &) {
+        propagated = true;
+    }
+    check(propagated, "B7 the guarded region's own exception propagated");
+
+    // If the throw had leaked the busy state, this tiny budget would fail.
+    const EntryOutcome after_throw = try_entry(lane, 20);
+    check(after_throw.acquired, "B7 lane is free after an exception unwound it");
+
+    // Same check for a holder that unwinds on another thread, which is the shape
+    // a gRPC handler failing mid-inference actually has.
+    Signal thrown;
+    std::thread unlucky([&] {
+        try {
+            LaneEntry entry(lane, 0);
+            throw GuardedRegionFailure{};
+        } catch (const GuardedRegionFailure &) {
+            thrown.raise();
+        }
+    });
+    check(thrown.await(5000), "B7 worker thread unwound its guarded region");
+    unlucky.join();
+
+    const EntryOutcome after_worker = try_entry(lane, 20);
+    check(after_worker.acquired,
+          "B7 lane is free after a worker thread unwound it");
+}
+
+// ---------------------------------------------------------------------------
+// B8: a waiter must not publish itself as the holder. If it did, its arrival
+// would restart the elapsed-time measurement and hide the real holder.
+//
+// Timeline, with the holder taking the lane at t0 and never letting go:
+//
+//   t0        holder acquires, elapsed measurement starts here and only here
+//   t0+100    waiter arrives with a 400 ms budget, blocks, and times out
+//   t0+500    late caller arrives with a 450 ms budget
+//
+// A correct lane measures 500 ms of holding at the late arrival, which is more
+// than 450, so the late caller fails fast. An implementation that let the
+// waiter stamp itself as holder measures only the 400 ms since the waiter
+// arrived, which is under 450, so the late caller would queue behind a stuck
+// run instead. The two paths are told apart by their wording.
+// ---------------------------------------------------------------------------
+static void test_waiter_does_not_become_the_holder() {
+    InferenceLane lane("stamp-model");
+    constexpr int kWaiterArrivesAfterMs = 100;
+    constexpr int kWaiterBudgetMs = 400;
+    constexpr int kLateBudgetMs = 450;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B8 holder took the lane");
+
+    // Margin: the waiter must not fail fast on arrival, which needs the
+    // holder's elapsed time to stay under 400 ms. Arriving at 100 ms leaves
+    // 300 ms of slack, so oversleeping under load does not flip the path.
+    nap(kWaiterArrivesAfterMs);
+
+    EntryOutcome waiter_outcome;
+    std::thread waiter([&] { waiter_outcome = try_entry(lane, kWaiterBudgetMs); });
+    waiter.join();
+    check(!waiter_outcome.acquired, "B8 mid-queue waiter did not acquire");
+    check(mentions(waiter_outcome.message, kTimedOutPhrase),
+          "B8 mid-queue waiter waited out its budget and timed out");
+
+    // Margin: the holder has now been in the lane for at least 500 ms against a
+    // 450 ms budget. Load lengthens both sleeps, so the measured age only grows
+    // and the fail-fast path only becomes more certain.
+    const EntryOutcome late = try_entry(lane, kLateBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!late.acquired, "B8 late caller did not acquire");
+    check(mentions(late.message, kStillRunningPhrase),
+          "B8 elapsed time is still measured from the real holder's acquisition");
+    check(late.took_ms < 200,
+          "B8 late caller failed fast rather than queueing behind the holder");
+}
+
+// ---------------------------------------------------------------------------
+// B8, other direction: the age of a run is measured from the moment its holder
+// acquired, not from the moment that holder arrived. A caller that queued for a
+// while and then got in is starting a fresh run, and its time in the queue must
+// not be billed to it: if it were, every handover would hand the new holder a
+// head start towards looking overrun, and short-budget callers would be turned
+// away from a run that has barely begun.
+//
+//   t0        first holder acquires and holds for 300 ms
+//   t0        second caller arrives and queues
+//   t0+300    second caller acquires, so its own run age is ~0 here
+//   t0+300    a third caller arrives with a 300 ms budget
+//
+// Correct: the third caller sees a run that just started, so it queues and then
+// times out. Billing the queue time to the second caller would show a 300 ms old
+// run instead, and the third caller would be turned away as an overrun.
+// ---------------------------------------------------------------------------
+static void test_run_age_starts_at_acquisition() {
+    InferenceLane lane("handover-model");
+    constexpr int kFirstHoldMs = 300;
+    constexpr int kThirdBudgetMs = 200;
+
+    Signal first_held;
+    Signal handed_over;
+    Signal release_second;
+
+    std::thread first([&] {
+        LaneEntry entry(lane, 0);
+        first_held.raise();
+        nap(kFirstHoldMs);
+    });
+    // Ordering matters only for the diagnosis, not for the assertion: waiting
+    // for the first holder guarantees the second caller really does queue, which
+    // is what gives it queue time to be wrongly billed for.
+    check(first_held.await(5000), "B8 first holder took the lane");
+
+    std::thread second([&] {
+        LaneEntry entry(lane, 0);
+        handed_over.raise();
+        release_second.await(10000);
+    });
+    check(handed_over.await(10000), "B8 queued caller was handed the lane");
+
+    // Margin: the new holder's run is a few ms old against a 200 ms budget, so
+    // this caller must queue. Load can only add a few ms of handover latency,
+    // well inside that slack, while it lengthens the queue time that the buggy
+    // version would bill, making the bug more visible rather than less.
+    const EntryOutcome third = try_entry(lane, kThirdBudgetMs);
+    release_second.raise();
+    second.join();
+    first.join();
+
+    check(!third.acquired, "B8 lane was still held by the queued caller");
+    check(mentions(third.message, kTimedOutPhrase),
+          "B8 a fresh holder's run age excludes the time it spent queueing");
+}
+
+// ---------------------------------------------------------------------------
+// B10: both failure modes carry a usable message, and the fail-fast one reports
+// a measurement rather than diagnosing a cause.
+// ---------------------------------------------------------------------------
+static void test_failure_messages_are_diagnosable() {
+    InferenceLane lane("diagnosable-model");
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B10 holder took the lane");
+
+    // The two budgets have to straddle the run's age, or both callers take the
+    // same path and the comparisons below are between two fail-fast messages
+    // that differ only in the budget they print.
+    //
+    // Margin: 30 ms is far under the ~120 ms age, and load only ages the run
+    // further, so `fast` fails fast. 400 ms is far over it, with the same
+    // slack and the same safe direction as the B3 test, so `slow` queues and
+    // then times out.
+    nap(120);
+    const EntryOutcome fast = try_entry(lane, 30);
+    const EntryOutcome slow = try_entry(lane, 400);
+    release.raise();
+    holder.join();
+
+    check(mentions(fast.message, kStillRunningPhrase) &&
+              mentions(slow.message, kTimedOutPhrase),
+          "B10 one caller took the fail-fast path and the other timed out");
+    check(fast.message != slow.message,
+          "B10 the two failure modes do not share one message");
+    check(!fast.message.empty() && !slow.message.empty(),
+          "B10 both failures carry text");
+    check(mentions(fast.message, "diagnosable-model") &&
+              mentions(slow.message, "diagnosable-model"),
+          "B10 both failures name the model");
+    // The fail-fast wording must not accuse the run of being stuck: a short
+    // budget meeting a legitimately long run reaches this path too.
+    for (const char *verdict : {"stuck", "wedged", "hung", "deadlock"}) {
+        check(!mentions(fast.message, verdict),
+              std::string("B10 fail-fast message avoids diagnosing '") +
+                  verdict + "'");
+    }
+}
+
+int main() {
+    // Last resort only. Every threaded test above is individually bounded, so
+    // this should never fire; it exists so that an implementation which parks a
+    // thread forever still ends the job instead of occupying a CI runner.
+    std::thread watchdog([] {
+        std::this_thread::sleep_for(std::chrono::seconds(30));
+        fprintf(stderr, "FAIL: watchdog fired, an inference_lane test hung\n");
+        fflush(stderr);
+        std::_Exit(1);
+    });
+    watchdog.detach();
+
+    test_budget_negotiation();
+    test_overrun_predicate();
+    test_mutual_exclusion();
+    test_unbounded_waits_out_the_holder();
+    test_bounded_wait_times_out();
+    test_fail_fast_against_a_long_run();
+    test_idle_lane_is_never_stuck();
+    test_release_on_exception();
+    test_waiter_does_not_become_the_holder();
+    test_run_age_starts_at_acquisition();
+    test_failure_messages_are_diagnosable();
+
+    if (failures == 0) {
+        fprintf(stderr, "\nAll inference_lane tests passed.\n");
+        return 0;
+    }
+    fprintf(stderr, "\n%d inference_lane test(s) failed.\n", failures);
+    return 1;
+}

backend/cpp/audio-cpp/live_watchdog.cpp

@@ -0,0 +1,75 @@
+#include "live_watchdog.h"
+
+#include <utility>
+
+namespace audiocpp_backend {
+
+IdleWatchdog::IdleWatchdog(std::chrono::milliseconds window,
+                           std::function<void()> on_idle)
+    : window_(window), on_idle_(std::move(on_idle)),
+      last_(std::chrono::steady_clock::now()) {
+    if (window_.count() <= 0) {
+        // Disabled: no thread at all, rather than a thread with an infinite
+        // deadline. A thread that exists is a thread that has to be joined on
+        // every exit path, and there is nothing for this one to do.
+        return;
+    }
+    thread_ = std::thread([this] { run(); });
+}
+
+IdleWatchdog::~IdleWatchdog() { disarm(); }
+
+void IdleWatchdog::touch() {
+    std::lock_guard<std::mutex> lock(mu_);
+    last_ = std::chrono::steady_clock::now();
+    // Deliberately does NOT notify. The waiter recomputes its deadline from
+    // last_ every time it wakes, so a touch that lands mid-window is picked up
+    // when the old deadline expires, and a touch is the hot path: it runs once
+    // per frame on the wire.
+}
+
+void IdleWatchdog::disarm() {
+    {
+        std::lock_guard<std::mutex> lock(mu_);
+        stop_ = true;
+    }
+    cv_.notify_all();
+    if (thread_.joinable()) {
+        thread_.join();
+    }
+}
+
+bool IdleWatchdog::fired() const {
+    std::lock_guard<std::mutex> lock(mu_);
+    return fired_;
+}
+
+void IdleWatchdog::run() {
+    std::unique_lock<std::mutex> lock(mu_);
+    while (!stop_) {
+        const auto deadline = last_ + window_;
+        if (cv_.wait_until(lock, deadline, [this] { return stop_; })) {
+            return; // disarmed
+        }
+        // The deadline passed, but last_ may have moved while this thread was
+        // waiting, and a condition variable may also wake spuriously. Re-read
+        // it: without this check a touch that landed mid-window would still be
+        // followed by a cancellation, i.e. a live client cut off mid-sentence.
+        if (std::chrono::steady_clock::now() < last_ + window_) {
+            continue;
+        }
+        fired_ = true;
+        auto callback = on_idle_;
+        lock.unlock();
+        if (callback) {
+            callback();
+        }
+        return; // one shot
+    }
+}
+
+bool live_frame_carries_audio(bool has_audio, bool pcm_empty) {
+    return has_audio && !pcm_empty;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/live_watchdog.h

@@ -0,0 +1,99 @@
+#pragma once
+
+// A one-shot idle timer for a bidirectional stream. Standard library only, so
+// it is tested without an audio.cpp checkout or a gRPC server.
+//
+// WHY IT EXISTS. AudioTranscriptionLive holds the model's inference lane for the
+// whole stream, because the streaming session is stateful and a concurrent run
+// would interleave two callers' audio. Every other RPC in this backend holds the
+// lane across COMPUTE, or across a write to a slow reader, and both of those
+// terminate on their own. A live stream instead blocks in a client-driven read,
+// and a peer that goes silent WITHOUT closing the stream never terminates
+// anything: the lane stays taken and every other RPC against that model queues
+// behind a client that has stopped speaking. A websocket death does cancel the
+// RPC and free it, but "the peer's TCP connection eventually dies" is not a
+// bound anyone can state, so this supplies one.
+//
+// HOW IT ENDS THE STREAM, and the part that is not obvious: gRPC's synchronous
+// ServerReaderWriter::Read has no timeout and cannot be given one. The only way
+// to unblock it from another thread is ServerContext::TryCancel, which is what
+// the callback is for. That means the client sees CANCELLED rather than whatever
+// status the handler goes on to return: the returned status is for the server's
+// own record. Releasing the lane is the point.
+//
+// ONE SHOT on purpose. Once the callback has run the stream is being torn down,
+// so there is nothing left to watch, and a repeating timer would call TryCancel
+// on a context the handler may already have returned from.
+
+#include <chrono>
+#include <condition_variable>
+#include <functional>
+#include <mutex>
+#include <thread>
+
+namespace audiocpp_backend {
+
+class IdleWatchdog {
+public:
+    // A window that is not positive DISABLES the watchdog entirely: no thread is
+    // started and fired() never becomes true. That is the operator's escape
+    // hatch for a client that legitimately holds a stream open through long
+    // pauses, and it is why the option carrying it documents 0 as "no limit"
+    // rather than as "expire immediately".
+    //
+    // `on_idle` runs on the watchdog's own thread with no lock held. It must be
+    // safe to call while the watched thread is blocked in a read, which is the
+    // only reason this class exists; ServerContext::TryCancel is documented as
+    // exactly that.
+    IdleWatchdog(std::chrono::milliseconds window, std::function<void()> on_idle);
+
+    // Joins the thread, so the callback can safely capture anything that
+    // outlives this object's scope and nothing else has to be reasoned about.
+    ~IdleWatchdog();
+
+    IdleWatchdog(const IdleWatchdog &) = delete;
+    IdleWatchdog &operator=(const IdleWatchdog &) = delete;
+
+    // Restarts the window. Call it whenever the peer proves it is still there.
+    void touch();
+
+    // Stops watching and joins. Idempotent, and REQUIRED before any long
+    // non-read work the window must not cover: the caller's own decode is not
+    // the peer going quiet, and cancelling in the middle of it would throw away
+    // a transcript the client is waiting for.
+    void disarm();
+
+    // True once the window elapsed and the callback ran. Stays true after
+    // disarm, so the caller can tell "the peer closed" from "we cancelled it".
+    bool fired() const;
+
+private:
+    void run();
+
+    const std::chrono::milliseconds window_;
+    std::function<void()> on_idle_;
+
+    mutable std::mutex mu_;
+    std::condition_variable cv_;
+    std::chrono::steady_clock::time_point last_;
+    bool stop_ = false;
+    bool fired_ = false;
+    std::thread thread_;
+};
+
+// Whether one message read off a live stream is a frame the decoder can
+// actually consume, which is the ONLY thing that counts as the peer proving it
+// is still there.
+//
+// Split out of the read loop so the distinction is testable, and because
+// getting it wrong is silent. The loop used to touch the watchdog on ANY
+// message, before it filtered on has_audio and on an empty pcm field, so a peer
+// writing unset-oneof or zero-length frames faster than the window held the
+// lane forever: no audio was ever fed, no work was ever done, and the timer
+// that exists to break exactly that grip was reset by the frames doing it.
+// There is one lane per model and one model per process, so that is a single
+// client denying the whole backend. The thrown message already said "no audio
+// frame arrived"; this is the code agreeing with it.
+bool live_frame_carries_audio(bool has_audio, bool pcm_empty);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/live_watchdog_test.cpp

@@ -0,0 +1,225 @@
+// Unit tests for the live stream idle watchdog. Standard library only; the
+// harness compiles this as a single translation unit, so the implementation is
+// included directly.
+//
+// These are TIMING tests, which is unavoidable: what is under test is a
+// deadline. Every window here is short and every assertion waits several
+// multiples of it, so a loaded machine slows the test down rather than
+// flipping its answer. The one thing never asserted is how SOON something
+// happens, only that it eventually does or never does.
+
+#include "live_watchdog.cpp"
+
+#include <atomic>
+#include <cstdio>
+#include <string>
+#include <thread>
+
+using namespace std::chrono_literals;
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+// A peer that goes quiet without closing. This is the whole point: the lane it
+// holds has to come back.
+static void test_it_fires_when_nothing_touches_it() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(100ms, [&calls] { ++calls; });
+    std::this_thread::sleep_for(600ms);
+    check(watchdog.fired(), "a window that elapses untouched fires");
+    check(calls.load() == 1, "the callback runs exactly once, not once per window");
+}
+
+// A peer that is still speaking must never be cut off. Touches land at a third
+// of the window, for six windows' worth of wall clock.
+static void test_touching_defers_it_indefinitely() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(300ms, [&calls] { ++calls; });
+    for (int i = 0; i < 20; ++i) {
+        std::this_thread::sleep_for(100ms);
+        watchdog.touch();
+    }
+    check(!watchdog.fired(),
+          "a stream touched inside every window is never cancelled");
+    check(calls.load() == 0, "no callback runs while the peer is still there");
+}
+
+// Disarm is what the handler calls when the read side closes, before a decode
+// that can take longer than the window. Firing after that would throw away the
+// transcript the client is waiting for.
+static void test_disarm_stops_it_before_the_window() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(200ms, [&calls] { ++calls; });
+    std::this_thread::sleep_for(20ms);
+    watchdog.disarm();
+    std::this_thread::sleep_for(600ms);
+    check(!watchdog.fired(), "a disarmed watchdog does not fire");
+    check(calls.load() == 0, "a disarmed watchdog runs no callback");
+}
+
+static void test_disarm_is_idempotent() {
+    audiocpp_backend::IdleWatchdog watchdog(50ms, [] {});
+    watchdog.disarm();
+    watchdog.disarm();
+    watchdog.disarm();
+    check(true, "disarming three times joins once and does not abort");
+}
+
+// The operator's escape hatch, for a client that legitimately holds a stream
+// open through long pauses. Not "expire immediately", which is what a naive
+// reading of a zero timeout would give.
+static void test_a_non_positive_window_disables_it() {
+    std::atomic<int> calls{0};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(0ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(300ms);
+        check(!watchdog.fired(), "a zero window never fires");
+    }
+    {
+        audiocpp_backend::IdleWatchdog watchdog(-5ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(300ms);
+        check(!watchdog.fired(), "a negative window never fires");
+    }
+    check(calls.load() == 0, "a disabled watchdog runs no callback");
+}
+
+// fired() has to survive the disarm, because the handler reads it AFTER the
+// read loop ends to tell "the peer closed" from "we cancelled the peer", and
+// those two get different statuses.
+static void test_fired_survives_a_later_disarm() {
+    audiocpp_backend::IdleWatchdog watchdog(80ms, [] {});
+    std::this_thread::sleep_for(500ms);
+    watchdog.disarm();
+    check(watchdog.fired(), "a watchdog that fired still says so after disarm");
+}
+
+// The destructor joins, so a callback capturing the handler's frame cannot run
+// after that frame is gone. Without the join this is a use after free that only
+// shows up under load.
+//
+// The window is LONGER than the scope on purpose. An earlier version of this
+// test slept past the window inside the scope, so the callback had already run
+// by the time the object was destroyed and a destructor that DETACHED the thread
+// instead of joining it passed unnoticed. Mutation testing is what found that;
+// the shape below kills it, because a detached thread wakes after the object is
+// gone and calls a callback that must never run.
+static void test_the_destructor_joins() {
+    std::atomic<int> calls{0};
+    std::atomic<bool> alive{true};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(200ms, [&calls, &alive] {
+            check(alive.load(),
+                  "the callback never runs after the watched scope ended");
+            ++calls;
+        });
+        std::this_thread::sleep_for(20ms);
+    }
+    alive.store(false);
+    std::this_thread::sleep_for(600ms);
+    check(calls.load() == 0,
+          "destruction stops the timer rather than leaving it running against a "
+          "dead frame");
+}
+
+// The other half of that pair: a callback that DOES fire inside the scope runs
+// exactly once, so the test above is not passing merely because nothing ever
+// fires.
+static void test_a_firing_watchdog_still_joins_cleanly() {
+    std::atomic<int> calls{0};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(50ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(400ms);
+    }
+    check(calls.load() == 1, "the callback ran once, inside the scope");
+}
+
+// The predicate the live read loop filters on.
+static void test_only_a_frame_with_audio_counts() {
+    using audiocpp_backend::live_frame_carries_audio;
+    check(live_frame_carries_audio(true, false),
+          "a frame with a non-empty pcm field carries audio");
+    check(!live_frame_carries_audio(true, true),
+          "an empty pcm field does not");
+    check(!live_frame_carries_audio(false, false),
+          "an unset audio oneof does not, whatever the pcm field looks like");
+    check(!live_frame_carries_audio(false, true),
+          "and neither does an unset oneof with an empty pcm field");
+}
+
+// The defect this closes, expressed as behaviour rather than as a call order:
+// a peer writing frames the decoder cannot consume, faster than the window,
+// used to hold the model's only inference lane forever, because the read loop
+// touched the watchdog before it filtered them out. One lane per model and one
+// model per process, so that is a single client denying the whole backend,
+// which is exactly what the watchdog exists to prevent.
+static void test_empty_frames_do_not_hold_the_lane() {
+    std::atomic<int> cancels{0};
+    std::atomic<bool> stop{false};
+    audiocpp_backend::IdleWatchdog watchdog(80ms, [&cancels] { ++cancels; });
+
+    // The read loop with the real filter in it: frames arrive continuously,
+    // none of them carries audio, and only a frame that does may touch.
+    std::thread peer([&] {
+        while (!stop.load()) {
+            if (audiocpp_backend::live_frame_carries_audio(false, true)) {
+                watchdog.touch();
+            }
+            std::this_thread::sleep_for(5ms);
+        }
+    });
+
+    std::this_thread::sleep_for(600ms);
+    stop.store(true);
+    peer.join();
+    watchdog.disarm();
+
+    check(cancels.load() == 1,
+          "a flood of frames with no audio in them still releases the lane");
+
+    // The mirror image, so this cannot pass merely because the watchdog always
+    // fires: a peer that keeps sending audio is left alone, exactly as before.
+    std::atomic<int> live_cancels{0};
+    std::atomic<bool> live_stop{false};
+    audiocpp_backend::IdleWatchdog live(80ms, [&live_cancels] { ++live_cancels; });
+    std::thread speaker([&] {
+        while (!live_stop.load()) {
+            if (audiocpp_backend::live_frame_carries_audio(true, false)) {
+                live.touch();
+            }
+            std::this_thread::sleep_for(5ms);
+        }
+    });
+    std::this_thread::sleep_for(600ms);
+    live_stop.store(true);
+    speaker.join();
+    live.disarm();
+    check(live_cancels.load() == 0,
+          "a peer that keeps sending audio is never cancelled");
+}
+
+int main() {
+    test_it_fires_when_nothing_touches_it();
+    test_touching_defers_it_indefinitely();
+    test_disarm_stops_it_before_the_window();
+    test_disarm_is_idempotent();
+    test_a_non_positive_window_disables_it();
+    test_fired_survives_a_later_disarm();
+    test_the_destructor_joins();
+    test_a_firing_watchdog_still_joins_cleanly();
+    test_only_a_frame_with_audio_counts();
+    test_empty_frames_do_not_hold_the_lane();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all live_watchdog checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/loaded_model.cpp

@@ -0,0 +1,802 @@
+#include "loaded_model.h"
+
+#include "family_gate.h"
+
+#include "engine/framework/assets/tensor_source.h"
+
+#include <algorithm>
+#include <cstddef>
+#include <cstdint>
+#include <filesystem>
+#include <utility>
+
+#if defined(__APPLE__)
+#include <mach-o/dyld.h>
+#include <cstdint>
+#include <vector>
+#elif defined(__linux__)
+#include <unistd.h>
+#endif
+
+namespace audiocpp_backend {
+
+// --------------------------------------------------------------------------
+// Enum coupling
+//
+// audiocpp_backend::Task mirrors engine::runtime::VoiceTaskKind positionally so
+// capability_routing can stay stdlib-only and testable without an audio.cpp
+// checkout. Nothing about that mirroring is enforced by the type system, and a
+// drift is silent in the worst possible way: every unit still compiles, every
+// test still passes, and the backend runs a different task than the one the
+// caller asked for.
+//
+// Two mechanisms pin it, and both are needed because they catch different edits:
+//
+//   1. The assertions below pin every enumerator's value on both sides. An
+//      insertion or a reorder anywhere before the last member shifts the values
+//      after it and fails the build here.
+//   2. An enumerator APPENDED after the last one shifts nothing, so no value
+//      assertion can see it. What sees it is the switch in from_engine_task,
+//      which covers the engine enum with no `default:` label. CMakeLists.txt
+//      compiles this file with -Werror=switch so that omission is an error
+//      rather than a warning nobody reads.
+//
+// Neither mechanism catches a pure RENAME of an upstream enumerator, but that
+// does not need catching: the switch stops naming an enumerator that exists and
+// the build fails on its own.
+// --------------------------------------------------------------------------
+
+namespace {
+
+constexpr int kEngine(engine::runtime::VoiceTaskKind kind) {
+    return static_cast<int>(kind);
+}
+constexpr int kMirror(Task task) { return static_cast<int>(task); }
+
+} // namespace
+
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Vad) == 0, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Asr) == 1, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Diarization) == 2, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SourceSeparation) == 3, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::AudioGeneration) == 4, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Tts) == 5, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceCloning) == 6, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceConversion) == 7, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SpeechToSpeech) == 8, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Alignment) == 9, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceDesign) == 10, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SpeakerRecognition) == 11, "VoiceTaskKind drifted");
+// The last member. Pinning it pins the member count too, as long as the
+// enumerators stay contiguous and unassigned, which upstream's declaration is.
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Svc) == 12,
+              "engine::runtime::VoiceTaskKind gained, lost or reordered a member. "
+              "audiocpp_backend::Task mirrors it positionally: update capability_routing.h, "
+              "to_engine_task and from_engine_task together, then move this pin.");
+
+static_assert(kMirror(Task::Vad) == 0, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Asr) == 1, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Diarization) == 2, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SourceSeparation) == 3, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::AudioGeneration) == 4, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Tts) == 5, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceCloning) == 6, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceConversion) == 7, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SpeechToSpeech) == 8, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Alignment) == 9, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceDesign) == 10, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SpeakerRecognition) == 11, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Svc) == 12, "Task drifted from VoiceTaskKind");
+
+static_assert(static_cast<int>(engine::runtime::RunMode::Offline) == 0, "RunMode drifted");
+static_assert(static_cast<int>(engine::runtime::RunMode::Streaming) == 1,
+              "engine::runtime::RunMode gained, lost or reordered a member. "
+              "audiocpp_backend::Mode mirrors it positionally.");
+static_assert(static_cast<int>(Mode::Offline) == 0, "Mode drifted from RunMode");
+static_assert(static_cast<int>(Mode::Streaming) == 1, "Mode drifted from RunMode");
+
+namespace {
+
+engine::core::BackendType parse_backend_type(const std::string &value) {
+    if (value == "cuda") {
+        return engine::core::BackendType::Cuda;
+    }
+    if (value == "vulkan") {
+        return engine::core::BackendType::Vulkan;
+    }
+    if (value == "metal") {
+        return engine::core::BackendType::Metal;
+    }
+    if (value == "best") {
+        return engine::core::BackendType::BestAvailable;
+    }
+    if (value == "cpu" || value.empty()) {
+        return engine::core::BackendType::Cpu;
+    }
+    throw ConfigError("audio-cpp: unknown backend option '" + value +
+                      "'. Known backends: cpu, cuda, vulkan, metal, best");
+}
+
+std::filesystem::path executable_directory() {
+#if defined(__APPLE__)
+    std::uint32_t size = 0;
+    _NSGetExecutablePath(nullptr, &size);
+    std::vector<char> buffer(size + 1, '\0');
+    if (_NSGetExecutablePath(buffer.data(), &size) != 0) {
+        return std::filesystem::current_path();
+    }
+    return std::filesystem::path(buffer.data()).parent_path();
+#elif defined(__linux__)
+    std::error_code ec;
+    const auto self = std::filesystem::read_symlink("/proc/self/exe", ec);
+    if (ec) {
+        return std::filesystem::current_path();
+    }
+    return self.parent_path();
+#else
+    return std::filesystem::current_path();
+#endif
+}
+
+// Runs the load gate. Throws ConfigError rather than returning a decision,
+// because the only caller is a delegating constructor whose member initializer
+// list has nowhere to put a failure.
+std::string require_family(const std::string &resolved_path,
+                           const ModelOptions &options) {
+    const std::filesystem::path path(resolved_path);
+    std::error_code ec;
+    if (!std::filesystem::exists(path, ec)) {
+        throw ConfigError("audio-cpp: model path does not exist: " + resolved_path);
+    }
+
+    const bool is_gguf = path_looks_like_gguf(resolved_path);
+    // Only a GGUF is asked for embedded metadata. A directory has no single
+    // file to read it from, and probing one would make the gate's refusal
+    // depend on which file happened to be inside.
+    const std::string embedded =
+        is_gguf ? read_gguf_family(resolved_path) : std::string();
+
+    const FamilyDecision decision =
+        decide_family(is_gguf, embedded, options.family);
+    if (!decision.ok) {
+        throw ConfigError(decision.error);
+    }
+    return decision.family;
+}
+
+// Refuses a GGUF whose weights are stored in a dtype the family cannot survive.
+//
+// The POLICY lives in family_gate's weight_dtype_is_supported, which is
+// stdlib-only and therefore testable; this is only the part that needs a file
+// and an engine to read one. See the table there for why an entry exists and
+// what has to be run before deleting it.
+//
+// Only GGUF paths are inspected. A directory of safetensors carries its dtypes
+// per file and has not been tested against this failure, so it is passed
+// through rather than guessed at.
+void require_supported_weight_dtypes(const std::string &family,
+                                     const std::string &resolved_path) {
+    // Asked as "is there an entry", not as "is the description non-empty": an
+    // entry with an empty allow list describes a family that can run nothing,
+    // and reading the description would skip the check on exactly that entry
+    // while weight_dtype_is_supported refused every dtype. No such entry exists
+    // today; the two questions are different ones and only one of them is this
+    // guard's.
+    if (!family_has_weight_dtype_allow_list(family) ||
+        !path_looks_like_gguf(resolved_path)) {
+        return;
+    }
+
+    std::string offending_dtype;
+    std::string offending_tensor;
+    try {
+        const auto source =
+            engine::assets::open_tensor_source(std::filesystem::path(resolved_path));
+        if (source == nullptr) {
+            return;
+        }
+        for (const auto &tensor : source->tensors()) {
+            if (!weight_dtype_is_supported(family, tensor.dtype)) {
+                offending_dtype = tensor.dtype;
+                offending_tensor = tensor.name;
+                break;
+            }
+        }
+    } catch (const std::exception &) {
+        // Unreadable as a tensor source. Not this guard's problem to report:
+        // the registry load below produces a message naming the real fault, and
+        // refusing here would turn every unusual packaging into this error.
+        return;
+    }
+
+    if (offending_dtype.empty()) {
+        return;
+    }
+    throw ConfigError(
+        "audio-cpp: family '" + family + "' cannot run weights stored as '" +
+        offending_dtype + "' (tensor '" + offending_tensor + "' in " +
+        resolved_path +
+        "); it aborts the backend process on the first request rather than "
+        "failing the request. Use the 'orig' GGUF package, whose weights are " +
+        supported_weight_dtypes(family) + ".");
+}
+
+} // namespace
+
+engine::runtime::VoiceTaskKind to_engine_task(Task task) {
+    using K = engine::runtime::VoiceTaskKind;
+    // An explicit switch, never a cast: a cast would keep compiling through
+    // exactly the drift the assertions above exist to catch.
+    switch (task) {
+    case Task::Vad:                return K::Vad;
+    case Task::Asr:                return K::Asr;
+    case Task::Diarization:        return K::Diarization;
+    case Task::SourceSeparation:   return K::SourceSeparation;
+    case Task::AudioGeneration:    return K::AudioGeneration;
+    case Task::Tts:                return K::Tts;
+    case Task::VoiceCloning:       return K::VoiceCloning;
+    case Task::VoiceConversion:    return K::VoiceConversion;
+    case Task::SpeechToSpeech:     return K::SpeechToSpeech;
+    case Task::Alignment:          return K::Alignment;
+    case Task::VoiceDesign:        return K::VoiceDesign;
+    case Task::SpeakerRecognition: return K::SpeakerRecognition;
+    case Task::Svc:                return K::Svc;
+    }
+    // Unreachable for any valid enumerator. No `default:` label, so -Wswitch
+    // still reports a member this switch stops covering.
+    return K::Vad;
+}
+
+Task from_engine_task(engine::runtime::VoiceTaskKind kind) {
+    using K = engine::runtime::VoiceTaskKind;
+    switch (kind) {
+    case K::Vad:                return Task::Vad;
+    case K::Asr:                return Task::Asr;
+    case K::Diarization:        return Task::Diarization;
+    case K::SourceSeparation:   return Task::SourceSeparation;
+    case K::AudioGeneration:    return Task::AudioGeneration;
+    case K::Tts:                return Task::Tts;
+    case K::VoiceCloning:       return Task::VoiceCloning;
+    case K::VoiceConversion:    return Task::VoiceConversion;
+    case K::SpeechToSpeech:     return Task::SpeechToSpeech;
+    case K::Alignment:          return Task::Alignment;
+    case K::VoiceDesign:        return Task::VoiceDesign;
+    case K::SpeakerRecognition: return Task::SpeakerRecognition;
+    case K::Svc:                return Task::Svc;
+    }
+    return Task::Vad;
+}
+
+engine::runtime::RunMode to_engine_mode(Mode mode) {
+    using M = engine::runtime::RunMode;
+    switch (mode) {
+    case Mode::Offline:   return M::Offline;
+    case Mode::Streaming: return M::Streaming;
+    }
+    return M::Offline;
+}
+
+Mode from_engine_mode(engine::runtime::RunMode mode) {
+    using M = engine::runtime::RunMode;
+    switch (mode) {
+    case M::Offline:   return Mode::Offline;
+    case M::Streaming: return Mode::Streaming;
+    }
+    return Mode::Offline;
+}
+
+Capabilities to_capabilities(const std::string &family,
+                             const engine::runtime::CapabilitySet &set) {
+    Capabilities caps;
+    caps.family = family;
+    caps.tasks.reserve(set.supported_tasks.size());
+    for (const auto &supported : set.supported_tasks) {
+        TaskCapability capability;
+        capability.task = from_engine_task(supported.task);
+        capability.modes.reserve(supported.modes.size());
+        for (const auto mode : supported.modes) {
+            capability.modes.push_back(from_engine_mode(mode));
+        }
+        caps.tasks.push_back(std::move(capability));
+    }
+    return caps;
+}
+
+std::string read_gguf_family(const std::string &path) {
+    try {
+        const auto spec =
+            engine::assets::read_gguf_embedded_model_spec(std::filesystem::path(path));
+        if (spec.has_value()) {
+            return spec->family;
+        }
+    } catch (...) {
+        // A file that is not a readable GGUF simply has no family. The load
+        // gate turns that into a clear refusal; a throw here would surface as
+        // an opaque internal error during backend probing.
+    }
+    return {};
+}
+
+std::string resolve_model_path(const std::string &model_path_dir,
+                               const std::string &model_file,
+                               const std::string &model_name) {
+    const std::string candidate = !model_file.empty() ? model_file : model_name;
+
+    // The bundled: form is looked for in BOTH fields, and in ModelOptions.Model
+    // FIRST, because that is the only field it survives in. LocalAI fills
+    // ModelFile by joining ModelPath onto the configured model string
+    // (pkg/model/loader.go, LoadModelWithFile), so a model YAML saying
+    // `model: bundled:silero_vad` arrives here as ModelFile
+    // "/models/bundled:silero_vad" and Model "bundled:silero_vad". Testing
+    // `candidate` alone therefore made the zero-download VAD path reachable only
+    // from a hand-written LoadModel call that left ModelFile empty, and every
+    // model YAML using it failed with "model path does not exist".
+    const std::string bundled_prefix = "bundled:";
+    for (const std::string *field : {&model_name, &model_file}) {
+        if (field->rfind(bundled_prefix, 0) == 0) {
+            const std::string name = field->substr(bundled_prefix.size());
+            return (executable_directory() / "assets" / name).string();
+        }
+    }
+
+    std::filesystem::path path(candidate);
+    if (path.is_absolute() || model_path_dir.empty()) {
+        return path.string();
+    }
+    return (std::filesystem::path(model_path_dir) / path).string();
+}
+
+LoadedModel::LoadedModel(const std::string &resolved_path,
+                         const ModelOptions &options,
+                         std::string model_identity)
+    : LoadedModel(resolved_path, options, require_family(resolved_path, options),
+                  std::move(model_identity)) {}
+
+LoadedModel::LoadedModel(const std::string &resolved_path,
+                         const ModelOptions &options, std::string family,
+                         std::string model_identity)
+    : lane_(family), registry_(engine::runtime::make_default_registry()),
+      identity_(std::move(model_identity)) {
+    if (!registry_.supports_family(family)) {
+        throw ConfigError("audio-cpp: unknown audio.cpp family '" + family + "'");
+    }
+
+    // Before the load, for the same reason parse_backend_type runs before it:
+    // a refusal a metadata read can produce should not cost a full model load.
+    // More importantly it must precede the FIRST REQUEST, since that is where
+    // an unsupported dtype aborts the process rather than failing.
+    require_supported_weight_dtypes(family, resolved_path);
+
+    // Session options are built BEFORE the load, because parse_backend_type
+    // rejects an unknown backend name. Validating after the load would make
+    // `backend:cudaa` cost a full model load, on a fault a string comparison
+    // could have caught.
+    session_options_.backend.type = parse_backend_type(options.backend);
+    session_options_.backend.device = options.device;
+    if (options.threads > 0) {
+        session_options_.backend.threads = options.threads;
+    }
+    for (const auto &entry : options.session_options) {
+        session_options_.options[entry.first] = entry.second;
+    }
+
+    pinned_task_ = options.task;
+    wait_budget_ceiling_ms_ = options.busy_timeout_ms;
+    live_idle_timeout_ms_ = options.live_idle_timeout_ms;
+
+    engine::runtime::ModelLoadRequest request;
+    request.model_path = std::filesystem::path(resolved_path);
+    request.family_hint = family;
+    if (!options.model_spec_override.empty()) {
+        request.model_spec_override =
+            std::filesystem::path(options.model_spec_override);
+    }
+    for (const auto &entry : options.load_options) {
+        request.options[entry.first] = entry.second;
+    }
+
+    try {
+        model_ = registry_.load(request);
+    } catch (const std::exception &err) {
+        throw ConfigError("audio-cpp: failed to load family '" + family +
+                          "' from " + resolved_path + ": " + err.what());
+    }
+    if (model_ == nullptr) {
+        throw ConfigError("audio-cpp: the registry returned no model for " +
+                          resolved_path);
+    }
+
+    const auto &metadata = model_->metadata();
+    const auto &engine_caps = model_->capabilities();
+    variant_ = metadata.variant;
+    description_ = metadata.description;
+    languages_ = engine_caps.languages;
+    supports_timestamps_ = engine_caps.supports_timestamps;
+    capabilities_ = to_capabilities(family, engine_caps);
+}
+
+Route LoadedModel::check_can_serve(Rpc rpc, const RequestShape &shape) const {
+    const Route route = resolve_route(rpc, shape, capabilities_);
+    if (!route.ok) {
+        throw CapabilityError(route.error);
+    }
+    // Returned so a handler can act on the task before running it. It is the
+    // same route session_for will resolve, since both read the immutable
+    // capabilities_ from the same shape.
+    return route;
+}
+
+LoadedModel::Session LoadedModel::session_for(Rpc rpc, const RequestShape &shape,
+                                              LaneEntry &lane) {
+    // Proof of holding only. Nothing here reads it, and nothing should: its
+    // whole job is to make a caller that has not taken the lane fail to
+    // compile. Non-const so it cannot bind to an inline acquire(), whose
+    // temporary would be released at the end of this call.
+    (void)lane;
+    const Route route = resolve_route(rpc, shape, capabilities_);
+    if (!route.ok) {
+        throw CapabilityError(route.error);
+    }
+
+    const SessionKey key{static_cast<int>(route.task), static_cast<int>(route.mode)};
+    auto found = sessions_.find(key);
+    const bool cache_hit = found != sessions_.end();
+    if (!cache_hit) {
+        engine::runtime::TaskSpec spec;
+        spec.task = to_engine_task(route.task);
+        spec.mode = to_engine_mode(route.mode);
+        std::unique_ptr<engine::runtime::IVoiceTaskSession> created;
+        try {
+            created = model_->create_task_session(spec, session_options_);
+        } catch (const std::exception &err) {
+            // NOT a CapabilityError. The family said it supports this pair, and
+            // a throw from here is overwhelmingly an environment fault: a ggml
+            // backend .so that package.sh did not ship, an out of memory, a CUDA
+            // device that is not there. UNIMPLEMENTED would tell LocalAI and
+            // every client "this model cannot do this, never retry", and send an
+            // operator hunting a capability bug instead of a packaging one. A
+            // plain runtime_error maps to INTERNAL, which is what a fixable
+            // deployment fault should look like.
+            throw std::runtime_error(
+                std::string("audio-cpp: family '") + capabilities_.family +
+                "' advertises " + task_name(route.task) + "/" +
+                mode_name(route.mode) + " but refused to create the session: " +
+                err.what());
+        }
+        if (created == nullptr) {
+            // A null return with no throw is the family declining, which is a
+            // genuine capability answer and stays UNIMPLEMENTED.
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' returned no session for " +
+                                  task_name(route.task) + "/" +
+                                  mode_name(route.mode));
+        }
+        found = sessions_.emplace(key, std::move(created)).first;
+    }
+
+    Session session;
+    session.task = route.task;
+    session.mode = route.mode;
+    engine::runtime::IVoiceTaskSession *raw = found->second.get();
+    if (route.mode == Mode::Streaming) {
+        session.streaming =
+            dynamic_cast<engine::runtime::IStreamingVoiceTaskSession *>(raw);
+        if (session.streaming == nullptr) {
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' advertises " + task_name(route.task) +
+                                  "/streaming but its session is not streaming");
+        }
+        // Deliberately NOT reset here, though a cached streaming session does
+        // carry state across chunks. reset() is not callable at this point:
+        // silero_vad's implementation throws "session prepare() must be called
+        // before Silero VAD reset()", so resetting on a cache hit would turn an
+        // ordinary second fetch into a hard error, which is worse than the leak
+        // it would prevent.
+        //
+        // The state is instead cleared by the sequence every streaming caller
+        // owes anyway. IStreamingVoiceTaskSession::start_stream's base
+        // implementation IS a call to reset(), so a caller that runs
+        // prepare(...) then start_stream(...) at the top of each stream gets a
+        // clean session for free. See the STATE CONTRACT in loaded_model.h.
+    } else {
+        session.offline =
+            dynamic_cast<engine::runtime::IOfflineVoiceTaskSession *>(raw);
+        if (session.offline == nullptr) {
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' advertises " + task_name(route.task) +
+                                  "/offline but its session is not offline");
+        }
+    }
+    return session;
+}
+
+LaneEntry LoadedModel::acquire(int requested_timeout_ms) {
+    // Constructed straight into the return value. C++17 requires that, which is
+    // what lets an immovable type be returned at all; a named local here would
+    // not compile.
+    return LaneEntry(lane_,
+                     resolve_wait_budget_ms(wait_budget_ceiling_ms_,
+                                            requested_timeout_ms));
+}
+
+std::unique_ptr<LaneEntry> LoadedModel::acquire_owned(int requested_timeout_ms) {
+    return std::make_unique<LaneEntry>(
+        lane_,
+        resolve_wait_budget_ms(wait_budget_ceiling_ms_, requested_timeout_ms));
+}
+
+engine::runtime::TaskResult run_offline(const LoadedModel::Session &session,
+                                        const engine::runtime::TaskRequest &request,
+                                        LaneEntry &lane) {
+    // Proof of holding only, as in session_for.
+    (void)lane;
+    if (session.offline == nullptr) {
+        throw CapabilityError("audio-cpp: no offline session for this request");
+    }
+    session.offline->prepare(engine::runtime::build_preparation_request(request));
+    return session.offline->run(request);
+}
+
+namespace {
+
+engine::runtime::IStreamingVoiceTaskSession &
+require_streaming(const LoadedModel::Session &session) {
+    if (session.streaming == nullptr) {
+        throw CapabilityError("audio-cpp: no streaming session for this request");
+    }
+    return *session.streaming;
+}
+
+// Clears the stream event sink on every exit from the driver, including the
+// exception path. The session is CACHED and outlives the call that installed
+// the sink, so a std::function left behind holding references into that call's
+// frame is called with dangling captures by whoever streams next.
+class ScopedStreamSink {
+public:
+    ScopedStreamSink(engine::runtime::IStreamingVoiceTaskSession &session,
+                     engine::runtime::StreamEventCallback sink)
+        : session_(session) {
+        session_.set_stream_event_sink(std::move(sink));
+    }
+    ~ScopedStreamSink() { session_.set_stream_event_sink(nullptr); }
+
+    ScopedStreamSink(const ScopedStreamSink &) = delete;
+    ScopedStreamSink &operator=(const ScopedStreamSink &) = delete;
+
+private:
+    engine::runtime::IStreamingVoiceTaskSession &session_;
+};
+
+// Frames per chunk to feed a streaming session, from its own policy.
+//
+// FRAMES, not floats. preferred_audio_chunk_samples is a per-channel count
+// everywhere upstream sets it (nemotron_asr uses its frontend sample rate,
+// i.e. one second), and vibevoice_asr refuses a chunk whose float count is not
+// divisible by its channel count, so slicing on floats would both mis-size the
+// window and hand a family a half frame.
+std::int64_t chunk_frames_for(const engine::runtime::StreamingPolicy &policy,
+                              int sample_rate) {
+    if (policy.preferred_audio_chunk_samples > 0) {
+        return policy.preferred_audio_chunk_samples;
+    }
+    // higgs_audio_stt states its window in seconds (4.0) and leaves the sample
+    // count at zero, so this branch is real rather than defensive.
+    if (policy.preferred_audio_chunk_seconds > 0.0 && sample_rate > 0) {
+        const auto frames = static_cast<std::int64_t>(
+            policy.preferred_audio_chunk_seconds * static_cast<double>(sample_rate));
+        if (frames > 0) {
+            return frames;
+        }
+    }
+    // The interface's own default, from IStreamingVoiceTaskSession::streaming_policy.
+    return 512;
+}
+
+} // namespace
+
+void begin_stream(const LoadedModel::Session &session,
+                  const engine::runtime::TaskRequest &request, LaneEntry &lane) {
+    // Proof of holding only, as in session_for.
+    (void)lane;
+    auto &streaming = require_streaming(session);
+    // Order is load-bearing: start_stream's reset() is illegal before prepare().
+    streaming.prepare(engine::runtime::build_preparation_request(request));
+    streaming.start_stream(request);
+}
+
+engine::runtime::TaskResult run_streaming_pull(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+    begin_stream(session, request, lane);
+    while (const auto event = streaming.next_stream_event()) {
+        if (on_event) {
+            on_event(*event);
+        }
+        // No pinned family sets is_final on a pulled event, so this is not what
+        // ends the loop today; the nullopt above is. Honoured anyway, because a
+        // family that does set it is saying the stream is over and pulling once
+        // more would be asking a finished session for another chunk.
+        if (event->is_final) {
+            break;
+        }
+    }
+    return streaming.finish_stream();
+}
+
+engine::runtime::TaskResult run_streaming_audio(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const engine::runtime::AudioBuffer &audio,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+
+    const int channels = audio.channels > 0 ? audio.channels : 1;
+    // REFUSED, not rounded away, and checked before anything is touched so a
+    // refusal leaves no half-started stream on a cached session.
+    //
+    // An interleaved buffer whose float count is not a whole number of frames
+    // is a truncated input, and the integer division below would silently drop
+    // the tail floats: they would never be fed, never reach the transcript, and
+    // nothing would say so. Upstream refuses the same condition rather than
+    // tolerating it, in two places: vibevoice_asr's audio_frame_count throws
+    // "VibeVoice-ASR audio samples must be divisible by channel count"
+    // (session.cpp:70-76), and its process_audio_chunk throws the same with
+    // "streamed" in the text about the chunks this driver hands it
+    // (session.cpp:742-747).
+    //
+    // ConfigError, i.e. INVALID_ARGUMENT, because the buffer came from the
+    // caller's file. read_audio_file's positive-rate path always answers mono
+    // and so cannot reach this, but its native-rate path passes the reader's
+    // sample count through unchanged, and a driver does not get to assume which
+    // path its caller took.
+    if (audio.samples.size() % static_cast<std::size_t>(channels) != 0) {
+        throw ConfigError(
+            "audio-cpp: streaming input is not a whole number of frames: " +
+            std::to_string(audio.samples.size()) + " samples across " +
+            std::to_string(channels) + " channels");
+    }
+
+    // Installed BEFORE the stream begins, so a family that reports during
+    // start_stream is not silently dropped, and destroyed after finish_stream,
+    // because nemotron_asr emits every one of its partials from inside
+    // finalize().
+    ScopedStreamSink sink(streaming,
+                          [&on_event](const engine::runtime::StreamEvent &event) {
+                              if (on_event) {
+                                  on_event(event);
+                              }
+                          });
+
+    begin_stream(session, request, lane);
+
+    const auto total_frames =
+        static_cast<std::int64_t>(audio.samples.size() / static_cast<size_t>(channels));
+    const std::int64_t chunk_frames =
+        chunk_frames_for(streaming.streaming_policy(), audio.sample_rate);
+
+    for (std::int64_t offset = 0; offset < total_frames; offset += chunk_frames) {
+        const std::int64_t end = std::min(offset + chunk_frames, total_frames);
+        engine::runtime::AudioChunk chunk;
+        chunk.sample_rate = audio.sample_rate;
+        chunk.channels = channels;
+        // A FRAME index, which is what every span in a returned event is
+        // expressed in. vibevoice_asr adds the chunk's own frame count to it to
+        // offset the spans it reports, so a float index here would place every
+        // span of a stereo stream at twice its real time.
+        chunk.start_sample = offset;
+        chunk.samples.assign(
+            audio.samples.begin() + static_cast<std::ptrdiff_t>(offset * channels),
+            audio.samples.begin() + static_cast<std::ptrdiff_t>(end * channels));
+        const auto event = streaming.process_audio_chunk(chunk);
+        if (on_event) {
+            on_event(event);
+        }
+    }
+    return streaming.finish_stream();
+}
+
+engine::runtime::TaskResult run_streaming_live(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<bool(std::vector<float> &)> &next_frames,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+
+    // The contract is the only thing that says what rate and layout the frames
+    // about to arrive are in, and prepare() needs it: see the header.
+    if (!request.audio_input.has_value()) {
+        throw ConfigError(
+            "audio-cpp: a live streaming request carries no audio contract");
+    }
+    const int sample_rate = request.audio_input->sample_rate;
+    const int channels =
+        request.audio_input->channels > 0 ? request.audio_input->channels : 1;
+
+    // Installed BEFORE the stream begins and cleared on every exit, including
+    // the exception path, for the reasons spelled out in run_streaming_audio.
+    ScopedStreamSink sink(streaming,
+                          [&on_event](const engine::runtime::StreamEvent &event) {
+                              if (on_event) {
+                                  on_event(event);
+                              }
+                          });
+
+    begin_stream(session, request, lane);
+
+    const std::int64_t chunk_frames =
+        chunk_frames_for(streaming.streaming_policy(), sample_rate);
+    // chunk_frames_for never returns a non-positive count, so this is never
+    // zero and the accumulation loop below always terminates.
+    const std::size_t chunk_floats = static_cast<std::size_t>(chunk_frames) *
+                                     static_cast<std::size_t>(channels);
+
+    std::int64_t fed_frames = 0;
+    const auto feed = [&](std::vector<float> samples) {
+        engine::runtime::AudioChunk chunk;
+        chunk.sample_rate = sample_rate;
+        chunk.channels = channels;
+        // A FRAME index, counted across the whole stream: vibevoice_asr offsets
+        // every span it reports by it, so restarting it per chunk would put
+        // every word at the top of the recording.
+        chunk.start_sample = fed_frames;
+        chunk.samples = std::move(samples);
+        fed_frames +=
+            static_cast<std::int64_t>(chunk.samples.size()) / channels;
+        const auto event = streaming.process_audio_chunk(chunk);
+        if (on_event) {
+            on_event(event);
+        }
+    };
+
+    std::vector<float> pending;
+    std::vector<float> incoming;
+    while (true) {
+        incoming.clear();
+        if (!next_frames(incoming)) {
+            break;
+        }
+        pending.insert(pending.end(), incoming.begin(), incoming.end());
+        while (pending.size() >= chunk_floats) {
+            std::vector<float> window(pending.begin(),
+                                      pending.begin() +
+                                          static_cast<std::ptrdiff_t>(chunk_floats));
+            pending.erase(pending.begin(),
+                          pending.begin() +
+                              static_cast<std::ptrdiff_t>(chunk_floats));
+            feed(std::move(window));
+        }
+    }
+
+    if (!pending.empty()) {
+        // The tail is whatever did not fill a window. Refused rather than
+        // truncated when it is not a whole number of frames, exactly as in
+        // run_streaming_audio: the division above would drop the stray floats
+        // from the transcript with no diagnostic. Unreachable for a mono live
+        // stream, which is every live stream today.
+        if (pending.size() % static_cast<std::size_t>(channels) != 0) {
+            throw ConfigError(
+                "audio-cpp: live stream ended mid-frame: " +
+                std::to_string(pending.size()) + " trailing samples across " +
+                std::to_string(channels) + " channels");
+        }
+        feed(std::move(pending));
+    }
+
+    if (fed_frames == 0) {
+        // Nothing was spoken. See the header: finalizing an empty stream is not
+        // legal for every family, and an empty transcript is the truthful
+        // answer rather than an engine-internal INTERNAL.
+        return engine::runtime::TaskResult{};
+    }
+    return streaming.finish_stream();
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/loaded_model.h

@@ -0,0 +1,378 @@
+#pragma once
+
+// Owns one audio.cpp model for the life of the process, plus a lazily created
+// session per (task, mode) so the same model answers both TTS and TTSStream.
+// This is the only unit that converts between the stdlib-only mirror types and
+// engine::runtime types.
+
+#include "capability_routing.h"
+#include "inference_lane.h"
+#include "model_options.h"
+
+#include "engine/framework/runtime/model.h"
+#include "engine/framework/runtime/registry.h"
+#include "engine/framework/runtime/session.h"
+
+#include <functional>
+#include <map>
+#include <memory>
+#include <stdexcept>
+#include <string>
+#include <utility>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// User-fixable configuration problem. grpc-server maps this to INVALID_ARGUMENT.
+class ConfigError : public std::runtime_error {
+public:
+    using std::runtime_error::runtime_error;
+};
+
+// The family cannot serve the requested RPC. Maps to UNIMPLEMENTED.
+class CapabilityError : public std::runtime_error {
+public:
+    using std::runtime_error::runtime_error;
+};
+
+engine::runtime::VoiceTaskKind to_engine_task(Task task);
+engine::runtime::RunMode to_engine_mode(Mode mode);
+Task from_engine_task(engine::runtime::VoiceTaskKind kind);
+Mode from_engine_mode(engine::runtime::RunMode mode);
+Capabilities to_capabilities(const std::string &family,
+                             const engine::runtime::CapabilitySet &set);
+
+// Reads audiocpp.model_spec.family from a GGUF. Returns an empty string when
+// the file is not a GGUF, carries no audio.cpp spec, or cannot be read. Never
+// throws: an unreadable file is the load gate's problem, not a crash.
+std::string read_gguf_family(const std::string &path);
+
+// Builds the absolute model path from LocalAI's (ModelPath, ModelFile, Model)
+// triple. Either the Model or the ModelFile field may carry the form
+// "bundled:<name>", which resolves to <executable dir>/assets/<name>, where
+// package.sh puts upstream's bundled silero_vad and marblenet_vad assets. BOTH
+// are checked because LocalAI fills ModelFile by joining ModelPath onto the
+// configured model string, so a model YAML using the form has it intact only in
+// Model.
+std::string resolve_model_path(const std::string &model_path_dir,
+                               const std::string &model_file,
+                               const std::string &model_name);
+
+class LoadedModel {
+public:
+    struct Session {
+        Task task = Task::Tts;
+        Mode mode = Mode::Offline;
+        // Exactly one of these is non-null, matching the resolved mode.
+        engine::runtime::IOfflineVoiceTaskSession *offline = nullptr;
+        engine::runtime::IStreamingVoiceTaskSession *streaming = nullptr;
+    };
+
+    // Throws ConfigError when the path does not exist, the family cannot be
+    // determined, or the registry rejects the family.
+    //
+    // `model_identity` is ModelOptions.Model verbatim: the UNTRANSLATED
+    // controller-side name. It is a constructor argument rather than a setter
+    // so identity and model are inseparable. llama-cpp keeps its equivalent in
+    // a separate global from the model, which leaves a window where a handler
+    // can read one without the other; here a handler that holds the model
+    // through snapshot() necessarily holds the identity it was loaded with.
+    LoadedModel(const std::string &resolved_path, const ModelOptions &options,
+                std::string model_identity);
+
+    LoadedModel(const LoadedModel &) = delete;
+    LoadedModel &operator=(const LoadedModel &) = delete;
+
+    const std::string &family() const noexcept { return capabilities_.family; }
+    // Empty when the controller predates ModelOptions.ModelIdentity, which the
+    // identity check reads as "skip". See check_model_identity in grpc-server.
+    const std::string &identity() const noexcept { return identity_; }
+    const std::string &variant() const noexcept { return variant_; }
+    const std::string &description() const noexcept { return description_; }
+    const std::vector<std::string> &languages() const noexcept { return languages_; }
+    const Capabilities &capabilities() const noexcept { return capabilities_; }
+    bool supports_timestamps() const noexcept { return supports_timestamps_; }
+    const engine::runtime::SessionOptions &session_options() const noexcept {
+        return session_options_;
+    }
+
+    // The model's `task:` option, empty when unset. Every handler must copy it
+    // into RequestShape::pinned_task before calling session_for: routing is
+    // otherwise derived from the RPC alone, and this is the option's only route
+    // from the load to the request that honours it.
+    const std::string &pinned_task() const noexcept { return pinned_task_; }
+
+    // The `live_idle_timeout_ms` option: how long AudioTranscriptionLive waits
+    // for the next audio frame before cancelling the stream to give this
+    // model's lane back. 0 means no limit. See the option in model_options.h
+    // for why it exists and how the default was chosen.
+    int live_idle_timeout_ms() const noexcept { return live_idle_timeout_ms_; }
+
+    // Throws the same CapabilityError session_for would throw when this family
+    // cannot serve the RPC, and RETURNS THE RESOLVED ROUTE otherwise.
+    //
+    // The route is returned rather than computed and dropped because a handler
+    // often has to know which task it is about to run BEFORE running it.
+    // AudioTransform refuses params[stem] on any route but source separation,
+    // and reading that off the route costs microseconds where reading it off
+    // the result costs a whole inference first. A caller with no such need
+    // ignores the value, which is what the three transcription-shaped handlers
+    // do.
+    //
+    // It exists so a refusal does not have to buy a place in the queue first.
+    // resolve_route is a pure function of capabilities_, which is fixed at
+    // construction and never written again, so unlike the session cache it
+    // needs no lane and no lock: a model that cannot transcribe can say so
+    // while another request is halfway through a thirty second run. Without
+    // this the refusal waits for that run to finish only to be told no.
+    //
+    // It does NOT replace the routing inside session_for, and must not be made
+    // to: session_for still needs the route to key the session cache. The two
+    // calls agree because both read the same immutable capabilities. What this
+    // one adds is only the ordering, so call it before acquire().
+    //
+    // Const and lane-free on purpose. If a future edit makes routing depend on
+    // mutable state, this must grow the lane parameter its siblings carry.
+    Route check_can_serve(Rpc rpc, const RequestShape &shape) const;
+
+    // Routes the RPC and returns the cached session, creating it on first use.
+    // Throws CapabilityError when this family cannot serve the RPC, and a plain
+    // runtime_error when it can but the session could not be built, which is an
+    // environment fault rather than a capability answer.
+    //
+    // The `lane` parameter is a PROOF OF HOLDING and is otherwise unused: it
+    // exists so the rule below is a compile error rather than prose. The
+    // session cache is an unsynchronised std::map and the sessions themselves
+    // are not reentrant, so this must only be called with the lane held; the
+    // lane admits one caller at a time, which is exactly the constraint the
+    // sessions impose. Pass the LaneEntry from acquire().
+    //
+    // NON-CONST reference on purpose, and do not "tidy" it to const. A const
+    // reference binds to a temporary, which makes this compile:
+    //
+    //     auto session = model->session_for(rpc, shape, model->acquire(0));
+    //     auto result  = run_offline(session, task, model->acquire(0));
+    //
+    // and each temporary dies at the end of its own full-expression, so the
+    // lane is released between the two calls. That is precisely the split this
+    // parameter exists to prevent, and it is the form a future caller is most
+    // likely to reach for because it reads as tidy. Requiring an lvalue forces
+    // a named entry whose scope spans both calls.
+    //
+    // What it proves is bounded, so do not over-trust it: it proves A lane was
+    // taken, not THIS model's lane. A caller determined to defeat it can
+    // construct an entry on an unrelated InferenceLane and pass that. It
+    // therefore catches the two mistakes that actually happen, forgetting the
+    // lane entirely and taking it after routing, and does not catch lane
+    // identity.
+    //
+    // STATE CONTRACT, and it is the CALLER'S to honour. Sessions are cached per
+    // (task, mode), so a streaming session is normally the same warm object the
+    // previous stream used, carrying that stream's state. session_for hands it
+    // back as it is.
+    //
+    // Every streaming caller must therefore begin a stream through
+    // begin_stream() below, which is prepare() then start_stream() in that
+    // order and is the ONLY implementation of that sequence. start_stream's
+    // base implementation is a call to reset(), which is what clears the
+    // previous stream, and reset() is only legal after prepare(): silero_vad
+    // throws "session prepare() must be called before Silero VAD reset()"
+    // otherwise. That ordering constraint is also why session_for cannot do
+    // this for you. Skipping it does not raise an error, it silently continues
+    // the previous stream.
+    //
+    // Offline sessions need no such care: their interface has no reset and
+    // run() takes a whole request.
+    Session session_for(Rpc rpc, const RequestShape &shape, LaneEntry &lane);
+
+    // Takes the inference lane, or throws LaneUnavailable. Serializes runs
+    // against this model. `requested_timeout_ms` is a per-request wait hint
+    // where a value <= 0 means "use the model's configured ceiling"; a hint may
+    // only tighten that ceiling, never loosen it.
+    //
+    // LaneEntry is deliberately immovable, so bind the result to a named local
+    // in the scope the inference happens in:
+    //
+    //     LaneEntry entry = model.acquire(request_hint_ms);
+    //
+    // which C++17 initializes in place. A handler that has to keep the lane
+    // beyond one scope, for instance in a member that outlives the call that
+    // took it, wants acquire_owned instead.
+    LaneEntry acquire(int requested_timeout_ms);
+
+    // Same lane, heap-allocated so it can be stored or handed on. Prefer
+    // acquire: this one adds a null state that the scoped form does not have.
+    std::unique_ptr<LaneEntry> acquire_owned(int requested_timeout_ms);
+
+private:
+    // Keyed by the enum values so the map needs no custom comparator.
+    using SessionKey = std::pair<int, int>;
+
+    // The public constructor runs the load gate, then delegates here. The
+    // detour exists because lane_ has to be built from the family in the member
+    // initializer list, and the family is only known after the gate has run.
+    // Four parameters rather than three so it cannot be confused with the
+    // public constructor, whose third argument is also a std::string.
+    LoadedModel(const std::string &resolved_path, const ModelOptions &options,
+                std::string family, std::string model_identity);
+
+    // MEMBER ORDER IS LOAD-BEARING BELOW THIS LINE. Members are destroyed in
+    // reverse declaration order.
+    //
+    // lane_ is first so it is destroyed last: nothing that runs during teardown
+    // can then find a lane that has already gone.
+    InferenceLane lane_;
+    // registry_ before model_: the registry owns the loader that produced the
+    // model, and the model may hold loader-owned state.
+    engine::runtime::ModelRegistry registry_;
+    // model_ before sessions_, so sessions_ is destroyed FIRST and the model
+    // second. A session is created from the model and must not outlive it. Do
+    // not reorder these two.
+    std::unique_ptr<engine::runtime::ILoadedVoiceModel> model_;
+    std::map<SessionKey, std::unique_ptr<engine::runtime::IVoiceTaskSession>> sessions_;
+
+    engine::runtime::SessionOptions session_options_;
+    Capabilities capabilities_;
+    std::string variant_;
+    std::string description_;
+    std::vector<std::string> languages_;
+    std::string pinned_task_;
+    std::string identity_;
+    bool supports_timestamps_ = false;
+    int wait_budget_ceiling_ms_ = 0;
+    int live_idle_timeout_ms_ = 0;
+};
+
+// Prepares and runs an offline session. prepare() is called for every run
+// rather than once per session, because SessionPreparationRequest is derived
+// from the request itself (audio contract, text, voice condition) and not from
+// the model: a second request with a different sample rate or length would
+// otherwise run against the first request's contract.
+//
+// `lane` is a PROOF OF HOLDING, unused at runtime, for the same reason
+// session_for takes one: the session is not reentrant and prepare() mutates it,
+// so running without the lane is a data race. Making it a parameter turns that
+// into a compile error instead of a comment. Non-const for the same reason as
+// session_for's: a const reference would bind to `model.acquire(0)` written
+// inline, and that temporary dies at the end of this call, releasing the lane
+// before the caller's next one.
+//
+// Throws CapabilityError when the session is not an offline one. That should be
+// unreachable through session_for, which already refuses a non-offline session
+// for an offline route, and is checked anyway because the alternative is a null
+// dereference.
+engine::runtime::TaskResult run_offline(const LoadedModel::Session &session,
+                                        const engine::runtime::TaskRequest &request,
+                                        LaneEntry &lane);
+
+// THE ONE IMPLEMENTATION of the streaming state obligation described in
+// session_for's STATE CONTRACT: prepare(), then start_stream(), in that order.
+//
+// It is a function rather than a comment because the obligation is invisible
+// when it is broken. Streaming sessions are CACHED per (task, mode), so the
+// object a second stream gets is the warm one the first stream left behind,
+// still holding its audio, its tokens and its started flag. What clears it is
+// start_stream, whose base implementation IS a reset() and whose seven family
+// overrides (nemotron_asr, vibevoice_asr, higgs_audio_stt, voxtral_realtime,
+// supertonic, omnivoice, voxcpm2) every one call reset() as their first
+// statement, verified in the pinned checkout. Nothing in the type system pins
+// that. A future override that dropped the reset would break every call site
+// at once with no compile error and no exception, only a second transcript
+// that begins with the first one's audio, so the fewer call sites there are to
+// break, the better: this is the only one.
+//
+// prepare() must come first and cannot be folded into session_for, because
+// reset() is illegal before prepare() (silero_vad throws "session prepare()
+// must be called before Silero VAD reset()"), and because the preparation
+// request is derived from the REQUEST, not the model: build_preparation_request
+// reads the audio contract, the text and the voice condition off it, so a
+// second stream with a different sample rate or length would otherwise run
+// against the first stream's contract.
+//
+// `lane` is a PROOF OF HOLDING, unused at runtime, exactly as in run_offline.
+//
+// Throws CapabilityError when the session is not a streaming one.
+void begin_stream(const LoadedModel::Session &session,
+                  const engine::runtime::TaskRequest &request, LaneEntry &lane);
+
+// Drives a streaming session that takes NO incremental input, which is the TTS
+// shape (StreamingInputKind::None, StreamingOutputKind::PullEvents): begin the
+// stream, pull events until the session says there are no more, then finish.
+//
+// NO STREAM EVENT SINK IS INSTALLED HERE, and that is deliberate rather than an
+// omission. voxcpm2's start_stream runs the whole synthesis and pushes every
+// chunk to the sink, then its next_stream_event replays those same chunks out
+// of the stored result, so a sink on this path would put every chunk of audio
+// on the wire twice. supertonic and omnivoice ignore set_stream_event_sink
+// outright. The pull loop is therefore the single delivery channel.
+//
+// The returned TaskResult is the session's own merged whole for all three
+// families, NOT a tail the pull loop missed. A caller that already emitted the
+// pulled events must not also emit its audio; see the TTSStream handler.
+engine::runtime::TaskResult run_streaming_pull(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+// Drives a streaming session that CONSUMES audio chunks, which is the ASR shape
+// (StreamingInputKind::AudioChunks): begin the stream, feed the buffer in
+// policy-sized chunks, then finalize.
+//
+// A STREAM EVENT SINK IS INSTALLED HERE, and it is not optional: nemotron_asr
+// reports its partial text ONLY through the sink, and only from inside
+// finalize(), because its decode does not start until the audio is complete.
+// Without the sink that family streams a transcript with no partials at all.
+// The sink is cleared again before returning, including on the exception path:
+// the session is cached and outlives this call, so a sink left holding a
+// reference to the caller's frame is a use after free waiting for the next
+// stream.
+//
+// Both delivery channels are consumed, the sink and the value process_audio_chunk
+// returns, because the families do not agree on which they use, and
+// voxtral_realtime uses BOTH for the same event. The duplicate that produces is
+// absorbed by TranscriptDeltaTracker in stream_delta.h rather than here.
+engine::runtime::TaskResult run_streaming_audio(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const engine::runtime::AudioBuffer &audio,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+// Drives the same ASR shape as run_streaming_audio when the audio DOES NOT
+// EXIST YET, which is the live-microphone case: instead of slicing a buffer it
+// pulls frames from the caller until the input side closes.
+//
+// `next_frames` fills `out` with interleaved float PCM and returns true, or
+// returns false when there is no more input. It is expected to BLOCK, since the
+// only real implementation is a gRPC stream Read, and it may throw: a request
+// the handler has to refuse mid-stream unwinds through here, and the sink is
+// cleared on that path like every other.
+//
+// The audio contract comes from `request.audio_input`, which for a live stream
+// is an EMPTY buffer carrying only the sample rate and channel count. It is not
+// optional: nemotron_asr's streaming prepare() throws "Nemotron ASR streaming
+// prepare() requires an audio contract" without one, and there is no buffer to
+// derive it from here.
+//
+// Frames are BUFFERED to the family's own preferred window rather than fed in
+// whatever sizes the wire delivered them in, because that window is a family's
+// statement about what it can decode (nemotron_asr asks for one second, higgs
+// for four), and a 512-sample gRPC frame is a property of the client's audio
+// callback rather than of the model. The tail shorter than a window is fed at
+// the end.
+//
+// A stream that carried NO AUDIO returns an empty TaskResult and never calls
+// finish_stream. Finalizing an empty stream is not universally legal:
+// nemotron_asr throws "Nemotron ASR finalize requires streamed audio", so a
+// client that opens a session and closes it without speaking would receive an
+// INTERNAL naming an engine internal instead of an empty transcript, which is
+// the truthful answer to "transcribe nothing".
+engine::runtime::TaskResult run_streaming_live(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<bool(std::vector<float> &)> &next_frames,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options.cpp

@@ -0,0 +1,153 @@
+#include "model_options.h"
+
+#include <cctype>
+#include <cerrno>
+#include <climits>
+#include <cstdlib>
+
+namespace audiocpp_backend {
+namespace {
+
+std::string trim(const std::string &value) {
+    size_t begin = 0;
+    while (begin < value.size() &&
+           std::isspace(static_cast<unsigned char>(value[begin])) != 0) {
+        ++begin;
+    }
+    size_t end = value.size();
+    while (end > begin &&
+           std::isspace(static_cast<unsigned char>(value[end - 1])) != 0) {
+        --end;
+    }
+    return value.substr(begin, end - begin);
+}
+
+// Parses a non-negative integer. Returns false on anything else, including
+// empty strings, signs, trailing garbage, and values too large for int.
+//
+// strtol rather than atoi: atoi is undefined behaviour once the digits exceed
+// long, and in practice it hands back a wrapped value. That would let
+// "device:2147483648" through as -2147483648 and send a negative index to the
+// ggml backend selector, from a function whose error text promises the caller a
+// non-negative integer.
+bool parse_non_negative_int(const std::string &value, int &out) {
+    if (value.empty()) {
+        return false;
+    }
+    for (const char ch : value) {
+        if (std::isdigit(static_cast<unsigned char>(ch)) == 0) {
+            return false;
+        }
+    }
+
+    errno = 0;
+    char *end = nullptr;
+    const long parsed = std::strtol(value.c_str(), &end, 10);
+    if (errno == ERANGE || end == nullptr || *end != '\0') {
+        return false;
+    }
+    if (parsed < 0 || parsed > INT_MAX) {
+        return false;
+    }
+
+    out = static_cast<int>(parsed);
+    return true;
+}
+
+bool starts_with(const std::string &value, const std::string &prefix) {
+    return value.size() >= prefix.size() &&
+           value.compare(0, prefix.size(), prefix) == 0;
+}
+
+} // namespace
+
+ParsedOptions parse_model_options(const std::vector<std::string> &entries) {
+    ParsedOptions parsed;
+
+    for (const auto &raw : entries) {
+        const std::string entry = trim(raw);
+        if (entry.empty()) {
+            continue;
+        }
+
+        // Split on the FIRST colon: values are often paths that contain more.
+        const size_t sep = entry.find(':');
+        if (sep == std::string::npos) {
+            parsed.error = "audio-cpp: option '" + entry +
+                           "' is not in key:value form";
+            return parsed;
+        }
+
+        const std::string key = trim(entry.substr(0, sep));
+        const std::string value = trim(entry.substr(sep + 1));
+
+        if (starts_with(key, "load.")) {
+            const std::string inner = key.substr(5);
+            if (inner.empty()) {
+                parsed.error = "audio-cpp: option '" + entry +
+                               "' has an empty load option name";
+                return parsed;
+            }
+            parsed.options.load_options[inner] = value;
+            continue;
+        }
+        if (starts_with(key, "session.")) {
+            const std::string inner = key.substr(8);
+            if (inner.empty()) {
+                parsed.error = "audio-cpp: option '" + entry +
+                               "' has an empty session option name";
+                return parsed;
+            }
+            parsed.options.session_options[inner] = value;
+            continue;
+        }
+
+        if (key == "family") {
+            parsed.options.family = value;
+        } else if (key == "task") {
+            parsed.options.task = value;
+        } else if (key == "backend") {
+            parsed.options.backend = value;
+        } else if (key == "model_spec_override") {
+            parsed.options.model_spec_override = value;
+        } else if (key == "device") {
+            if (!parse_non_negative_int(value, parsed.options.device)) {
+                parsed.error = "audio-cpp: option 'device' needs a non-negative "
+                               "integer, got '" + value + "'";
+                return parsed;
+            }
+            parsed.options.device_set = true;
+        } else if (key == "threads") {
+            if (!parse_non_negative_int(value, parsed.options.threads)) {
+                parsed.error = "audio-cpp: option 'threads' needs a non-negative "
+                               "integer, got '" + value + "'";
+                return parsed;
+            }
+        } else if (key == "busy_timeout_ms") {
+            if (!parse_non_negative_int(value, parsed.options.busy_timeout_ms)) {
+                parsed.error = "audio-cpp: option 'busy_timeout_ms' needs a "
+                               "non-negative integer, got '" + value + "'";
+                return parsed;
+            }
+        } else if (key == "live_idle_timeout_ms") {
+            if (!parse_non_negative_int(value,
+                                        parsed.options.live_idle_timeout_ms)) {
+                parsed.error = "audio-cpp: option 'live_idle_timeout_ms' needs a "
+                               "non-negative integer, got '" + value + "'";
+                return parsed;
+            }
+        } else {
+            // Quotes the whole entry, not just the key: an entry like ":value"
+            // has an empty key and would otherwise leave nothing to grep for.
+            parsed.error = "audio-cpp: unknown option key '" + entry +
+                           "'. Known keys: family, task, backend, device, "
+                           "threads, model_spec_override, busy_timeout_ms, "
+                           "live_idle_timeout_ms, load.<key>, session.<key>";
+            return parsed;
+        }
+    }
+
+    return parsed;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options.h

@@ -0,0 +1,66 @@
+#pragma once
+
+// Parses the model YAML's `options:` list (ModelOptions.Options in
+// backend.proto) into a struct. Standard library only: this unit is compiled
+// and tested by backend/cpp/run-unit-tests.sh without an audio.cpp checkout.
+
+#include <map>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+struct ModelOptions {
+    // audio.cpp model family. Empty means "derive from the GGUF's embedded
+    // audiocpp.model_spec.family key"; a non-GGUF path with an empty family is
+    // rejected at load time, not here.
+    std::string family;
+    // Pins the audio.cpp task, overriding RPC-based routing. Empty means route.
+    std::string task;
+    // ggml backend: cpu, cuda, vulkan, metal, best.
+    std::string backend = "cpu";
+    int device = 0;
+    // True once a `device:` entry has been seen. 0 is both the default and a
+    // legitimate device index, so the value alone cannot tell an explicit
+    // `device:0` from an unset option, and a caller merging in its own fallback
+    // would silently override the explicit choice.
+    bool device_set = false;
+    // 0 means "let the runtime decide".
+    int threads = 0;
+    std::string model_spec_override;
+    // 0 disables the run guard's fail-fast, restoring an unbounded wait.
+    int busy_timeout_ms = 0;
+    // How long AudioTranscriptionLive waits for the next audio frame before it
+    // cancels the stream and gives the model's lane back. 0 means NO LIMIT.
+    //
+    // It exists because that RPC holds the lane for the whole stream, so a peer
+    // that stops sending WITHOUT closing blocks every other request against this
+    // model for as long as its socket stays up. No other RPC can do that: they
+    // hold the lane across compute, which ends on its own.
+    //
+    // 30 seconds, and the number is picked from what the only in-tree client
+    // does. core/http/endpoints/openai/realtime.go drives a 300 ms ticker and
+    // feeds every tick that produced new audio while a turn is open, so 30 s of
+    // silence is a hundred ticks that delivered nothing: the peer is gone, or
+    // its socket is wedged. It is also comfortably longer than any pause a
+    // speaker takes mid-utterance, which is the case that must never be cut off,
+    // and backend.proto allows one stream to span many utterances, so a client
+    // that pauses for longer than this between them should raise it rather than
+    // discover it. Lowering it below a few seconds risks cancelling a live
+    // speaker; 0 turns the limit off for a client that legitimately idles.
+    int live_idle_timeout_ms = 30000;
+    // `load.<key>:<value>` entries, prefix stripped.
+    std::map<std::string, std::string> load_options;
+    // `session.<key>:<value>` entries, prefix stripped.
+    std::map<std::string, std::string> session_options;
+};
+
+struct ParsedOptions {
+    ModelOptions options;
+    // Non-empty means the caller must fail the load with INVALID_ARGUMENT.
+    std::string error;
+};
+
+ParsedOptions parse_model_options(const std::vector<std::string> &entries);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options_test.cpp

@@ -0,0 +1,165 @@
+// Unit tests for model_options. Standard library only, so
+// backend/cpp/run-unit-tests.sh picks this up with no engine checkout.
+//
+// The harness compiles this file as a single translation unit with no other
+// sources, so the implementation is included directly rather than linked.
+//
+// Build and run standalone:
+//   g++ -std=c++17 -I. model_options_test.cpp -o t && ./t
+
+#include "model_options.cpp"
+
+#include <cstdio>
+#include <map>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+// Returns the mapped value, or an empty string when the key is absent. map::at
+// would throw on a miss and abort the whole binary, so one prefix off-by-one
+// would hide every check that follows it instead of failing a single one.
+static std::string lookup(const std::map<std::string, std::string> &values,
+                          const std::string &key) {
+    const auto found = values.find(key);
+    return found == values.end() ? std::string() : found->second;
+}
+
+using audiocpp_backend::parse_model_options;
+
+static void test_defaults() {
+    auto r = parse_model_options({});
+    check(r.error.empty(), "empty option list is not an error");
+    check(r.options.family.empty(), "family defaults to empty");
+    check(r.options.task.empty(), "task defaults to empty");
+    check(r.options.backend == "cpu", "backend defaults to cpu");
+    check(r.options.device == 0, "device defaults to 0");
+    check(r.options.threads == 0, "threads defaults to 0");
+    check(r.options.busy_timeout_ms == 0, "busy_timeout_ms defaults to 0");
+    // NOT zero, unlike every other numeric option here. A live stream holds the
+    // model's lane while it waits on the client, so the default has to bound
+    // that wait; 0 is the explicit "no limit" the operator opts into.
+    check(r.options.live_idle_timeout_ms == 30000,
+          "live_idle_timeout_ms defaults to 30000");
+    check(r.options.load_options.empty(), "load_options defaults empty");
+    check(r.options.session_options.empty(), "session_options defaults empty");
+}
+
+static void test_scalar_options() {
+    auto r = parse_model_options({
+        "family:qwen3_tts",
+        "task:tts",
+        "backend:cuda",
+        "device:1",
+        "threads:8",
+        "busy_timeout_ms:30000",
+        "live_idle_timeout_ms:5000",
+    });
+    check(r.error.empty(), "scalar options parse without error");
+    check(r.options.family == "qwen3_tts", "family parsed");
+    check(r.options.task == "tts", "task parsed");
+    check(r.options.backend == "cuda", "backend parsed");
+    check(r.options.device == 1, "device parsed");
+    check(r.options.threads == 8, "threads parsed");
+    check(r.options.busy_timeout_ms == 30000, "busy_timeout_ms parsed");
+    check(r.options.live_idle_timeout_ms == 5000, "live_idle_timeout_ms parsed");
+    check(parse_model_options({"live_idle_timeout_ms:0"}).options.live_idle_timeout_ms == 0,
+          "an explicit 0 turns the live idle limit off rather than reverting to "
+          "the default");
+}
+
+// Values containing colons must survive: split on the FIRST colon only.
+static void test_value_containing_colon() {
+    auto r = parse_model_options({"model_spec_override:/models/a:b/spec.json"});
+    check(r.error.empty(), "colon-bearing value is not an error");
+    check(r.options.model_spec_override == "/models/a:b/spec.json",
+          "value keeps every colon after the first separator");
+}
+
+static void test_namespaced_options() {
+    auto r = parse_model_options({
+        "load.weight_type:q8_0",
+        "session.miocodec.weight_type:f16",
+        "session.graph_capacity:tiered",
+    });
+    check(r.error.empty(), "namespaced options parse without error");
+    check(r.options.load_options.size() == 1, "one load option");
+    check(lookup(r.options.load_options, "weight_type") == "q8_0", "load prefix stripped");
+    check(r.options.session_options.size() == 2, "two session options");
+    check(lookup(r.options.session_options, "miocodec.weight_type") == "f16",
+          "session prefix stripped, inner dots kept");
+    check(lookup(r.options.session_options, "graph_capacity") == "tiered",
+          "second session option parsed");
+}
+
+static void test_errors() {
+    check(!parse_model_options({"family"}).error.empty(),
+          "entry without a colon is rejected");
+    check(!parse_model_options({"nonsense:1"}).error.empty(),
+          "unknown key is rejected");
+    check(!parse_model_options({"device:abc"}).error.empty(),
+          "non-numeric device is rejected");
+    check(!parse_model_options({"threads:-1"}).error.empty(),
+          "negative threads is rejected");
+    check(!parse_model_options({"load.:x"}).error.empty(),
+          "empty load key is rejected");
+    check(!parse_model_options({"session.:x"}).error.empty(),
+          "empty session key is rejected");
+    check(!parse_model_options({"busy_timeout_ms:abc"}).error.empty(),
+          "non-numeric busy_timeout_ms is rejected");
+    check(!parse_model_options({"live_idle_timeout_ms:abc"}).error.empty(),
+          "non-numeric live_idle_timeout_ms is rejected");
+    check(!parse_model_options({"live_idle_timeout_ms:-1"}).error.empty(),
+          "negative live_idle_timeout_ms is rejected");
+    check(!parse_model_options({"device:-1"}).error.empty(),
+          "negative device is rejected");
+    check(!parse_model_options({"threads:x"}).error.empty(),
+          "non-numeric threads is rejected");
+
+    // Values too large for int must be rejected, not silently wrapped into a
+    // negative device index that then reaches the ggml backend selector.
+    check(!parse_model_options({"device:2147483648"}).error.empty(),
+          "device above INT_MAX is rejected");
+    check(!parse_model_options({"threads:99999999999999"}).error.empty(),
+          "threads above INT_MAX is rejected");
+
+    // The error text must name the offending entry so a user can fix their YAML.
+    const auto r = parse_model_options({"nonsense:1"});
+    check(r.error.find("nonsense") != std::string::npos,
+          "error names the offending key");
+
+    // An empty key still has to give the user something to grep for.
+    const auto empty_key = parse_model_options({":value"});
+    check(empty_key.error.find(":value") != std::string::npos,
+          "unknown-key error names the entry even when the key is empty");
+}
+
+static void test_blank_entries_ignored() {
+    auto r = parse_model_options({"", "   ", "family:supertonic"});
+    check(r.error.empty(), "blank entries are skipped, not rejected");
+    check(r.options.family == "supertonic", "real entry still parsed");
+}
+
+int main() {
+    test_defaults();
+    test_scalar_options();
+    test_value_containing_colon();
+    test_namespaced_options();
+    test_errors();
+    test_blank_entries_ignored();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all model_options checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/package.sh

@@ -0,0 +1,228 @@
+#!/bin/bash
+# Assemble backend/cpp/audio-cpp/package, which becomes the whole content of the
+# FROM scratch backend image. Nothing outside this directory exists at run time.
+set -euo pipefail
+
+CURDIR=$(dirname "$(realpath "$0")")
+REPO_ROOT="${CURDIR}/../../.."
+PACKAGE_DIR="$CURDIR/package"
+BUILD_DIR="$CURDIR/build"
+
+rm -rf "$PACKAGE_DIR"
+mkdir -p "$PACKAGE_DIR/lib" "$PACKAGE_DIR/assets"
+
+cp -avf "$CURDIR/grpc-server" "$PACKAGE_DIR/"
+cp -fv  "$CURDIR/run.sh"      "$PACKAGE_DIR/"
+
+# ENGINE_ENABLE_CPU_ALL_VARIANTS builds the ggml backends as shared objects that
+# are dlopened at run time, so ldd cannot see them and the dependency walk below
+# would leave the image with no CPU backend at all. They also cannot go in lib/:
+# ggml DISCOVERS them by listing dirname(/proc/self/exe) and the current
+# directory, so being on a library path is not enough, they have to be in a
+# directory ggml scans. run.sh execs the bundled loader from the package root
+# exactly so that directory is this one. cmake writes them to build/bin, not
+# next to build/grpc-server, which is why this reads from bin/.
+#
+# -a keeps the libggml.so -> libggml.so.0 -> libggml.so.0.12.0 symlink chain,
+# so the SONAME the binary asks for still names a file here.
+for pattern in '*.so*' '*.dylib*'; do
+    if compgen -G "$BUILD_DIR/bin/$pattern" > /dev/null; then
+        # shellcheck disable=SC2086
+        cp -avf "$BUILD_DIR/bin/"$pattern "$PACKAGE_DIR/"
+    fi
+done
+
+# Upstream ships silero_vad and marblenet_vad as small runtime assets.
+# resolve_model_path() expands "bundled:<name>" to
+# dirname(/proc/self/exe)/assets/<name>, so copying them here is what makes VAD
+# work with nothing downloaded.
+for asset in silero_vad marblenet_vad; do
+    src="$CURDIR/audio.cpp/assets/framework/models/$asset"
+    if [ -d "$src" ]; then
+        cp -rfv "$src" "$PACKAGE_DIR/assets/"
+    else
+        echo "package.sh: bundled asset missing: $src" >&2
+        echo "package.sh: run 'make audio.cpp' before packaging" >&2
+        exit 1
+    fi
+done
+
+# Everything below this point is Linux-only: a bundled ELF loader, an ldd walk
+# and an ld.so --list validation. The macOS equivalent is the otool -L closure in
+# scripts/build/audio-cpp-darwin.sh, which picks up from the exit 0 below.
+#
+# WARNING FOR ANYONE REWORKING THAT SCRIPT. The obvious move is to copy
+# scripts/build/privacy-filter-darwin.sh, and that script assembles its own
+# package under build/darwin and never calls package.sh at all. Adapted as-is it
+# will silently omit assets/, and the bundled: model path form then resolves to
+# nothing, which takes the only zero-download verification path in this backend
+# with it. That is why audio-cpp-darwin.sh copies THIS directory instead of
+# rebuilding one. The Darwin package needs the same root-level layout as the
+# Linux one: grpc-server, run.sh, the ggml dylibs and assets/ in ONE directory,
+# with lib/ for the rest. run.sh's Darwin branch execs grpc-server directly, so
+# _NSGetExecutablePath already names the package root; nothing else is needed
+# beyond putting the files there.
+UNAME_S=$(uname -s)
+if [ "$UNAME_S" = "Darwin" ]; then
+    echo "package.sh: Darwin dylib bundling is deferred to scripts/build/audio-cpp-darwin.sh"
+    ls -lah "$PACKAGE_DIR/" "$PACKAGE_DIR/assets/"
+    exit 0
+fi
+
+# The loader goes in the package ROOT, not in lib/. run.sh explains why at
+# length; the short version is that exec'ing it makes dirname(/proc/self/exe)
+# the directory it sits in, and both the ggml backend scan and the bundled:
+# asset lookup need that to be the package root.
+if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
+    cp -arfLv /lib64/ld-linux-x86-64.so.2 "$PACKAGE_DIR/ld.so"
+elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
+    cp -arfLv /lib/ld-linux-aarch64.so.1 "$PACKAGE_DIR/ld.so"
+else
+    echo "package.sh: unknown architecture" >&2
+    exit 1
+fi
+
+# THE LAYOUT ASSERTION. Everything else in this script checks that the package
+# can LINK. This checks that it can RESOLVE, which is a different property and
+# the one with no other guard on it.
+#
+# The loader, assets/ and the dlopened ggml objects only agree while they share
+# one directory, because run.sh execs the loader and all three are reached
+# through dirname(/proc/self/exe). A tidy-up that moves the loader into lib/,
+# following the llama-cpp layout, produces a package that builds, ships, and
+# then fails at run time with "model path does not exist: <pkg>/lib/assets/..."
+# or "Failed to initialize CPU backend". Fail the build instead.
+#
+# This sits immediately after the loader copy rather than at the end of the
+# script on purpose: everything below dereferences $PACKAGE_DIR/ld.so, so a
+# misplaced loader would otherwise surface as "No such file or directory" from
+# the validation gate and never reach an assertion that could explain it.
+if [ ! -f "$PACKAGE_DIR/ld.so" ]; then
+    echo "package.sh: the bundled loader must be at the package root, not in lib/." >&2
+    echo "package.sh: run.sh execs it, so its directory is dirname(/proc/self/exe)," >&2
+    echo "package.sh: which is where resolve_model_path looks for assets/ and where" >&2
+    echo "package.sh: ggml looks for the CPU variants." >&2
+    exit 1
+fi
+if [ ! -d "$PACKAGE_DIR/assets" ]; then
+    echo "package.sh: assets/ must sit beside the loader at the package root." >&2
+    exit 1
+fi
+# Only assert the ggml half when this build produced CPU variants at all: a
+# cublas or vulkan build links ggml statically and ships none.
+if compgen -G "$BUILD_DIR/bin/libggml-cpu-*.so" > /dev/null && \
+   ! compgen -G "$PACKAGE_DIR/libggml-cpu-*.so" > /dev/null; then
+    echo "package.sh: the build produced libggml-cpu-*.so but none reached the" >&2
+    echo "package.sh: package root, so ggml's scan of dirname(/proc/self/exe)" >&2
+    echo "package.sh: will find no CPU backend." >&2
+    exit 1
+fi
+
+# Libraries the host GPU driver stack owns. package_gpu_libs deliberately ships
+# the CUDA/Vulkan runtime but not the driver, because the driver has to match
+# the kernel module on whatever host runs the image. Copying the build host's
+# copy in would pin it to the build host instead.
+#
+# One regex, used by both the copy loop and the validation gate below. They have
+# to agree: exempting a library from the copy but not from the gate makes the
+# gate reject the very absence the copy loop just created.
+DRIVER_LIB_RE='^(libcuda\.so|libnvidia-)'
+# awk applies string-escape processing to a -v assignment before compiling the
+# regex, so a lone backslash is eaten and awk warns about it. Double them here
+# rather than keeping a second hand-written copy of the pattern, which is the
+# drift this single-source-of-truth exists to prevent.
+DRIVER_LIB_RE_AWK=${DRIVER_LIB_RE//\\/\\\\}
+is_driver_lib() {
+    [[ "$(basename "$1")" =~ $DRIVER_LIB_RE ]]
+}
+
+# Bundle the full dependency closure. grpc-server links the distro gRPC,
+# protobuf and absl stack; copying only the C/C++ runtime leaves the scratch
+# image unable to start. The walk runs over the PACKAGED binary, not the one in
+# $CURDIR, because its RUNPATH is $ORIGIN: only from inside the package does
+# libggml.so.0 resolve to the copy shipped above rather than to nothing.
+# The dlopened ggml objects are walked too, since a dependency of theirs that
+# grpc-server does not itself link would otherwise be missed.
+{
+    ldd "$PACKAGE_DIR/grpc-server"
+    for so in "$PACKAGE_DIR"/*.so*; do
+        [ -f "$so" ] || continue
+        ldd "$so"
+    done
+} | awk '$2 == "=>" && $3 ~ /^\// { print $3 }' | sort -u | \
+while read -r so; do
+    # Skip what is already inside the package: the ggml objects resolve through
+    # $ORIGIN and re-copying them into lib/ would ship two copies of each.
+    case "$so" in "$PACKAGE_DIR"/*) continue ;; esac
+    if is_driver_lib "$so"; then
+        echo "package.sh: leaving driver-owned library to the host: $so"
+        continue
+    fi
+    cp -arfLv "$so" "$PACKAGE_DIR/lib/"
+done
+
+GPU_LIB_SCRIPT="${REPO_ROOT}/scripts/build/package-gpu-libs.sh"
+if [ -f "$GPU_LIB_SCRIPT" ]; then
+    echo "Packaging GPU libraries for BUILD_TYPE=${BUILD_TYPE:-cpu}..."
+    # shellcheck source=/dev/null
+    source "$GPU_LIB_SCRIPT" "$PACKAGE_DIR/lib"
+    package_gpu_libs
+fi
+
+# Resolve every dependency through the same loader and library path the
+# from-scratch image uses. Two distinct failures are rejected, because the
+# loader can still fall back to the host's default directories: a dependency it
+# could not resolve at all, and one it resolved to a file OUTSIDE the package,
+# which would validate here and be absent in the image.
+#
+# The driver libraries are exempt from BOTH rejections, and that exemption is
+# load-bearing on GPU builds rather than tidiness. With BUILD_TYPE=cublas ggml
+# is static (no CPU_ALL_VARIANTS), and ggml/CMakeLists.txt defaults
+# GGML_CUDA_NO_VMM=OFF, so ggml-cuda links CUDA::cuda_driver and grpc-server
+# itself carries DT_NEEDED libcuda.so.1. The copy loop above deliberately leaves
+# that to the host, so inside the CUDA builder it resolves either to a host path
+# or to nothing. Without this exemption every cublas build would fail here and
+# CI would produce no image at all.
+#
+# LD_TRACE_LOADED_OBJECTS + LD_LIBRARY_PATH, NOT `ld.so --library-path --list`,
+# and the difference is not cosmetic. Measured on a stub object built to
+# DT_NEEDED an absent libcuda.so.1: `--list` refuses to trace at all, printing
+# "libdrivertest.so: error while loading shared libraries: libcuda.so.1: cannot
+# open shared object file" and exiting 127, so no per-library line is ever
+# produced and no exemption below could apply. The env form prints
+# "libcuda.so.1 => not found" and exits 0, which is what makes both the
+# unresolved rule and its driver exemption reachable. It is also closer to what
+# run.sh actually does, since run.sh exports LD_LIBRARY_PATH rather than passing
+# --library-path.
+validation_failed=0
+validate_object() {
+    local object="$1"
+    LD_TRACE_LOADED_OBJECTS=1 LD_LIBRARY_PATH="$PACKAGE_DIR/lib:$PACKAGE_DIR" \
+        "$PACKAGE_DIR/ld.so" "$object" | awk -v pkg="$PACKAGE_DIR/" -v obj="$object" \
+                                             -v driver_re="$DRIVER_LIB_RE_AWK" '
+        function base(p,   n, parts) { n = split(p, parts, "/"); return parts[n] }
+        $2 == "=>" && $3 == "not" {
+            if ($1 ~ driver_re) next
+            print "package.sh: unresolved dependency of " obj ": " $1 > "/dev/stderr"
+            bad = 1
+        }
+        $2 == "=>" && $3 ~ /^\// && index($3, pkg) != 1 {
+            if (base($3) ~ driver_re) next
+            print "package.sh: dependency of " obj " resolved outside the package: " $0 > "/dev/stderr"
+            bad = 1
+        }
+        END { exit bad }
+    '
+}
+
+validate_object "$PACKAGE_DIR/grpc-server" || validation_failed=1
+for so in "$PACKAGE_DIR"/*.so*; do
+    [ -f "$so" ] || continue
+    validate_object "$so" || validation_failed=1
+done
+if [ "$validation_failed" -ne 0 ]; then
+    exit 1
+fi
+
+echo "audio-cpp package contents:"
+ls -lah "$PACKAGE_DIR/" "$PACKAGE_DIR/lib/" "$PACKAGE_DIR/assets/"

backend/cpp/audio-cpp/result_map.cpp

@@ -0,0 +1,85 @@
+#include "result_map.h"
+
+#include "transcript_assembly.h"
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+void fill_transcript_result(const engine::runtime::TaskResult &result,
+                            int sample_rate, float duration_seconds,
+                            backend::TranscriptResult *out) {
+    // No null guard on `out`, deliberately. gRPC always hands a handler a
+    // response message, so a null here would be a programming error in a
+    // caller, and a guard that returned quietly would answer the client with an
+    // untouched, empty transcript and an OK status. That is the same
+    // indistinguishable-from-silence failure the rest of this unit exists to
+    // prevent; crashing on the developer's machine is the cheaper outcome.
+    std::vector<Span> speech_segments;
+    speech_segments.reserve(result.speech_segments.size());
+    for (const auto &segment : result.speech_segments) {
+        speech_segments.push_back(
+            Span{segment.span.start_sample, segment.span.end_sample});
+    }
+
+    std::vector<SpeakerSpan> speaker_turns;
+    speaker_turns.reserve(result.speaker_turns.size());
+    for (const auto &turn : result.speaker_turns) {
+        speaker_turns.push_back(
+            SpeakerSpan{Span{turn.span.start_sample, turn.span.end_sample},
+                        turn.speaker_id});
+    }
+
+    std::vector<WordSpan> words;
+    words.reserve(result.word_timestamps.size());
+    for (const auto &word : result.word_timestamps) {
+        words.push_back(
+            WordSpan{Span{word.span.start_sample, word.span.end_sample},
+                     word.word});
+    }
+
+    // The ONLY read of transcript text in this function, and the only one there
+    // may ever be. See THE RULE in the header.
+    const std::string text =
+        result.text_output.has_value() ? result.text_output->text : std::string();
+
+    const AssembledTranscript assembled = assemble_transcript(
+        text, speech_segments, speaker_turns, words, sample_rate);
+
+    out->set_text(assembled.text);
+    // language has no source inside transcript_assembly, which is span-shaped
+    // only, so it is read straight off the engine result here. Left untouched
+    // when the family reported no text output at all: an empty string would be
+    // indistinguishable from a family that genuinely detected no language, and
+    // the field is documented as optional.
+    if (result.text_output.has_value()) {
+        out->set_language(result.text_output->language);
+    }
+    out->set_duration(duration_seconds);
+
+    // Cleared rather than appended to. A caller that fills the same message
+    // twice (a stream's final_result being rebuilt, say) would otherwise emit
+    // every segment twice, and the second call's ids would restart at 0 and
+    // collide with the first call's.
+    out->clear_segments();
+    for (const auto &segment : assembled.segments) {
+        auto *out_segment = out->add_segments();
+        out_segment->set_id(segment.id);
+        // NANOSECONDS. TranscriptSegment and TranscriptWord are the only
+        // messages in backend.proto that use them; VADSegment and DiarizeSegment
+        // are float seconds. assemble_transcript has already converted.
+        out_segment->set_start(segment.start_ns);
+        out_segment->set_end(segment.end_ns);
+        out_segment->set_text(segment.text);
+        out_segment->set_speaker(segment.speaker);
+        for (const auto &word : segment.words) {
+            auto *out_word = out_segment->add_words();
+            out_word->set_start(word.start_ns);
+            out_word->set_end(word.end_ns);
+            out_word->set_text(word.text);
+        }
+    }
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/result_map.h

@@ -0,0 +1,37 @@
+#pragma once
+
+// Converts engine::runtime results into LocalAI proto messages. All of the
+// non-trivial shaping lives in transcript_assembly, which is stdlib-only and
+// unit tested; this unit is the thin engine-typed boundary around it.
+
+#include "backend.pb.h"
+
+#include "engine/framework/runtime/session.h"
+
+namespace audiocpp_backend {
+
+// Fills text, language, duration, segments and per-segment words.
+//
+// THE RULE: the top-level text is TaskResult.text_output verbatim. It is never
+// derived from segments or words. audio.cpp carries transcript text in
+// text_output and nowhere else: speech_segments, speaker_turns and
+// word_timestamps carry spans and labels and no text at all. Deriving the
+// transcript from them therefore returns an EMPTY text for every producer that
+// reports segments without word timing, which real VibeVoice diarized ASR does.
+// An earlier attempt at this backend shipped exactly that bug. assemble_transcript
+// enforces the rule and is heavily tested; this unit's job is not to re-derive
+// it but to not undo it at the proto boundary.
+//
+// `sample_rate` is the rate the result's spans are expressed in, which is the
+// rate of the AudioBuffer that was handed to the session, NOT the rate of the
+// file the caller uploaded. Those differ whenever read_audio_file resampled,
+// which is why the handler passes the buffer's rate rather than the file's.
+//
+// Segments are replaced, not appended to, so a message filled twice does not
+// accumulate. `out` must be non-null and is not checked; see the note at the
+// top of the implementation for why that is not an oversight.
+void fill_transcript_result(const engine::runtime::TaskResult &result,
+                            int sample_rate, float duration_seconds,
+                            backend::TranscriptResult *out);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/result_map_ctest.cpp

@@ -0,0 +1,247 @@
+// Tests for result_map, the engine-to-proto boundary.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE. backend/cpp/run-unit-tests.sh globs
+// every *_test.cpp under backend/cpp/ and compiles it as a single standalone
+// translation unit with no include path beyond its own directory. This file
+// needs backend.pb.h and the audio.cpp framework headers, so it is built and
+// run by ctest instead:
+//
+//   make -C backend/cpp/audio-cpp test-engine
+//
+// Renaming it to *_test.cpp would break the standalone suite for every backend.
+//
+// What is worth testing here is exactly one thing, and it is not the field
+// copying: THE RULE. TaskResult carries transcript text in text_output and
+// nowhere else, so the proto's text must be that string verbatim. An earlier
+// attempt at this backend derived it from the segments, which returns an empty
+// transcript for every producer that reports segments without word timing.
+// transcript_assembly already enforces the rule and is tested on its own; these
+// checks are here so that a future edit cannot undo it at the boundary.
+
+#include "result_map.h"
+
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+namespace rt = engine::runtime;
+
+static const int kRate = 16000;
+
+static rt::SpeechSegment speech(std::int64_t start, std::int64_t end) {
+    rt::SpeechSegment segment;
+    segment.span.start_sample = start;
+    segment.span.end_sample = end;
+    return segment;
+}
+
+static rt::SpeakerTurn turn(std::int64_t start, std::int64_t end,
+                            const std::string &speaker) {
+    rt::SpeakerTurn out;
+    out.span.start_sample = start;
+    out.span.end_sample = end;
+    out.speaker_id = speaker;
+    return out;
+}
+
+static rt::WordTimestamp word(std::int64_t start, std::int64_t end,
+                              const std::string &text) {
+    rt::WordTimestamp out;
+    out.span.start_sample = start;
+    out.span.end_sample = end;
+    out.word = text;
+    return out;
+}
+
+// THE REGRESSION. A diarized ASR result: real text, real speaker turns, and no
+// word timing at all. This is the vibevoice_asr shape, and it is the one that
+// came back empty before.
+static void test_text_survives_segments_without_words() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "hello there general kenobi";
+    transcript.language = "en";
+    result.text_output = transcript;
+    result.speaker_turns.push_back(turn(0, 16000, "speaker_0"));
+    result.speaker_turns.push_back(turn(16000, 32000, "speaker_1"));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 2.0f, &out);
+
+    check(out.text() == "hello there general kenobi",
+          "diarized result keeps text_output verbatim");
+    check(out.language() == "en", "language comes from text_output");
+    check(out.segments_size() == 2, "both speaker turns become segments");
+    if (out.segments_size() == 2) {
+        check(out.segments(0).speaker() == "speaker_0",
+              "first segment keeps its own speaker label");
+        check(out.segments(1).speaker() == "speaker_1",
+              "second segment keeps its own speaker label");
+        check(out.segments(1).start() == 1000000000LL,
+              "segment start is nanoseconds, not samples");
+        check(out.segments(1).end() == 2000000000LL,
+              "segment end is nanoseconds, not samples");
+    }
+}
+
+// The same rule seen from the other side: text present, spans present, and the
+// per-segment text empty because there is nothing truthful to split. A boundary
+// that derived the top-level text from these segments would produce "".
+static void test_speech_segments_do_not_supply_the_text() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "one two three";
+    result.text_output = transcript;
+    result.speech_segments.push_back(speech(0, 8000));
+    result.speech_segments.push_back(speech(8000, 16000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text() == "one two three",
+          "speech segments without words do not empty the transcript");
+    check(out.segments_size() == 2, "both speech segments are emitted");
+    if (out.segments_size() == 2) {
+        check(out.segments(0).text().empty() && out.segments(1).text().empty(),
+              "per-segment text stays empty when there is no word timing");
+    }
+}
+
+static void test_words_reach_the_proto_in_nanoseconds() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "hi there";
+    result.text_output = transcript;
+    result.word_timestamps.push_back(word(0, 8000, "hi"));
+    result.word_timestamps.push_back(word(8000, 16000, "there"));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text() == "hi there", "word-timed result keeps text_output");
+    check(out.segments_size() == 1, "words with no spans yield one covering segment");
+    if (out.segments_size() == 1) {
+        const auto &segment = out.segments(0);
+        check(segment.words_size() == 2, "both words are emitted");
+        if (segment.words_size() == 2) {
+            check(segment.words(0).text() == "hi", "first word text");
+            check(segment.words(0).start() == 0, "first word start");
+            check(segment.words(0).end() == 500000000LL,
+                  "first word end is 0.5 s in nanoseconds");
+            check(segment.words(1).start() == 500000000LL, "second word start");
+            check(segment.words(1).end() == 1000000000LL, "second word end");
+        }
+    }
+}
+
+// The buffer's rate, not the file's, is what the spans mean. Passing 8000 for
+// the same spans has to halve every timestamp, which is what makes resampling
+// the input at read time load-bearing rather than cosmetic.
+static void test_sample_rate_scales_the_timestamps() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "x";
+    result.text_output = transcript;
+    result.speech_segments.push_back(speech(0, 8000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, 8000, 1.0f, &out);
+
+    check(out.segments_size() == 1, "one segment at 8 kHz");
+    if (out.segments_size() == 1) {
+        check(out.segments(0).end() == 1000000000LL,
+              "8000 samples at 8 kHz is one second");
+    }
+}
+
+static void test_duration_is_carried_through() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "x";
+    result.text_output = transcript;
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 14.07f, &out);
+
+    check(out.duration() > 14.06f && out.duration() < 14.08f,
+          "duration is set from the argument");
+}
+
+// No text output at all. A VAD-shaped result reaching this boundary must not
+// invent a transcript, and must not overwrite a language the caller had already
+// decided on.
+static void test_missing_text_output_leaves_language_alone() {
+    rt::TaskResult result;
+    result.speech_segments.push_back(speech(0, 16000));
+
+    backend::TranscriptResult out;
+    out.set_language("it");
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text().empty(), "no text_output means no text");
+    check(out.language() == "it",
+          "a result with no text_output does not clear the language");
+    check(out.segments_size() == 1, "spans are still emitted");
+}
+
+// Filling the same message twice must replace, not accumulate: the second
+// call's ids restart at 0 and would collide with the first call's.
+static void test_refilling_replaces_the_segments() {
+    rt::TaskResult first;
+    rt::Transcript transcript;
+    transcript.text = "first";
+    first.text_output = transcript;
+    first.speech_segments.push_back(speech(0, 16000));
+    first.speech_segments.push_back(speech(16000, 32000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(first, kRate, 2.0f, &out);
+
+    rt::TaskResult second;
+    rt::Transcript replacement;
+    replacement.text = "second";
+    second.text_output = replacement;
+    second.speech_segments.push_back(speech(0, 16000));
+    fill_transcript_result(second, kRate, 1.0f, &out);
+
+    check(out.text() == "second", "the second fill replaces the text");
+    check(out.segments_size() == 1,
+          "the second fill replaces the segments instead of appending");
+}
+
+static void test_empty_result_is_empty() {
+    rt::TaskResult result;
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 0.0f, &out);
+
+    check(out.text().empty(), "empty result has no text");
+    check(out.segments_size() == 0, "empty result has no segments");
+}
+
+int main() {
+    test_text_survives_segments_without_words();
+    test_speech_segments_do_not_supply_the_text();
+    test_words_reach_the_proto_in_nanoseconds();
+    test_sample_rate_scales_the_timestamps();
+    test_duration_is_carried_through();
+    test_missing_text_output_leaves_language_alone();
+    test_refilling_replaces_the_segments();
+    test_empty_result_is_empty();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all result_map checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/run.sh

@@ -0,0 +1,58 @@
+#!/bin/bash
+# Entry point for the audio-cpp backend image and for BACKEND_BINARY mode.
+#
+# The image's final stage is FROM scratch, so the package root is / and there is
+# no system loader, no system libc and no fallback library path. Everything the
+# process opens has to be inside the package, and it has to be findable by the
+# two mechanisms that actually do the finding: the dynamic linker, and
+# audio.cpp's own directory scans.
+set -e
+
+CURDIR=$(dirname "$(realpath "$0")")
+
+if [ "$(uname -s)" = "Darwin" ]; then
+	export DYLD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$DYLD_LIBRARY_PATH"
+	exec "$CURDIR/grpc-server" "$@"
+fi
+
+# $CURDIR is on the path as well as $CURDIR/lib: the ggml shared objects the
+# CPU-all-variants build produces sit in the package root, next to the binary,
+# not in lib/. See the comment below for why they cannot live in lib/.
+export LD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$LD_LIBRARY_PATH"
+
+# THE BUNDLED LOADER IS AT THE PACKAGE ROOT, NOT AT lib/ld.so. DO NOT MOVE IT.
+#
+# Exec'ing the loader is what pins the bundled glibc to the matching ld.so, and
+# every other C++ backend here does it. The cost is that /proc/self/exe then
+# names the LOADER rather than grpc-server, and this backend has two consumers
+# of /proc/self/exe that both have to land on the package root:
+#
+#   - ggml's backend registry DISCOVERS the per-microarch libggml-cpu-*.so by
+#     scanning dirname(/proc/self/exe) and the current directory. Those files
+#     are dlopened, never linked, so no library path and no RUNPATH reaches
+#     them: they have to be in a directory ggml scans.
+#   - resolve_model_path() turns "bundled:<name>" into
+#     dirname(/proc/self/exe)/assets/<name>, which is how the bundled
+#     silero_vad and marblenet_vad models resolve with nothing downloaded.
+#
+# backend/cpp/llama-cpp/package.sh answers the first of these by keeping
+# lib/ld.so and moving the ggml objects INTO lib/. That does not generalise
+# here, because it would also drag assets/ into lib/ to keep the second
+# consumer working. Putting the loader in the package root instead makes
+# dirname(/proc/self/exe) the package root, so the binary, the ggml objects and
+# assets/ all sit in the one directory that all three mechanisms agree on.
+#
+# The ggml half has a second chance that the bundled: half does not: LocalAI
+# sets the backend process cwd to the directory holding run.sh
+# (pkg/model/process.go), so ggml's fs::current_path() fallback would find the
+# objects in normal operation whatever the loader's placement. That fallback is
+# worth little here. It holds only for the launcher that sets that cwd, it is
+# gone the moment anyone runs the binary by hand or through a wrapper that
+# chdirs, and resolve_model_path has no equivalent, which would leave the only
+# zero-download path in this backend resting on it. Rooting the loader is the
+# one layout where all three mechanisms agree without depending on the cwd.
+if [ -f "$CURDIR/ld.so" ]; then
+	exec "$CURDIR/ld.so" "$CURDIR/grpc-server" "$@"
+fi
+
+exec "$CURDIR/grpc-server" "$@"

backend/cpp/audio-cpp/stem_selection.cpp

@@ -0,0 +1,129 @@
+#include "stem_selection.h"
+
+#include <cstddef>
+#include <filesystem>
+#include <string>
+
+namespace audiocpp_backend {
+namespace {
+
+// The stem every separation family this backend can reach names its lead vocal
+// track, and the one a caller who names no stem almost always wants: it is what
+// the OpenAI-shaped "isolate the voice" request means. htdemucs (drums, bass,
+// other, vocals) and mel_band_roformer (vocals, instrumental) both have it, and
+// in htdemucs's case it is NOT the first output, which is the whole reason this
+// preference is written down rather than left as "take index 0".
+const char *const kPreferredStem = "vocals";
+
+std::string join_names(const std::vector<std::string> &names) {
+    std::string out;
+    for (const auto &name : names) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += name;
+    }
+    return out;
+}
+
+// Whether a model-supplied stem name can be used as one component of a file
+// name. Deliberately a whitelist of refusals rather than a sanitiser: silently
+// rewriting "vo/cals" to "cals" would make the file the caller receives
+// disagree with the name they would have to ask for.
+bool name_is_writable(const std::string &name) {
+    if (name.empty() || name == "." || name == "..") {
+        return false;
+    }
+    // Control bytes, NUL above all. GGUF strings are length prefixed and demucs
+    // reads its source names out of JSON, which can encode one, so a
+    // std::string holding an embedded NUL survives all the way here. Two such
+    // names differing only AFTER the NUL are distinct std::strings, so the
+    // duplicate check below waves them through, and then path::c_str()
+    // truncates both at the NUL and they open the same file: precisely the
+    // silent overwrite the duplicate check exists to prevent, with the ".wav"
+    // stripped off as well. The rest of the range goes with it, since a newline
+    // or an escape sequence in a file name is a terminal and log injection
+    // nuisance with no legitimate use.
+    for (const char byte : name) {
+        const auto value = static_cast<unsigned char>(byte);
+        if (value < 0x20 || value == 0x7f) {
+            return false;
+        }
+    }
+    // Both separators, not just the host's. A GGUF is a downloaded file and its
+    // strings are not this host's to trust, so a name written on Windows must
+    // not become a directory traversal wherever the check happens to run.
+    return name.find('/') == std::string::npos &&
+           name.find('\\') == std::string::npos;
+}
+
+} // namespace
+
+StemChoice select_named_output(const std::vector<std::string> &names,
+                               const std::string &requested) {
+    StemChoice choice;
+    if (names.empty()) {
+        // Not an error here. The caller distinguishes "this family produces one
+        // unnamed output" from "this family produced nothing", and only it can
+        // tell them apart.
+        return choice;
+    }
+
+    // Every name is checked, not merely the selected one, because every stem is
+    // written. A bad name in the fourth output would otherwise be discovered
+    // only after three files had already been created.
+    for (std::size_t i = 0; i < names.size(); ++i) {
+        if (!name_is_writable(names[i])) {
+            choice.error = "audio-cpp: this model names an output stem '" +
+                           names[i] +
+                           "' that cannot be used as a file name; stems: " +
+                           join_names(names);
+            return choice;
+        }
+        for (std::size_t seen = 0; seen < i; ++seen) {
+            if (names[seen] == names[i]) {
+                choice.error =
+                    "audio-cpp: this model produces two output stems both named '" +
+                    names[i] + "'; one would silently overwrite the other";
+                return choice;
+            }
+        }
+    }
+
+    if (!requested.empty()) {
+        for (std::size_t i = 0; i < names.size(); ++i) {
+            if (names[i] == requested) {
+                choice.index = static_cast<int>(i);
+                return choice;
+            }
+        }
+        choice.error = "audio-cpp: no stem named '" + requested +
+                       "' in this model's output; available stems: " +
+                       join_names(names);
+        return choice;
+    }
+
+    for (std::size_t i = 0; i < names.size(); ++i) {
+        if (names[i] == kPreferredStem) {
+            choice.index = static_cast<int>(i);
+            return choice;
+        }
+    }
+    choice.index = 0;
+    return choice;
+}
+
+std::string sibling_stem_path(const std::string &dst, const std::string &name) {
+    const std::filesystem::path path(dst);
+    // The dst extension is reused rather than forced to ".wav" so the siblings
+    // look like the file the caller named. write_audio_file writes WAV bytes
+    // whatever the extension says, for dst as much as for the siblings, so this
+    // keeps the set consistent instead of making the siblings honest about a
+    // format dst is already lying about.
+    const std::string extension =
+        path.has_extension() ? path.extension().string() : std::string(".wav");
+    return (path.parent_path() / (path.stem().string() + "." + name + extension))
+        .string();
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stem_selection.h

@@ -0,0 +1,63 @@
+#pragma once
+
+// Decides which of a separation model's named stems the AudioTransform response
+// carries in `dst`, and names the sibling files every other stem is written to.
+//
+// It exists because AudioTransformResult carries ONE dst while htdemucs and
+// mel_band_roformer produce several named outputs from a single run. Running
+// once per stem would cost four full inferences for a four stem model, so the
+// handler runs once, writes every stem beside dst, and puts the selected one in
+// dst itself.
+//
+// Standard library only, so backend/cpp/run-unit-tests.sh compiles and runs its
+// test without an audio.cpp checkout. grpc-server.cpp flattens the engine's
+// NamedAudioBuffer list into the plain name vector taken here; nothing in this
+// unit knows about engine::runtime.
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+struct StemChoice {
+    // Index into the `names` vector. -1 means nothing was chosen, which happens
+    // for an empty list (the family produced a single unnamed output) and for
+    // every refusal.
+    //
+    // THE CONTRACT THE CALLER INDEXES ON: when `error` is empty and `names` was
+    // not, this is always a valid index into `names`. It is never -1 in that
+    // case, so a caller that checks `error` first can index without a further
+    // guard, and a caller that does not check `error` first would index with a
+    // negative value. Check the error.
+    int index = -1;
+    // Non-empty when the request must be refused, and suitable verbatim as an
+    // INVALID_ARGUMENT message. Two things land here: the caller named a stem
+    // this model does not produce, and the model named stems that cannot both
+    // be written (an unusable file name, or two stems sharing one).
+    std::string error;
+};
+
+// Picks the stem that goes to dst. Preference order: an explicit `requested`,
+// then "vocals", then the first output.
+//
+// An explicit but unknown `requested` is an ERROR rather than a fallback. A
+// caller who asks for "drums" and silently receives "vocals" gets a 200 and a
+// wrong file, which is the failure mode nobody can see; the message therefore
+// lists the stem names this model really has.
+//
+// The names are also validated, because they come from the MODEL (htdemucs
+// reads them from the GGUF's config.sources) and each one becomes a component
+// of a file path this backend writes. A name carrying a path separator would
+// write outside the caller's output directory, and two stems sharing a name
+// would silently overwrite each other. Both are refused before anything is
+// written, which is also why selection has to happen before the first write
+// rather than after the loop: a refused request must leave no files behind.
+StemChoice select_named_output(const std::vector<std::string> &names,
+                               const std::string &requested);
+
+// "/generated/transform-1.wav" + "drums" -> "/generated/transform-1.drums.wav".
+// A dst with no extension gets ".wav", since that is what write_audio_file
+// produces whatever the caller called the file.
+std::string sibling_stem_path(const std::string &dst, const std::string &name);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stem_selection_test.cpp

@@ -0,0 +1,250 @@
+// Unit tests for stem_selection. Standard library only. The harness
+// (backend/cpp/run-unit-tests.sh) compiles this as a single translation unit,
+// so the implementation is included directly.
+//
+// What is actually at stake here: AudioTransformResult carries one dst, a
+// separation model produces several stems, and the caller cannot see which one
+// they got. Every check below is about a wrong file arriving with a 200.
+
+#include "stem_selection.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_equal(const std::string &got, const std::string &want,
+                        const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\", want \"" + want + "\")");
+}
+
+using namespace audiocpp_backend;
+
+// htdemucs's real source order, taken from its GGUF config.sources. vocals is
+// LAST, which is why "first output" is not the default.
+static const std::vector<std::string> kDemucs = {"drums", "bass", "other",
+                                                 "vocals"};
+// mel_band_roformer's, where vocals is first.
+static const std::vector<std::string> kRoformer = {"vocals", "instrumental"};
+
+static void test_default_selection() {
+    const auto demucs = select_named_output(kDemucs, "");
+    check(demucs.error.empty(), "an unrequested selection is not an error");
+    check(demucs.index == 3, "no stem asked for picks vocals, not the first output");
+
+    const auto roformer = select_named_output(kRoformer, "");
+    check(roformer.index == 0, "vocals is picked when it is already first");
+
+    // No vocals anywhere: the first output is the documented fallback.
+    const auto novocals = select_named_output({"accompaniment", "drums"}, "");
+    check(novocals.index == 0 && novocals.error.empty(),
+          "a family with no vocals stem falls back to the first output");
+
+    // Substring matches must not count: "vocals_2" is a different stem.
+    const auto near = select_named_output({"vocals_2", "backing"}, "");
+    check(near.index == 0 && near.error.empty(),
+          "'vocals_2' is not 'vocals', so the fallback and not the preference applies");
+    const auto near_second = select_named_output({"backing", "vocals_2"}, "");
+    check(near_second.index == 0,
+          "a near miss on the preferred name does not pull it to the front");
+}
+
+static void test_explicit_selection() {
+    for (int i = 0; i < 4; ++i) {
+        const auto choice = select_named_output(kDemucs, kDemucs[static_cast<size_t>(i)]);
+        check(choice.index == i && choice.error.empty(),
+              "explicit '" + kDemucs[static_cast<size_t>(i)] + "' selects its own index");
+    }
+    // Including the one the default would have chosen anyway: asking for it
+    // must not be treated as "no request".
+    const auto vocals = select_named_output(kDemucs, "vocals");
+    check(vocals.index == 3, "explicitly asking for vocals still selects vocals");
+}
+
+static void test_unknown_stem_is_refused() {
+    const auto choice = select_named_output(kDemucs, "kazoo");
+    check(choice.index == -1, "an unknown stem selects nothing");
+    check(!choice.error.empty(), "an unknown stem is refused rather than substituted");
+    check(choice.error.find("kazoo") != std::string::npos,
+          "the refusal names the stem that was asked for");
+    // The real names, so the caller can fix the request without guessing.
+    for (const auto &name : kDemucs) {
+        check(choice.error.find(name) != std::string::npos,
+              "the refusal lists the real stem '" + name + "'");
+    }
+    check(choice.error.find("drums, bass, other, vocals") != std::string::npos,
+          "the refusal lists the stems in the model's own order");
+
+    // Case matters: the engine's ids are exact, so a wrong case is a wrong name
+    // rather than a near miss to be forgiven.
+    const auto wrong_case = select_named_output(kDemucs, "Vocals");
+    check(wrong_case.index == -1 && !wrong_case.error.empty(),
+          "stem names are matched case sensitively");
+}
+
+static void test_no_named_outputs() {
+    const auto choice = select_named_output({}, "");
+    check(choice.index == -1, "an empty output list selects nothing");
+    check(choice.error.empty(),
+          "an empty output list is not an error here: the caller decides");
+    const auto requested = select_named_output({}, "vocals");
+    check(requested.index == -1 && requested.error.empty(),
+          "an empty output list stays the caller's decision even when a stem was asked for");
+}
+
+static void test_unwritable_names_are_refused() {
+    // Model-supplied names become file path components. A separator would write
+    // outside the caller's output directory.
+    const std::vector<std::string> traversal = {"vocals", "../../etc/passwd"};
+    const auto escaped = select_named_output(traversal, "vocals");
+    check(escaped.index == -1 && !escaped.error.empty(),
+          "a stem name containing a path separator is refused");
+    check(escaped.error.find("../../etc/passwd") != std::string::npos,
+          "the refusal names the offending stem");
+
+    check(!select_named_output({"vo\\cals", "drums"}, "").error.empty(),
+          "a backslash separator is refused too");
+    check(!select_named_output({"drums", ""}, "").error.empty(),
+          "an empty stem name is refused");
+    check(!select_named_output({"drums", "."}, "").error.empty(),
+          "a stem named '.' is refused");
+    check(!select_named_output({"drums", ".."}, "").error.empty(),
+          "a stem named '..' is refused");
+
+    // The check covers EVERY name, not only the selected one: all of them are
+    // written, so a bad fourth name must not be found after three files exist.
+    const auto late = select_named_output({"vocals", "drums", "bass", "a/b"}, "vocals");
+    check(late.index == -1 && !late.error.empty(),
+          "an unwritable name after the selected one still refuses the whole request");
+
+    // Control bytes, and the NUL case is why the whole range is refused. These
+    // two names are DIFFERENT std::strings, so the duplicate check does not
+    // fire, yet both truncate to "vocals" at path::c_str() and would open one
+    // file: the silent overwrite the duplicate check exists to prevent, with
+    // the ".wav" stripped off into the bargain.
+    const std::string nul_a("vocals\0drums", 12);
+    const std::string nul_b("vocals\0bass", 11);
+    check(nul_a != nul_b, "the two NUL names really are distinct std::strings");
+    check(std::string(nul_a.c_str()) == "vocals" &&
+              std::string(nul_b.c_str()) == "vocals",
+          "and both truncate to the same C string, which is the hazard");
+    const auto nul_pair = select_named_output({nul_a, nul_b}, "");
+    check(nul_pair.index == -1 && !nul_pair.error.empty(),
+          "two stem names differing only after an embedded NUL are refused");
+    check(!select_named_output({"drums", std::string("vo\0cals", 7)}, "").error.empty(),
+          "a single embedded NUL is refused on its own");
+    check(!select_named_output({"drums", "voc\nals"}, "").error.empty(),
+          "a newline in a stem name is refused");
+    check(!select_named_output({"drums", "voc\tals"}, "").error.empty(),
+          "a tab in a stem name is refused");
+    check(!select_named_output({"drums", "voc\033[31mals"}, "").error.empty(),
+          "an escape sequence in a stem name is refused");
+    check(!select_named_output({"drums", "voc\177als"}, "").error.empty(),
+          "DEL in a stem name is refused");
+
+    // The boundary below the refused range is the space, which is an ordinary
+    // file name character and must stay usable, or this check would be
+    // refusing real stem names.
+    const auto spaced = select_named_output({"lead vocals", "drums"}, "lead vocals");
+    check(spaced.index == 0 && spaced.error.empty(),
+          "a space is not a control character and stays usable");
+    // And every byte above DEL: a UTF-8 stem name is ordinary, and signed char
+    // would make those bytes compare as negative.
+    const auto utf8 = select_named_output({"vocals", "b\xc3\xa4sse"}, "b\xc3\xa4sse");
+    check(utf8.index == 1 && utf8.error.empty(),
+          "a UTF-8 stem name is not mistaken for a control character");
+
+    // A leading dot is not a traversal and must stay usable.
+    const auto dotted = select_named_output({".vocals", "drums"}, ".vocals");
+    check(dotted.index == 0 && dotted.error.empty(),
+          "a leading dot in a stem name is allowed");
+}
+
+static void test_duplicate_names_are_refused() {
+    const auto choice = select_named_output({"vocals", "drums", "vocals"}, "vocals");
+    check(choice.index == -1 && !choice.error.empty(),
+          "two stems sharing a name are refused: one file would overwrite the other");
+    check(choice.error.find("vocals") != std::string::npos,
+          "the duplicate refusal names the repeated stem");
+}
+
+static void test_sibling_paths() {
+    check_equal(sibling_stem_path("/generated/transform-1.wav", "drums"),
+                "/generated/transform-1.drums.wav", "sibling beside an absolute dst");
+    check_equal(sibling_stem_path("sep.wav", "vocals"), "sep.vocals.wav",
+                "sibling of a bare file name has no directory");
+    check_equal(sibling_stem_path("/out/sep", "vocals"), "/out/sep.vocals.wav",
+                "an extensionless dst gets .wav");
+    check_equal(sibling_stem_path("/out/take.2.wav", "bass"), "/out/take.2.bass.wav",
+                "only the final extension is treated as the extension");
+    check_equal(sibling_stem_path("/out/sep.WAV", "bass"), "/out/sep.bass.WAV",
+                "the caller's extension spelling is preserved");
+    check_equal(sibling_stem_path("/a b/c d.wav", "other"), "/a b/c d.other.wav",
+                "spaces in the destination survive");
+
+    // The property that matters: no stem can ever be written over dst itself,
+    // or the "dst holds the selected stem" contract would depend on write order.
+    const std::string dst = "/out/sep.wav";
+    for (const auto &name : kDemucs) {
+        check(sibling_stem_path(dst, name) != dst,
+              "the sibling for '" + name + "' is not dst itself");
+    }
+    // Distinct stems must land in distinct files.
+    check(sibling_stem_path(dst, "drums") != sibling_stem_path(dst, "bass"),
+          "two stems get two different sibling paths");
+}
+
+// The contract grpc-server.cpp indexes on: an accepted choice over a non-empty
+// name list is always in range, so the handler needs no bounds guard of its own.
+// A -1 reaching the subscript would become a colossal size_t.
+static void test_accepted_index_is_always_in_range() {
+    const std::vector<std::vector<std::string>> lists = {
+        kDemucs, kRoformer, {"solo"}, {"accompaniment", "drums"}, {"a", "b", "c"}};
+    const std::vector<std::string> requests = {"", "vocals", "drums", "solo", "c",
+                                               "kazoo", "..", "a/b"};
+    for (const auto &names : lists) {
+        for (const auto &requested : requests) {
+            const auto choice = select_named_output(names, requested);
+            if (!choice.error.empty()) {
+                check(choice.index == -1,
+                      "a refusal never carries an index (request '" + requested + "')");
+                continue;
+            }
+            check(choice.index >= 0 &&
+                      choice.index < static_cast<int>(names.size()),
+                  "an accepted choice is in range (request '" + requested + "')");
+            // And the selected name is the one that was asked for, when one was.
+            if (!requested.empty()) {
+                check(names[static_cast<size_t>(choice.index)] == requested,
+                      "an accepted explicit request selects that exact name");
+            }
+        }
+    }
+}
+
+int main() {
+    test_accepted_index_is_always_in_range();
+    test_default_selection();
+    test_explicit_selection();
+    test_unknown_stem_is_refused();
+    test_no_named_outputs();
+    test_unwritable_names_are_refused();
+    test_duplicate_names_are_refused();
+    test_sibling_paths();
+    if (failures != 0) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all stem_selection checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/stream_delta.cpp

@@ -0,0 +1,204 @@
+#include "stream_delta.h"
+
+namespace audiocpp_backend {
+namespace {
+
+// True when `text` begins with `prefix`. An empty prefix matches everything,
+// which is what makes the first fragment take the cumulative branch and the
+// incremental branch alike: they agree there.
+bool starts_with(const std::string &text, const std::string &prefix) {
+    return text.size() >= prefix.size() &&
+           text.compare(0, prefix.size(), prefix) == 0;
+}
+
+// Number of leading bytes of `text` that cannot BEGIN a character, i.e. orphan
+// continuation bytes with no lead byte in front of them.
+//
+// They are unrecoverable rather than early: the byte that would have led them
+// has already gone past, and nothing can be prepended to a fragment after the
+// fact. Holding them would stall the stream for good, and emitting them puts
+// invalid UTF-8 on the wire, so the caller DROPS them. Losing a byte keeps the
+// stream alive; emitting one ends it, and takes the final_result still to come
+// with it.
+std::size_t utf8_orphan_prefix_length(const std::string &text) {
+    std::size_t index = 0;
+    while (index < text.size() &&
+           (static_cast<unsigned char>(text[index]) & 0xC0) == 0x80) {
+        ++index;
+    }
+    return index;
+}
+
+// Length of the longest prefix of `text` that does NOT end inside a multi-byte
+// UTF-8 sequence, i.e. the most that can go on the wire without splitting a
+// character in half.
+//
+// Only the TRAILING sequence is examined; the leading side is
+// utf8_orphan_prefix_length's job. Bytes in the MIDDLE are neither one's
+// business: a family that emitted a malformed sequence inside its own text
+// cannot be repaired here without deleting part of that transcript, and the
+// stream is lost anyway, because final_result.text carries the same bytes
+// through the same proto3 string field.
+//
+// Anything that can never complete is reported as complete, so it goes out
+// rather than being held forever: a lead byte the encoding does not define, and
+// a run of five or more continuation bytes, are both passed through. A tracker
+// that stalled on undecodable input would turn one bad byte into a permanently
+// silent stream, which is worse than the bad byte.
+std::size_t utf8_complete_prefix_length(const std::string &text) {
+    std::size_t index = text.size();
+    std::size_t continuations = 0;
+    while (index > 0 && continuations < 4) {
+        const auto byte = static_cast<unsigned char>(text[index - 1]);
+        if ((byte & 0xC0) == 0x80) {
+            --index;
+            ++continuations;
+            continue;
+        }
+        std::size_t needed = 1;
+        if ((byte & 0x80) == 0x00) {
+            needed = 1;
+        } else if ((byte & 0xE0) == 0xC0) {
+            needed = 2;
+        } else if ((byte & 0xF0) == 0xE0) {
+            needed = 3;
+        } else if ((byte & 0xF8) == 0xF0) {
+            needed = 4;
+        } else {
+            // Not a lead byte this encoding defines, so nothing is waiting on
+            // it and it must not be held.
+            needed = 1;
+        }
+        if (continuations + 1 >= needed) {
+            return text.size();
+        }
+        // The trailing sequence is short by at least one byte: cut before its
+        // lead byte and keep the rest for the next fragment.
+        return index - 1;
+    }
+    return text.size();
+}
+
+} // namespace
+
+std::string TranscriptDeltaTracker::release(const std::string &fragment) {
+    // Held-back bytes go in FRONT of whatever arrived next, or the character
+    // they begin is reassembled in the wrong order.
+    std::string candidate = pending_ + fragment;
+    // A fragment must not BEGIN mid-character either. pending_ always starts on
+    // a lead byte, so this only bites when nothing was held and the caller's
+    // rules dropped the lead byte somewhere upstream; it is the backstop that
+    // makes "no delta this class returns is ever invalid UTF-8" true of the
+    // FRONT as well as the back, independently of those rules being right.
+    candidate.erase(0, utf8_orphan_prefix_length(candidate));
+    const std::size_t cut = utf8_complete_prefix_length(candidate);
+    pending_ = candidate.substr(cut);
+    std::string emitted = candidate.substr(0, cut);
+    assembled_ += emitted;
+    return emitted;
+}
+
+std::string TranscriptDeltaTracker::observe(const std::string &partial_text) {
+    if (partial_text.empty()) {
+        return {};
+    }
+    // The comparisons run against everything KNOWN, delivered plus held back,
+    // rather than against the delivered text alone. Comparing against the
+    // delivered text would treat the held-back byte as new on the very next
+    // report and emit it twice.
+    const std::string known = assembled_ + pending_;
+    // An EXACT repeat, and nothing looser. This absorbs the duplicate delivery
+    // voxtral_realtime produces, which is the ONLY thing rule 2 was ever needed
+    // for: process_available_stream_chunks hands each event it produces to the
+    // sink from inside its loop and RETURNS the last of the batch
+    // (session.cpp:385-386), so that last event arrives twice with byte-equal
+    // text both times. A duplicate IS an exact repeat, so equality covers it.
+    //
+    // It used to discard any report the known text merely STARTED WITH, and that
+    // cost far more than it bought. Two separate defects came out of it, and
+    // both were found by randomized traces rather than by reading:
+    //
+    //   1. A short INCREMENTAL fragment that happens to be a byte prefix of the
+    //      transcript so far was read as a repeat and dropped, losing text with
+    //      a 200 and no diagnostic. Both incremental families emit fragments
+    //      that small routinely: nemotron_asr cuts at a byte offset
+    //      (decoder.cpp:550) and vibevoice_asr at a common prefix
+    //      (session.cpp:89-105). 9.50% of randomized pure-ASCII traces and
+    //      29.12% of French ones ended with a corrupted transcript.
+    //   2. When such a fragment was the LEAD BYTE of a multi-byte character, its
+    //      continuation bytes then arrived alone and began the next delta, which
+    //      is invalid UTF-8, which the Go runtime refuses to unmarshal, which
+    //      ends the stream and the final_result with it.
+    //
+    // What the narrowing gives up is the shrinking-hypothesis case: a cumulative
+    // report SHORTER than what is known is now read as an incremental fragment
+    // and duplicates those bytes at the client. No pinned family produces one.
+    // voxtral is the only cumulative reporter, and its hypothesis is
+    // tokenizer_.decode(streaming_token_ids_) over a vector that is only ever
+    // push_back'ed (session.cpp:436) and cleared by reset() (session.cpp:257),
+    // so within a stream it can only grow. Measured: narrowing this changed not
+    // one byte of 30,000 randomized cumulative traces.
+    //
+    // KEPT DELIBERATELY THOUGH NO TEST CAN SEE IT. Once narrowed to equality
+    // this rule became redundant with rule 3 below: an equal partial has an
+    // empty suffix, so rule 3 would call release("") and emit nothing either
+    // way. Deleting it is therefore an equivalent mutation, and the mutation
+    // harness reports it as a survivor, which is the honest result and not a
+    // gap in the tests. It stays for two reasons: it states the duplicate
+    // absorption where the citation for it lives, and it is independent of rule
+    // 3's condition. A future tightening of rule 3 to, say, require a STRICTLY
+    // longer partial would otherwise send every duplicate down the incremental
+    // branch and put the whole transcript on the wire a second time.
+    if (partial_text == known) {
+        return {};
+    }
+    if (starts_with(partial_text, known)) {
+        // Cumulative: the report is the whole transcript so far.
+        return release(partial_text.substr(known.size()));
+    }
+    // Incremental: the fragment is new text to append.
+    //
+    // A family that REWRITES its hypothesis lands here too, and the client's
+    // view is then wrong in a way nothing downstream can fix. nemotron's
+    // decoder has such a branch (decoder.cpp:552-554): when the new text is not
+    // an extension of what it already emitted, it emits the whole new text. So
+    // "the cat sat" followed by "the cat sap" leaves the client holding
+    // "the cat satthe cat sap", and reconcile then correctly refuses to append
+    // to a contradicted assembly, which leaves concat(deltas) != final with no
+    // signal on the wire. This is NOT repaired here, and the reason is that a
+    // delta stream has no retraction: emitting only the differing suffix would
+    // read as "sap" appended to "the cat sat", which is a different wrong
+    // answer, and emitting a correction would need a wire field that does not
+    // exist. final_result carries the authoritative text either way. It did not
+    // fire in a 331 delta run, because an RNN-T decode is monotonic in practice.
+    return release(partial_text);
+}
+
+std::string TranscriptDeltaTracker::reconcile(const std::string &final_text) {
+    if (final_text.empty() || final_text == assembled_) {
+        // Nothing further is owed. Held-back bytes are dropped rather than
+        // flushed: they are not in the authoritative text, so sending them
+        // would contradict it.
+        pending_.clear();
+        return {};
+    }
+    if (!starts_with(final_text, assembled_)) {
+        // Contradicted. Nothing sent can be taken back, so nothing more is
+        // sent; final_result carries the authoritative text.
+        pending_.clear();
+        return {};
+    }
+    // Compared against the DELIVERED text, so the fragment below already
+    // contains whatever was held back. pending_ is therefore cleared rather
+    // than prepended, or those bytes would go out twice.
+    //
+    // The fragment ends on a character boundary whenever final_text is
+    // well-formed, which is the normal case and the reason a held-back sequence
+    // is always flushed here. It is still cut, so a family handing back a final
+    // text that is itself truncated mid-character cannot put a partial sequence
+    // on the wire through this path either.
+    pending_.clear();
+    return release(final_text.substr(assembled_.size()));
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stream_delta.h

@@ -0,0 +1,133 @@
+#pragma once
+
+// Turns whatever a streaming session calls a "partial transcript" into the
+// incremental deltas AudioTranscriptionStream is contracted to send. Standard
+// library only, so it is tested without an audio.cpp checkout.
+//
+// THIS UNIT EXISTS BECAUSE THE FAMILIES DISAGREE, and the disagreement is
+// invisible at the interface: StreamEvent::partial_text is a Transcript either
+// way. Read out of the pinned upstream, one family at a time:
+//
+//   nemotron_asr      INCREMENTAL. decoder.cpp emits
+//                     current_text.substr(emitted_text.size()) per non-blank
+//                     token, and only through the stream event SINK, during
+//                     finalize(). process_audio_chunk returns empty events.
+//   vibevoice_asr     INCREMENTAL. process_audio_chunk returns
+//                     text.substr(common_prefix_size(...)); the sink is
+//                     deliberately swapped out around its internal run_single,
+//                     so the fragment arrives once, on the return value.
+//   higgs_audio_stt   INCREMENTAL. Same shape as vibevoice_asr.
+//   voxtral_realtime  CUMULATIVE. partial_text is
+//                     tokenizer_.decode(streaming_token_ids_), the whole
+//                     hypothesis so far. process_available_stream_chunks hands
+//                     every event it produces to the sink from INSIDE its loop
+//                     (session.cpp:385-386) and RETURNS only the last of the
+//                     batch, so the last event of each batch arrives twice and
+//                     the others arrive once.
+//
+// Applying either convention to the other family corrupts the transcript: read
+// a cumulative report as a delta and the client sees the transcript repeated on
+// every event; read an incremental fragment as cumulative and the suffix
+// arithmetic eats the front of it. So the tracker decides per fragment, from
+// what it has already delivered, and the one rule it enforces is that TEXT THE
+// CLIENT HAS ALREADY BEEN SENT IS NEVER SENT AGAIN.
+//
+// The cumulative reading is provably safe for voxtral, which is the family it
+// matters for: its decode is a pure concatenation of per-token byte strings
+// (tokenizer_text.cpp:171-183), so decode(ids[0..n]) is an unconditional BYTE
+// PREFIX of decode(ids[0..n+1]) and one of its reports can never be mistaken
+// for an incremental fragment.
+//
+// UTF-8 IS THE OTHER HALF OF THAT SAME FACT. Because that decode concatenates
+// raw token BYTES, a multi-byte character is split across token boundaries, and
+// the difference between two consecutive cumulative reports is then a lone
+// continuation byte. TranscriptStreamResponse.delta is a proto3 `string`, whose
+// wire format REQUIRES valid UTF-8: the C++ runtime serializes an invalid one
+// with at most a warning, but the Go runtime refuses to unmarshal it, and the
+// client loses every remaining delta AND the final_result. So no fragment this
+// class returns ever BEGINS OR ENDS inside a character: an incomplete trailing
+// sequence is held back and merged into the next fragment, and a leading orphan
+// continuation byte, which nothing can ever complete, is dropped.
+//
+// It is NOT a voxtral-only concern, which is what the first attempt at this
+// assumed. The incremental families split characters by the same arithmetic:
+// nemotron_asr's decoder cuts at a BYTE offset (decoder.cpp:550) and
+// vibevoice_asr's common_prefix_size compares BYTES (session.cpp:80-86). Nor is
+// European text the worst case: a Japanese transcript, whose every character is
+// three bytes, carried at least one invalid delta in 33.74% of traces until
+// rule 2 below learned to leave an incomplete fragment alone.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+class TranscriptDeltaTracker {
+public:
+    // Takes one StreamEvent::partial_text and returns the fragment to put on
+    // the wire, empty when there is nothing new.
+    //
+    // The rules, in order, all of them against everything KNOWN (delivered
+    // plus held back), never against the delivered text alone:
+    //   1. An empty partial says nothing.
+    //   2. A partial IDENTICAL to the known text is a repeat: nothing is
+    //      emitted. Identical, not merely a prefix of it. That absorbs voxtral's
+    //      repeat of the last event in each batch, which is the only duplicate
+    //      any pinned family produces and which carries byte-equal text both
+    //      times. Discarding a mere PREFIX used to swallow an incremental
+    //      fragment that coincided with the start of the transcript, corrupting
+    //      the text silently and, when that fragment was a character's lead
+    //      byte, killing the stream outright; see the note at the rule in the
+    //      implementation.
+    //   3. A partial that EXTENDS the known text is a cumulative report: only
+    //      its new suffix is emitted.
+    //   4. Anything else is an incremental fragment: it is emitted whole and
+    //      appended.
+    //
+    // Rule 3 is the one judgement call, since a fragment that happens to begin
+    // with the entire transcript so far is indistinguishable from a cumulative
+    // report. It is read as cumulative because every cumulative family produces
+    // that shape on EVERY event, while an incremental family produces it only
+    // when one fragment repeats everything before it, which no tokenizer output
+    // does in practice.
+    //
+    // What comes back is the fragment MINUS any incomplete trailing UTF-8
+    // sequence, which is carried into the next call, and minus any leading
+    // orphan continuation byte, which is dropped. So an empty return can also
+    // mean "the only new bytes were half a character", and the caller needs no
+    // knowledge of that: writing nothing is exactly right.
+    std::string observe(const std::string &partial_text);
+
+    // Reconciles against TaskResult::text_output, which is authoritative, and
+    // returns the fragment that makes appending every delta equal it.
+    //
+    // This is what makes the OFFLINE FALLBACK a single line rather than its own
+    // branch: with no partials observed, the assembly is empty and the whole
+    // final text comes back as one delta.
+    //
+    // It is also what FLUSHES a held-back UTF-8 sequence, and it can always do
+    // so: the final text is complete, so the fragment from the last delivered
+    // byte to its end ends on a character boundary.
+    //
+    // A final text that CONTRADICTS what was already sent returns empty. A
+    // fragment on the wire cannot be retracted, so the alternative would be to
+    // send the transcript a second time and let the client hold it twice.
+    // final_result carries the authoritative text either way.
+    std::string reconcile(const std::string &final_text);
+
+    // Everything the client has been sent, concatenated. Held-back bytes are
+    // deliberately NOT included: this is what the client holds, not what the
+    // tracker knows.
+    const std::string &assembled() const noexcept { return assembled_; }
+
+private:
+    // Appends the emittable prefix of `fragment` to assembled_ and returns it,
+    // keeping any incomplete trailing UTF-8 sequence in pending_.
+    std::string release(const std::string &fragment);
+
+    std::string assembled_;
+    // An incomplete trailing UTF-8 sequence, computed but not sent. Always a
+    // proper prefix of one character, so at most three bytes.
+    std::string pending_;
+};
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stream_delta_test.cpp

@@ -0,0 +1,486 @@
+// Unit tests for stream_delta. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+//
+// The traces below are transcribed from the pinned upstream sessions rather
+// than invented, because the whole reason this unit exists is that the four
+// streaming ASR families do NOT agree on what partial_text means.
+
+#include "stream_delta.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+using audiocpp_backend::TranscriptDeltaTracker;
+
+// Feeds a whole trace and returns what a client appending every emitted
+// fragment would end up holding, which is the only property that matters.
+static std::string client_view(TranscriptDeltaTracker &tracker,
+                               const std::vector<std::string> &partials,
+                               std::vector<std::string> *emitted = nullptr) {
+    std::string view;
+    for (const auto &partial : partials) {
+        const std::string fragment = tracker.observe(partial);
+        if (emitted != nullptr && !fragment.empty()) {
+            emitted->push_back(fragment);
+        }
+        view += fragment;
+    }
+    return view;
+}
+
+// nemotron_asr: decoder.cpp emits current_text.substr(emitted_text.size()) on
+// every non-blank token, i.e. INCREMENTAL fragments, through the stream event
+// sink during finalize().
+static void test_incremental_family() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view =
+        client_view(tracker, {"Local", " AI", " now", " speaks."}, &emitted);
+    check_eq(view, "Local AI now speaks.", "incremental deltas concatenate");
+    check(emitted.size() == 4, "incremental family emits one fragment per partial");
+    check_eq(tracker.assembled(), "Local AI now speaks.",
+             "incremental family assembles the whole transcript");
+    check_eq(tracker.reconcile("Local AI now speaks."), "",
+             "a final result the deltas already cover adds nothing");
+}
+
+// voxtral_realtime: process_one_stream_chunk sets partial_text to
+// tokenizer_.decode(streaming_token_ids_), the WHOLE accumulated hypothesis,
+// and process_available_stream_chunks then hands the SAME event to both the
+// sink and the caller, so every partial arrives twice.
+static void test_cumulative_family_with_duplicate_delivery() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view = client_view(
+        tracker, {"Local", "Local", "Local AI", "Local AI", "Local AI now",
+                  "Local AI now"},
+        &emitted);
+    check_eq(view, "Local AI now", "cumulative partials are not repeated to the client");
+    check(emitted.size() == 3,
+          "the duplicate delivery of each cumulative event emits nothing twice");
+    check_eq(emitted.empty() ? "" : emitted[0], "Local", "first cumulative fragment");
+    check_eq(emitted.size() < 2 ? "" : emitted[1], " AI", "second cumulative fragment");
+    check_eq(emitted.size() < 3 ? "" : emitted[2], " now", "third cumulative fragment");
+}
+
+// The rule that separates the two: a report IDENTICAL to everything known is a
+// repeat and is never sent again.
+//
+// Only an identical one. Rule 2 used to discard any report the known text merely
+// STARTED WITH, and that cost more than it bought: see
+// test_a_short_fragment_is_not_mistaken_for_a_repeat.
+static void test_an_exact_repeat_is_never_resent() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("hello world"), "hello world", "first fragment");
+    check_eq(tracker.observe("hello world"), "", "an identical repeat emits nothing");
+    check_eq(tracker.observe("hello world"), "", "and a third delivery emits nothing");
+    check_eq(tracker.assembled(), "hello world", "the assembly is unchanged by repeats");
+}
+
+// THE TRADE, pinned so it is a decision rather than a surprise. A CUMULATIVE
+// report that SHRINKS is no longer absorbed: it is read as an incremental
+// fragment and duplicates a few bytes at the client.
+//
+// No pinned family produces one. voxtral_realtime is the only cumulative
+// reporter, and its hypothesis is tokenizer_.decode(streaming_token_ids_) over a
+// vector that is only ever push_back'ed (session.cpp:436) and cleared by reset()
+// (session.cpp:257), so within a stream it grows and never shrinks. The
+// duplicate delivery rule 2 really exists for is an EXACT repeat, which the test
+// above still covers.
+static void test_a_shrinking_hypothesis_is_read_as_incremental() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("hello world"), "hello world", "first fragment");
+    check_eq(tracker.observe("hello"), "hello",
+             "a shortened hypothesis is now read as an incremental fragment");
+    check_eq(tracker.assembled(), "hello worldhello",
+             "which duplicates those bytes at the client: the accepted cost");
+}
+
+static void test_empty_partials_are_ignored() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe(""), "", "an empty partial emits nothing");
+    check_eq(tracker.observe("a"), "a", "a real partial after an empty one still emits");
+    check_eq(tracker.observe(""), "", "a later empty partial emits nothing");
+    check_eq(tracker.assembled(), "a", "empty partials do not disturb the assembly");
+}
+
+// The offline fallback: a family with no streaming ASR runs once, so nothing is
+// ever observed and the reconciliation IS the single delta the RPC promises.
+static void test_offline_fallback_is_one_delta() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.reconcile("the whole transcript"), "the whole transcript",
+             "with no partials the final text is emitted whole");
+    check_eq(tracker.assembled(), "the whole transcript",
+             "the reconciliation is recorded as delivered");
+    check_eq(tracker.reconcile("the whole transcript"), "",
+             "reconciling twice does not duplicate");
+}
+
+// A streaming family whose partials stopped short of the final text: the tail
+// is emitted so that appending every delta still equals final_result.text.
+static void test_reconcile_emits_the_tail() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("Local AI"), "Local AI", "partial arrives");
+    check_eq(tracker.reconcile("Local AI now speaks."), " now speaks.",
+             "the untold tail of the final text is emitted");
+    check_eq(tracker.assembled(), "Local AI now speaks.", "tail is recorded");
+}
+
+// Divergence. nemotron's decoder has a rewrite branch: when the new hypothesis
+// is NOT an extension of what it already emitted, it emits the whole new text.
+// Nothing can retract a fragment already written to the wire, so the tracker
+// must not try: it emits nothing further and leaves final_result authoritative.
+static void test_divergent_final_text_is_not_appended() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("the cat"), "the cat", "first hypothesis");
+    check_eq(tracker.reconcile("the dog"), "",
+             "a final text that contradicts the deltas is not appended to them");
+    check_eq(tracker.assembled(), "the cat",
+             "a contradicted assembly is left as it was actually sent");
+}
+
+static void test_empty_final_text() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("something"), "something", "partial arrives");
+    check_eq(tracker.reconcile(""), "", "an empty final text emits nothing");
+    check_eq(tracker.assembled(), "something", "an empty final text changes nothing");
+}
+
+// A whitespace-only fragment is real text: the space between two words is
+// exactly what an incremental family delivers on its own.
+static void test_whitespace_fragments_survive() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("one"), "one", "word");
+    check_eq(tracker.observe(" "), " ", "a bare separator is emitted");
+    check_eq(tracker.observe("two"), "two", "next word");
+    check_eq(tracker.assembled(), "one two", "separator is kept in the assembly");
+}
+
+// The ambiguity this unit cannot resolve, pinned so that a future reader sees
+// the choice rather than rediscovering it: a fragment that EXTENDS everything
+// delivered so far is read as a cumulative report, because that is what every
+// cumulative family produces on every event, while an incremental family
+// producing one is the rare coincidence of a fragment repeating the whole
+// transcript so far.
+static void test_prefix_extension_is_read_as_cumulative() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("I"), "I", "first fragment");
+    check_eq(tracker.observe("I'm"), "'m",
+             "a fragment extending the assembly is treated as a cumulative report");
+    check_eq(tracker.assembled(), "I'm", "cumulative reading assembles once");
+}
+
+// --------------------------------------------------------------------------
+// UTF-8 boundaries
+//
+// TranscriptStreamResponse.delta is a proto3 `string`, and the wire format
+// REQUIRES a string field to be valid UTF-8. The C++ runtime serializes an
+// invalid one with at most a warning; the Go runtime refuses to unmarshal it,
+// so the client loses every delta AND the final_result still to come.
+//
+// Not hypothetical. voxtral_realtime reports the whole hypothesis as
+// tokenizer_.decode(streaming_token_ids_), and that decode is a pure
+// concatenation of raw token BYTES (tokenizer_text.cpp:171-183), so a
+// multi-byte character is split across token boundaries and the cumulative
+// difference between two consecutive reports is a lone continuation byte.
+// --------------------------------------------------------------------------
+
+// True when `text` is well-formed UTF-8. Written out here rather than reused
+// from the implementation on purpose: a test that shares the implementation's
+// idea of a boundary cannot catch the implementation's idea being wrong.
+static bool is_valid_utf8(const std::string &text) {
+    size_t i = 0;
+    while (i < text.size()) {
+        const auto lead = static_cast<unsigned char>(text[i]);
+        size_t length = 0;
+        if ((lead & 0x80) == 0x00) {
+            length = 1;
+        } else if ((lead & 0xE0) == 0xC0) {
+            length = 2;
+        } else if ((lead & 0xF0) == 0xE0) {
+            length = 3;
+        } else if ((lead & 0xF8) == 0xF0) {
+            length = 4;
+        } else {
+            return false;
+        }
+        if (i + length > text.size()) {
+            return false;
+        }
+        for (size_t k = 1; k < length; ++k) {
+            if ((static_cast<unsigned char>(text[i + k]) & 0xC0) != 0x80) {
+                return false;
+            }
+        }
+        i += length;
+    }
+    return true;
+}
+
+// Written as byte escapes so the test does not depend on the encoding of this
+// source file.
+static const std::string kEAcute = "\xC3\xA9";        // 2 bytes
+static const std::string kEuro = "\xE2\x82\xAC";      // 3 bytes
+static const std::string kEmoji = "\xF0\x9F\x8E\xA7"; // 4 bytes
+
+// A cumulative family advancing its hypothesis one BYTE at a time, which is
+// what voxtral_realtime does across a multi-byte character.
+static void test_cumulative_split_multibyte_character() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string full = "5" + kEuro;
+    std::vector<std::string> partials;
+    for (size_t n = 1; n <= full.size(); ++n) {
+        partials.push_back(full.substr(0, n));
+    }
+    const std::string view = client_view(tracker, partials, &emitted);
+
+    check_eq(view, full, "a byte-at-a-time cumulative report still assembles");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "cumulative fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check_eq(tracker.reconcile(full), "", "the final text adds nothing");
+}
+
+// An incremental family splitting a character across two fragments.
+static void test_incremental_split_multibyte_character() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view = client_view(
+        tracker, {"caf" + kEAcute.substr(0, 1), kEAcute.substr(1), " au lait"},
+        &emitted);
+
+    check_eq(view, "caf" + kEAcute + " au lait",
+             "an incremental split character still assembles");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "incremental fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check(emitted.size() == 3, "one fragment out per partial, none swallowed");
+    if (emitted.size() == 3) {
+        check_eq(emitted[0], "caf", "the lead byte of the character is held back");
+        check_eq(emitted[1], kEAcute,
+                 "the held byte is merged into the next fragment, not sent alone");
+        check_eq(emitted[2], " au lait", "the rest follows unchanged");
+    }
+}
+
+// A 4 byte character split three ways, so the held-back buffer has to survive
+// more than one round.
+static void test_four_byte_character_split_three_ways() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view =
+        client_view(tracker,
+                    {"listen " + kEmoji.substr(0, 1), kEmoji.substr(1, 2),
+                     kEmoji.substr(3), " now"},
+                    &emitted);
+    check_eq(view, "listen " + kEmoji + " now",
+             "a 4 byte character survives three splits");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "4 byte fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+}
+
+// The held-back bytes must reach the client. reconcile can always flush them,
+// because the final text is complete by construction.
+static void test_reconcile_flushes_a_held_back_sequence() {
+    TranscriptDeltaTracker tracker;
+    const std::string first = tracker.observe("done" + kEuro.substr(0, 2));
+    check_eq(first, "done", "the incomplete trailing sequence is held back");
+    check(is_valid_utf8(first), "what was emitted is valid UTF-8");
+    const std::string tail = tracker.reconcile("done" + kEuro);
+    check_eq(tail, kEuro, "reconcile flushes the completed character");
+    check(is_valid_utf8(tail), "the flushed tail is valid UTF-8");
+    check_eq(tracker.assembled(), "done" + kEuro, "the client holds the whole text");
+}
+
+// Held-back bytes are not lost track of: the next cumulative report emits the
+// whole character rather than only the bytes that just arrived.
+static void test_held_bytes_join_the_next_fragment() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("a" + kEuro.substr(0, 1)), "a",
+             "only the complete prefix goes out");
+    check_eq(tracker.observe("a" + kEuro), kEuro,
+             "the next cumulative report emits the whole character at once");
+    check_eq(tracker.assembled(), "a" + kEuro, "assembly is correct");
+}
+
+// A whole multi-byte character arriving at once must NOT be held back: holding
+// a complete sequence would stall every stream by one character.
+static void test_a_complete_character_is_not_held() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("x" + kEuro), "x" + kEuro,
+             "a fragment ending on a boundary is emitted immediately");
+    check_eq(tracker.observe("x" + kEuro + kEmoji), kEmoji,
+             "and so is the next one");
+}
+
+// Bytes that can never complete must not be held forever: a stray continuation
+// byte or an invalid lead is passed through rather than stalling the stream.
+// Repairing a family's malformed output is not something a delta tracker can do
+// without altering the transcript.
+static void test_undecodable_bytes_are_not_held_forever() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe(std::string("ok\x80")), std::string("ok\x80"),
+             "a stray continuation byte is passed through, not held");
+    check_eq(tracker.observe(std::string("ok\x80") + "next"), "next",
+             "the stream continues");
+
+    // A lead byte the encoding does not define (0xF8 and above). Nothing can
+    // ever complete it, so holding it would stall the stream for good.
+    TranscriptDeltaTracker invalid_lead;
+    check_eq(invalid_lead.observe(std::string("ok\xFE")), std::string("ok\xFE"),
+             "an undefined lead byte is passed through, not held");
+    check_eq(invalid_lead.observe(std::string("ok\xFE") + "more"), "more",
+             "the stream continues past an undefined lead byte");
+
+    // The same byte followed by continuation bytes, which is the shape that
+    // looks most like a real sequence waiting to be completed.
+    TranscriptDeltaTracker invalid_run;
+    check_eq(invalid_run.observe(std::string("\xFE\x80\x80")), std::string("\xFE\x80\x80"),
+             "an undefined lead with continuations is passed through");
+
+    // Five continuation bytes with no lead in sight. They are DROPPED, not
+    // held: nothing can ever precede them, so holding would stall the stream
+    // for good, and emitting them would put invalid UTF-8 on the wire. See
+    // test_a_fragment_never_begins_mid_character.
+    TranscriptDeltaTracker orphans;
+    check_eq(orphans.observe(std::string("\x80\x80\x80\x80\x80")), "",
+             "a run of orphan continuation bytes is dropped, not emitted");
+    check_eq(orphans.observe("after"), "after",
+             "and the stream continues past them");
+}
+
+// THE SECOND HALF OF THE SAME BUG, and the one that survived fix round 1.
+//
+// Rule 2 discards a fragment the known text already starts with. When that
+// fragment is the LEAD BYTE of a NEW character it looks exactly like a repeat of
+// an earlier character beginning with the same byte, so it was discarded and
+// never held. Its continuation bytes then arrived on their own and began the
+// next delta, which is invalid UTF-8 at the FRONT, and utf8_complete_prefix_length
+// only ever inspected the TRAILING sequence.
+//
+// Reachable from shipping families, not synthetic: nemotron_asr's
+// decoder.cpp:550 cuts at a BYTE offset (current_text.substr(emitted_text.size()))
+// and vibevoice_asr's common_prefix_size (session.cpp:80-86) compares BYTES, so
+// both split characters mid-sequence. The trace below is exactly how they split
+// "ssee" spelled with the German sharp s, an e-acute, a euro sign and an o-grave,
+// three of which begin with the same 0xC3 lead byte.
+static void test_a_repeated_lead_byte_is_not_swallowed() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string sharp_s = "\xC3\x9F";  // U+00DF
+    const std::string e_acute = "\xC3\xA9";  // U+00E9
+    const std::string euro = "\xE2\x82\xAC"; // U+20AC
+    const std::string o_grave = "\xC3\xB2";  // U+00F2
+    const std::string full = sharp_s + e_acute + euro + o_grave;
+
+    const std::string view = client_view(tracker,
+                                         {sharp_s.substr(0, 1), sharp_s.substr(1),
+                                          e_acute.substr(0, 1),
+                                          e_acute.substr(1) + euro,
+                                          o_grave.substr(0, 1), o_grave.substr(1)},
+                                         &emitted);
+
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "repeated-lead fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check_eq(view, full, "no character is lost to a repeated lead byte");
+    check_eq(tracker.reconcile(full), "", "the final text adds nothing");
+}
+
+// The same shape one layer down, as a backstop: a fragment that BEGINS with
+// orphan continuation bytes must never go on the wire, whatever produced it.
+// Dropping bytes keeps the stream alive; emitting them ends it, and takes the
+// final_result that was still to come with it.
+static void test_a_fragment_never_begins_mid_character() {
+    TranscriptDeltaTracker tracker;
+    const std::string first = tracker.observe(std::string("\xA9") + "rest");
+    check(is_valid_utf8(first), "a leading orphan continuation byte is not emitted");
+    check_eq(first, "rest", "the rest of the fragment still goes out");
+
+    TranscriptDeltaTracker all_orphans;
+    check_eq(all_orphans.observe(std::string("\x82\xAC")), "",
+             "a fragment that is nothing but orphans emits nothing");
+    check_eq(all_orphans.assembled(), "",
+             "and nothing is recorded as delivered");
+    check_eq(all_orphans.observe("after"), "after", "the stream continues");
+}
+
+// PURE ASCII, no multi-byte character anywhere, and the transcript still comes
+// out wrong: a short incremental fragment that happens to be a byte prefix of
+// everything known was read as an already-delivered repeat and discarded.
+//
+// This is the shape both incremental families produce. nemotron_asr emits
+// current_text.substr(emitted_text.size()) per non-blank token (decoder.cpp:550)
+// and vibevoice_asr emits text.substr(common_prefix_size(...)) (session.cpp:89-105),
+// so a one-character fragment is ordinary output, and any of the transcript's
+// own leading characters will eventually arrive as one.
+//
+// Measured over 5,000 randomized traces per transcript before this was fixed:
+// 9.50% of pure-ASCII traces and 29.12% of French ones ended with the client
+// holding something other than final_result.text, with a 200 and no diagnostic.
+static void test_a_short_fragment_is_not_mistaken_for_a_repeat() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string full = "pure ascii transcript";
+    const std::string view = client_view(
+        tracker, {"pure ", "ascii ", "trans", "c", "ri", "p", "t"}, &emitted);
+
+    check_eq(view, full, "the whole transcript reaches the client");
+    check_eq(tracker.assembled(), full, "and the assembly agrees with it");
+    check_eq(tracker.reconcile(full), "",
+             "the final text adds nothing, because nothing was lost");
+    check(emitted.size() == 7, "every fragment produced exactly one delta");
+}
+
+int main() {
+    test_incremental_family();
+    test_cumulative_family_with_duplicate_delivery();
+    test_an_exact_repeat_is_never_resent();
+    test_a_shrinking_hypothesis_is_read_as_incremental();
+    test_a_short_fragment_is_not_mistaken_for_a_repeat();
+    test_empty_partials_are_ignored();
+    test_offline_fallback_is_one_delta();
+    test_reconcile_emits_the_tail();
+    test_divergent_final_text_is_not_appended();
+    test_empty_final_text();
+    test_whitespace_fragments_survive();
+    test_prefix_extension_is_read_as_cumulative();
+    test_cumulative_split_multibyte_character();
+    test_incremental_split_multibyte_character();
+    test_four_byte_character_split_three_ways();
+    test_reconcile_flushes_a_held_back_sequence();
+    test_held_bytes_join_the_next_fragment();
+    test_a_complete_character_is_not_held();
+    test_undecodable_bytes_are_not_held_forever();
+    test_a_repeated_lead_byte_is_not_swallowed();
+    test_a_fragment_never_begins_mid_character();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all stream_delta checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/streaming_driver_ctest.cpp

@@ -0,0 +1,976 @@
+// Tests for the streaming drivers in loaded_model: begin_stream,
+// run_streaming_pull, run_streaming_audio and run_streaming_live, plus
+// resolve_model_path, which lives in the same engine-linked unit.
+//
+// Engine-linked, so this runs through ctest rather than
+// backend/cpp/run-unit-tests.sh. It builds no model and loads no file: a
+// LoadedModel::Session is a plain struct holding a pointer to an engine
+// interface, so a fake session exercises the drivers directly, which is the
+// only way to assert the STATE OBLIGATION (prepare, then start_stream, on every
+// stream) without a GPU and a gigabyte of weights.
+
+#include "inference_lane.h"
+#include "loaded_model.h"
+
+#include "engine/framework/runtime/session.h"
+
+#include <cstddef>
+#include <cstdio>
+#include <functional>
+#include <memory>
+#include <optional>
+#include <stdexcept>
+#include <string>
+#include <vector>
+
+namespace rt = engine::runtime;
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+static std::string join(const std::vector<std::string> &parts) {
+    std::string out;
+    for (const auto &part : parts) {
+        if (!out.empty()) {
+            out += "|";
+        }
+        out += part;
+    }
+    return out;
+}
+
+// --------------------------------------------------------------------------
+// Fakes
+// --------------------------------------------------------------------------
+
+// Consumes audio chunks, like every streaming ASR family.
+//
+// It deliberately does NOT override start_stream, so every test that drives it
+// also pins the claim the drivers rely on: IStreamingVoiceTaskSession's BASE
+// start_stream is a call to reset(). If upstream ever changes that base, the
+// replay test below fails rather than the backend silently continuing the
+// previous stream.
+class FakeAudioSession : public rt::IStreamingVoiceTaskSession {
+public:
+    std::vector<std::string> calls;
+    std::vector<rt::AudioChunk> chunks;
+    rt::StreamingPolicy policy;
+    // Set to make process_audio_chunk throw on the nth call (1-based).
+    int throw_on_chunk = 0;
+    // Cumulative partial text, which is voxtral_realtime's convention.
+    bool report_partials = true;
+
+    std::string family() const override { return "fake_audio"; }
+    rt::VoiceTaskKind task_kind() const override { return rt::VoiceTaskKind::Asr; }
+    rt::RunMode run_mode() const override { return rt::RunMode::Streaming; }
+
+    void prepare(const rt::SessionPreparationRequest &request) override {
+        calls.push_back("prepare");
+        prepared_ = true;
+        prepared_rate_ = request.audio.has_value() ? request.audio->sample_rate : 0;
+    }
+
+    rt::StreamingPolicy streaming_policy() const override { return policy; }
+
+    void set_stream_event_sink(rt::StreamEventCallback sink) override {
+        calls.push_back(sink ? "sink+" : "sink-");
+        sink_ = std::move(sink);
+    }
+
+    void reset() override {
+        if (!prepared_) {
+            // Exactly what silero_vad does, and the reason prepare() has to come
+            // first rather than being folded into session_for.
+            throw std::runtime_error("fake: prepare() must be called before reset()");
+        }
+        calls.push_back("reset");
+        seen_frames_ = 0;
+        seen_chunks_ = 0;
+        text_.clear();
+    }
+
+    rt::StreamEvent process_audio_chunk(const rt::AudioChunk &chunk) override {
+        calls.push_back("chunk");
+        chunks.push_back(chunk);
+        ++seen_chunks_;
+        if (throw_on_chunk == seen_chunks_) {
+            throw std::runtime_error("fake: chunk failure");
+        }
+        const int channels = chunk.channels > 0 ? chunk.channels : 1;
+        seen_frames_ += static_cast<std::int64_t>(chunk.samples.size()) / channels;
+        text_ += "w" + std::to_string(seen_chunks_);
+        rt::StreamEvent event;
+        if (report_partials) {
+            event.partial_text = rt::Transcript{text_, "en"};
+        }
+        return event;
+    }
+
+    rt::TaskResult finalize() override {
+        calls.push_back("finalize");
+        rt::TaskResult result;
+        result.text_output = rt::Transcript{
+            text_ + "/frames=" + std::to_string(seen_frames_), "en"};
+        return result;
+    }
+
+    // Emits through the SINK the way nemotron_asr does, from inside the final
+    // step rather than from process_audio_chunk.
+    void emit_through_sink(const std::string &fragment) {
+        if (!sink_) {
+            return;
+        }
+        rt::StreamEvent event;
+        event.partial_text = rt::Transcript{fragment, "en"};
+        sink_(event);
+    }
+
+    bool sink_installed() const { return static_cast<bool>(sink_); }
+    int prepared_rate() const { return prepared_rate_; }
+
+private:
+    rt::StreamEventCallback sink_;
+    bool prepared_ = false;
+    int prepared_rate_ = 0;
+    std::int64_t seen_frames_ = 0;
+    int seen_chunks_ = 0;
+    std::string text_;
+};
+
+// nemotron_asr's shape: partials arrive only through the sink, and only from
+// inside the finalize step.
+class SinkOnlyAudioSession : public FakeAudioSession {
+public:
+    SinkOnlyAudioSession() { report_partials = false; }
+
+    rt::TaskResult finalize() override {
+        emit_through_sink("late ");
+        emit_through_sink("partial");
+        return FakeAudioSession::finalize();
+    }
+};
+
+// Pulls events, like every streaming TTS family. Overrides start_stream the way
+// the seven real families do, calling reset() first.
+class FakePullSession : public rt::IStreamingVoiceTaskSession {
+public:
+    std::vector<std::string> calls;
+    std::size_t event_count = 3;
+    bool final_on_second = false;
+
+    std::string family() const override { return "fake_pull"; }
+    rt::VoiceTaskKind task_kind() const override { return rt::VoiceTaskKind::Tts; }
+    rt::RunMode run_mode() const override { return rt::RunMode::Streaming; }
+
+    void prepare(const rt::SessionPreparationRequest &) override {
+        calls.push_back("prepare");
+        prepared_ = true;
+    }
+
+    rt::StreamingPolicy streaming_policy() const override {
+        rt::StreamingPolicy policy;
+        policy.input = rt::StreamingInputKind::None;
+        policy.output = rt::StreamingOutputKind::PullEvents;
+        return policy;
+    }
+
+    void start_stream(const rt::TaskRequest &request) override {
+        calls.push_back("start_stream");
+        (void)request;
+        reset();
+    }
+
+    void set_stream_event_sink(rt::StreamEventCallback sink) override {
+        calls.push_back(sink ? "sink+" : "sink-");
+        sink_ = std::move(sink);
+    }
+
+    void reset() override {
+        if (!prepared_) {
+            throw std::runtime_error("fake: prepare() must be called before reset()");
+        }
+        calls.push_back("reset");
+        emitted_ = 0;
+    }
+
+    std::optional<rt::StreamEvent> next_stream_event() override {
+        if (emitted_ >= event_count) {
+            return std::nullopt;
+        }
+        rt::StreamEvent event;
+        rt::AudioBuffer audio;
+        audio.sample_rate = 24000;
+        audio.channels = 1;
+        audio.samples.assign(4, 0.25F);
+        // named_audio_outputs, NOT audio_output: this is where supertonic,
+        // omnivoice and voxcpm2 all put their streamed chunks.
+        event.named_audio_outputs.push_back(
+            {"chunk_" + std::to_string(emitted_), std::move(audio), {}});
+        ++emitted_;
+        if (final_on_second && emitted_ == 2) {
+            event.is_final = true;
+        }
+        calls.push_back("pull");
+        return event;
+    }
+
+    rt::StreamEvent process_audio_chunk(const rt::AudioChunk &) override {
+        throw std::runtime_error("fake_pull consumes no audio");
+    }
+
+    rt::TaskResult finalize() override {
+        calls.push_back("finalize");
+        rt::TaskResult result;
+        rt::AudioBuffer merged;
+        merged.sample_rate = 24000;
+        merged.channels = 1;
+        merged.samples.assign(4 * emitted_, 0.25F);
+        result.audio_output = std::move(merged);
+        return result;
+    }
+
+    bool sink_installed() const { return static_cast<bool>(sink_); }
+
+private:
+    rt::StreamEventCallback sink_;
+    bool prepared_ = false;
+    std::size_t emitted_ = 0;
+};
+
+// --------------------------------------------------------------------------
+// Helpers
+// --------------------------------------------------------------------------
+
+static audiocpp_backend::LoadedModel::Session
+streaming_session(rt::IStreamingVoiceTaskSession &fake,
+                  audiocpp_backend::Task task) {
+    audiocpp_backend::LoadedModel::Session session;
+    session.task = task;
+    session.mode = audiocpp_backend::Mode::Streaming;
+    session.streaming = &fake;
+    return session;
+}
+
+static rt::TaskRequest audio_request(int sample_rate, int channels,
+                                     std::int64_t frames) {
+    rt::TaskRequest request;
+    rt::AudioBuffer audio;
+    audio.sample_rate = sample_rate;
+    audio.channels = channels;
+    audio.samples.assign(static_cast<std::size_t>(frames * channels), 0.5F);
+    request.audio_input = std::move(audio);
+    return request;
+}
+
+// --------------------------------------------------------------------------
+// Tests
+// --------------------------------------------------------------------------
+
+static void test_begin_stream_prepares_then_starts() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakePullSession fake;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Tts);
+
+    rt::TaskRequest request;
+    audiocpp_backend::begin_stream(session, request, entry);
+
+    check_eq(join(fake.calls), "prepare|start_stream|reset",
+             "begin_stream prepares before it starts, and start_stream resets");
+}
+
+// The base implementation of start_stream IS a reset(). FakeAudioSession does
+// not override start_stream, so this is that guarantee, read out of the pinned
+// header rather than assumed.
+static void test_base_start_stream_resets() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    audiocpp_backend::begin_stream(session, audio_request(16000, 1, 10), entry);
+    check_eq(join(fake.calls), "prepare|reset",
+             "the interface's own start_stream resets the session");
+}
+
+static void test_begin_stream_refuses_a_non_streaming_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    audiocpp_backend::LoadedModel::Session session;
+    session.mode = audiocpp_backend::Mode::Offline;
+
+    bool threw_capability = false;
+    try {
+        rt::TaskRequest request;
+        audiocpp_backend::begin_stream(session, request, entry);
+    } catch (const audiocpp_backend::CapabilityError &) {
+        threw_capability = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_capability,
+          "begin_stream on an offline session throws CapabilityError, not a null deref");
+}
+
+// THE ONE THIS TASK IS ABOUT. A streaming session is cached, so the second
+// stream gets the object the first one left behind. Two identical runs against
+// the SAME session must produce identical output.
+static void test_a_refetched_session_replays_identically() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1536);
+
+    std::vector<std::string> first_fragments;
+    const auto first = audiocpp_backend::run_streaming_audio(
+        session, request, *request.audio_input,
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                first_fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    std::vector<std::string> second_fragments;
+    const auto second = audiocpp_backend::run_streaming_audio(
+        session, request, *request.audio_input,
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                second_fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    check_eq(join(second_fragments), join(first_fragments),
+             "a re-fetched streaming session replays the same partials");
+    check_eq(second.text_output.has_value() ? second.text_output->text : "",
+             first.text_output.has_value() ? first.text_output->text : "",
+             "a re-fetched streaming session replays the same final text");
+    check_eq(first.text_output.has_value() ? first.text_output->text : "",
+             "w1w2w3/frames=1536",
+             "the first run saw exactly the audio it was given");
+    // Not a tautology: without the reset the second run reports six words and
+    // 3072 frames, and both checks above fail.
+    check_eq(join(first_fragments), "w1|w1w2|w1w2w3", "cumulative partials");
+}
+
+static void test_run_streaming_audio_installs_and_clears_the_sink() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    SinkOnlyAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 1024;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1024);
+
+    std::vector<std::string> fragments;
+    const auto result = audiocpp_backend::run_streaming_audio(
+        session, request, *request.audio_input,
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    check_eq(join(fragments), "late |partial",
+             "a family that reports only through the sink is not silent");
+    check(!fake.sink_installed(),
+          "the sink is cleared before returning, so the cached session holds no "
+          "reference to the caller's frame");
+    check_eq(join(fake.calls), "sink+|prepare|reset|chunk|finalize|sink-",
+             "the sink is installed before the stream begins and cleared after it ends");
+    check(result.text_output.has_value(), "the final result still comes back");
+}
+
+static void test_the_sink_is_cleared_when_the_stream_throws() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    fake.throw_on_chunk = 1;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1024);
+
+    bool threw = false;
+    try {
+        audiocpp_backend::run_streaming_audio(
+            session, request, *request.audio_input,
+            [](const rt::StreamEvent &) {}, entry);
+    } catch (const std::exception &) {
+        threw = true;
+    }
+    check(threw, "a failing chunk propagates");
+    check(!fake.sink_installed(),
+          "the sink is cleared on the exception path too");
+}
+
+static void test_chunking_honours_the_policy_sample_count() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 16000;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    // 2.5 chunks, so the last one is short.
+    const auto request = audio_request(16000, 1, 40000);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 3, "40000 frames at 16000 per chunk is three chunks");
+    if (fake.chunks.size() == 3) {
+        check(fake.chunks[0].samples.size() == 16000, "first chunk is full");
+        check(fake.chunks[1].samples.size() == 16000, "second chunk is full");
+        check(fake.chunks[2].samples.size() == 8000, "last chunk is the remainder");
+        check(fake.chunks[0].start_sample == 0, "first chunk starts at zero");
+        check(fake.chunks[1].start_sample == 16000, "second chunk start index");
+        check(fake.chunks[2].start_sample == 32000, "third chunk start index");
+        check(fake.chunks[0].sample_rate == 16000, "chunk carries the buffer's rate");
+        check(fake.chunks[0].channels == 1, "chunk carries the buffer's channel count");
+    }
+}
+
+// A buffer whose float count is not a whole number of frames is REFUSED rather
+// than truncated. The integer division would otherwise drop the tail floats
+// from the fed audio, and therefore from the transcript, with no diagnostic.
+static void test_a_partial_trailing_frame_is_refused() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 100;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    // 501 floats across 2 channels: 250 whole frames and one stray float.
+    rt::TaskRequest request;
+    rt::AudioBuffer audio;
+    audio.sample_rate = 48000;
+    audio.channels = 2;
+    audio.samples.assign(501, 0.5F);
+    request.audio_input = std::move(audio);
+
+    bool threw_config = false;
+    try {
+        audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                              [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::ConfigError &) {
+        threw_config = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config,
+          "a buffer that is not a whole number of frames is refused with ConfigError");
+    check(fake.calls.empty(),
+          "the refusal precedes every call into the session, so no half-started "
+          "stream is left on the cached one");
+}
+
+// higgs_audio_stt states its window in seconds and leaves the sample count at
+// zero, so this branch is a real family's path rather than a defensive one.
+static void test_chunking_falls_back_to_the_policy_seconds() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 0;
+    fake.policy.preferred_audio_chunk_seconds = 4.0;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 96000); // 6 s
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 2, "6 s at a 4 s window is two chunks");
+    if (fake.chunks.size() == 2) {
+        check(fake.chunks[0].samples.size() == 64000, "first window is 4 s");
+        check(fake.chunks[1].samples.size() == 32000, "second window is the 2 s remainder");
+    }
+}
+
+static void test_chunking_falls_back_to_the_interface_default() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 0;
+    fake.policy.preferred_audio_chunk_seconds = 0.0;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1024);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 2, "a policy naming no window uses the interface's 512");
+    if (!fake.chunks.empty()) {
+        check(fake.chunks[0].samples.size() == 512, "default window is 512 frames");
+    }
+}
+
+// A zero sample rate must not turn a seconds-only policy into a zero-length
+// chunk, which would loop forever.
+static void test_a_seconds_policy_with_no_rate_falls_through() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 0;
+    fake.policy.preferred_audio_chunk_seconds = 4.0;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(0, 1, 1024);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+    check(fake.chunks.size() == 2, "a rateless buffer still chunks at the default 512");
+}
+
+// FRAMES, not floats. vibevoice_asr refuses a chunk whose sample count is not
+// divisible by its channel count, and offsets every span it reports by the
+// chunk's start_sample.
+static void test_stereo_chunks_are_frame_aligned() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 300;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(48000, 2, 750);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 3, "750 frames at 300 frames per chunk is three chunks");
+    for (const auto &chunk : fake.chunks) {
+        check(chunk.samples.size() % 2 == 0, "every stereo chunk is a whole number of frames");
+    }
+    if (fake.chunks.size() == 3) {
+        check(fake.chunks[0].samples.size() == 600, "300 stereo frames is 600 floats");
+        check(fake.chunks[1].start_sample == 300,
+              "start_sample counts frames, not floats");
+        check(fake.chunks[2].samples.size() == 300, "the remainder is 150 frames");
+    }
+}
+
+static void test_pull_drains_every_event_and_installs_no_sink() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakePullSession fake;
+    fake.event_count = 3;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Tts);
+
+    std::vector<std::string> ids;
+    rt::TaskRequest request;
+    const auto result = audiocpp_backend::run_streaming_pull(
+        session, request,
+        [&](const rt::StreamEvent &event) {
+            for (const auto &named : event.named_audio_outputs) {
+                ids.push_back(named.id);
+            }
+        },
+        entry);
+
+    check_eq(join(ids), "chunk_0|chunk_1|chunk_2", "every pulled event reaches the caller");
+    check(!fake.sink_installed(),
+          "no stream event sink is installed on the pull path, so voxcpm2 cannot "
+          "deliver every chunk twice");
+    check_eq(join(fake.calls), "prepare|start_stream|reset|pull|pull|pull|finalize",
+             "prepare, start, drain, finish");
+    check(result.audio_output.has_value(), "the merged result comes back");
+    check(result.audio_output.has_value() && result.audio_output->samples.size() == 12,
+          "the merged result is the whole synthesis, not a tail");
+}
+
+static void test_pull_stops_on_a_final_event() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakePullSession fake;
+    fake.event_count = 5;
+    fake.final_on_second = true;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Tts);
+
+    int events = 0;
+    rt::TaskRequest request;
+    audiocpp_backend::run_streaming_pull(
+        session, request, [&](const rt::StreamEvent &) { ++events; }, entry);
+
+    check(events == 2, "an event marked final ends the pull loop");
+}
+
+static void test_pull_refuses_a_non_streaming_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    audiocpp_backend::LoadedModel::Session session;
+
+    bool threw_capability = false;
+    try {
+        rt::TaskRequest request;
+        audiocpp_backend::run_streaming_pull(
+            session, request, [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::CapabilityError &) {
+        threw_capability = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_capability, "run_streaming_pull refuses a session with no streaming half");
+}
+
+// prepare() runs on EVERY stream, not once per session: the preparation request
+// is derived from the request (audio contract, text, voice), so a second stream
+// at a different rate would otherwise run against the first one's contract.
+static void test_prepare_tracks_the_request_not_the_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 4096;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    const auto first = audio_request(16000, 1, 4096);
+    audiocpp_backend::run_streaming_audio(session, first, *first.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+    check(fake.prepared_rate() == 16000, "the first stream prepares at its own rate");
+
+    const auto second = audio_request(44100, 1, 4096);
+    audiocpp_backend::run_streaming_audio(session, second, *second.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+    check(fake.prepared_rate() == 44100,
+          "the second stream prepares at ITS rate, not the first one's");
+}
+
+// --------------------------------------------------------------------------
+// run_streaming_live
+// --------------------------------------------------------------------------
+
+// Hands the driver a fixed list of wire frames, the way a client's audio
+// callback would, and then closes.
+static std::function<bool(std::vector<float> &)>
+frames_from(const std::vector<std::size_t> &sizes) {
+    auto index = std::make_shared<std::size_t>(0);
+    auto list = std::make_shared<std::vector<std::size_t>>(sizes);
+    return [index, list](std::vector<float> &out) {
+        if (*index >= list->size()) {
+            return false;
+        }
+        out.assign((*list)[*index], 0.5F);
+        ++*index;
+        return true;
+    };
+}
+
+static rt::TaskRequest live_request(int sample_rate, int channels) {
+    rt::TaskRequest request;
+    rt::AudioBuffer contract;
+    contract.sample_rate = sample_rate;
+    contract.channels = channels;
+    request.audio_input = std::move(contract); // no samples: none exist yet
+    return request;
+}
+
+// The wire's frame size is a property of the client's audio callback. The
+// family's window is a statement about what it can decode. The driver feeds the
+// second, not the first.
+static void test_live_buffers_wire_frames_into_policy_windows() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 1600;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    // Ten 512-sample frames: 5120 samples, i.e. three full 1600 windows and a
+    // 320 sample tail.
+    const auto result = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1),
+        frames_from(std::vector<std::size_t>(10, 512)),
+        [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 4,
+          "5120 wire samples at a 1600 frame window is three windows and a tail");
+    if (fake.chunks.size() == 4) {
+        check(fake.chunks[0].samples.size() == 1600, "first window is full");
+        check(fake.chunks[2].samples.size() == 1600, "third window is full");
+        check(fake.chunks[3].samples.size() == 320, "the tail is what was left");
+        check(fake.chunks[0].start_sample == 0, "the first window starts at zero");
+        check(fake.chunks[1].start_sample == 1600, "start_sample counts frames");
+        check(fake.chunks[3].start_sample == 4800, "the tail is offset by all of it");
+        check(fake.chunks[0].sample_rate == 16000, "the chunk carries the session rate");
+    }
+    check(result.text_output.has_value() &&
+              result.text_output->text == "w1w2w3w4/frames=5120",
+          "every wire sample reaches the family exactly once");
+}
+
+// nemotron_asr's shape: no partials from process_audio_chunk, every one of them
+// through the sink from inside finalize.
+static void test_live_installs_and_clears_the_sink() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    SinkOnlyAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    std::vector<std::string> fragments;
+    audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({512}),
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    check_eq(join(fragments), "late |partial",
+             "a family that reports only through the sink is not silent live either");
+    check(!fake.sink_installed(),
+          "the sink is cleared before returning, so the cached session holds no "
+          "reference to this call's frame");
+    check_eq(join(fake.calls), "sink+|prepare|reset|chunk|finalize|sink-",
+             "sink installed before the stream begins, cleared after it ends");
+}
+
+// A client that opens a session and closes it without speaking. finalize is NOT
+// called: nemotron_asr throws "finalize requires streamed audio", and an empty
+// transcript is the truthful answer to transcribing nothing.
+static void test_live_with_no_audio_never_finalizes() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    const auto result = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({}),
+        [](const rt::StreamEvent &) {}, entry);
+
+    check_eq(join(fake.calls), "sink+|prepare|reset|sink-",
+             "an empty live stream begins and ends without a chunk or a finalize");
+    check(!result.text_output.has_value(),
+          "an empty live stream reports no transcript rather than an error");
+}
+
+// A tail shorter than a window is still fed. Without this the last fragment of
+// speech never reaches the model, and nothing says so.
+static void test_live_feeds_a_short_tail() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 16000;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    audiocpp_backend::run_streaming_live(session, live_request(16000, 1),
+                                         frames_from({100, 200}),
+                                         [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 1,
+          "300 samples against a 16000 frame window is one short chunk, not none");
+    if (!fake.chunks.empty()) {
+        check(fake.chunks[0].samples.size() == 300, "the tail carries everything fed");
+    }
+}
+
+// A live request carries no samples, so the CONTRACT is the only thing that says
+// what rate the frames are in, and prepare() needs it: nemotron_asr's streaming
+// prepare throws without one.
+static void test_live_prepares_at_the_contract_rate() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    audiocpp_backend::run_streaming_live(session, live_request(16000, 1),
+                                         frames_from({512}),
+                                         [](const rt::StreamEvent &) {}, entry);
+    check(fake.prepared_rate() == 16000,
+          "the empty contract buffer still carries the rate into prepare()");
+
+    bool threw_config = false;
+    try {
+        rt::TaskRequest bare; // no audio_input at all
+        audiocpp_backend::run_streaming_live(session, bare, frames_from({512}),
+                                             [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::ConfigError &) {
+        threw_config = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config, "a live request with no audio contract is refused");
+}
+
+// The pull function is the gRPC read, and a request the handler has to refuse
+// mid-stream unwinds through the driver. The sink must not survive it.
+static void test_live_clears_the_sink_when_the_puller_throws() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    bool threw = false;
+    try {
+        audiocpp_backend::run_streaming_live(
+            session, live_request(16000, 1),
+            [](std::vector<float> &) -> bool {
+                throw std::runtime_error("fake: the client vanished");
+            },
+            [](const rt::StreamEvent &) {}, entry);
+    } catch (const std::exception &) {
+        threw = true;
+    }
+    check(threw, "a failing pull propagates");
+    check(!fake.sink_installed(), "the sink is cleared on the pull's exception path");
+}
+
+// Two live streams over the SAME cached session must not run into each other.
+static void test_live_replays_identically_on_a_refetched_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    const auto first = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({512, 512}),
+        [](const rt::StreamEvent &) {}, entry);
+    const auto second = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({512, 512}),
+        [](const rt::StreamEvent &) {}, entry);
+
+    check_eq(second.text_output.has_value() ? second.text_output->text : "",
+             first.text_output.has_value() ? first.text_output->text : "",
+             "a re-fetched live session replays the same transcript");
+    // Not a tautology: without the reset the second run reports w1..w4 and 2048
+    // frames.
+    check_eq(first.text_output.has_value() ? first.text_output->text : "",
+             "w1w2/frames=1024", "the first live run saw exactly what was fed");
+}
+
+static void test_live_refuses_a_non_streaming_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    audiocpp_backend::LoadedModel::Session session;
+    session.mode = audiocpp_backend::Mode::Offline;
+
+    bool threw_capability = false;
+    try {
+        audiocpp_backend::run_streaming_live(session, live_request(16000, 1),
+                                             frames_from({512}),
+                                             [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::CapabilityError &) {
+        threw_capability = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_capability,
+          "run_streaming_live refuses a session with no streaming half");
+}
+
+// --------------------------------------------------------------------------
+// resolve_model_path
+// --------------------------------------------------------------------------
+//
+// It lives in loaded_model.cpp and is a pure (dir, file, name) -> string, so it
+// is tested here rather than in a standalone unit: that file cannot compile
+// without the engine headers.
+//
+// It is tested at all because it shipped a bug no test could have caught. THE
+// SHAPES BELOW ARE THE PRODUCTION SHAPES, not convenient ones, and that
+// distinction is the entire point. Task 15 verified the bundled: form with a
+// hand-written LoadModel that left ModelFile empty, which is the one shape the
+// server never produces: pkg/model/loader.go's LoadModelWithFile always fills
+// ModelFile with filepath.Join(ModelPath, model), and core/backend/options.go
+// only overrides it for a managed artifact. The first case below is therefore
+// the regression test; the other three are what it must not have broken.
+
+// std::string::ends_with is C++20 and this target is C++17.
+static bool ends_with(const std::string &value, const std::string &suffix) {
+    return value.size() >= suffix.size() &&
+           value.compare(value.size() - suffix.size(), suffix.size(), suffix) == 0;
+}
+
+// THE REGRESSION CASE. What a model YAML saying `model: bundled:silero_vad`
+// actually arrives as: Model intact, ModelFile joined onto the models directory.
+static void test_bundled_in_model_survives_a_joined_model_file() {
+    const std::string resolved = audiocpp_backend::resolve_model_path(
+        "/models", "/models/bundled:silero_vad", "bundled:silero_vad");
+
+    check(ends_with(resolved, "/assets/silero_vad"),
+          "bundled: in Model resolves under the package assets dir (got \"" +
+              resolved + "\")");
+    // Checked separately from the suffix because this is the failure that
+    // shipped: the joined ModelFile came back verbatim and the load died on
+    // "model path does not exist: /models/bundled:silero_vad".
+    check(resolved.find("/models/") == std::string::npos,
+          "bundled: in Model is not resolved against the models directory (got \"" +
+              resolved + "\")");
+}
+
+// Task 15's shape: the form in ModelFile with Model empty. It worked before the
+// fix and must keep working.
+static void test_bundled_in_model_file_still_resolves() {
+    const std::string resolved =
+        audiocpp_backend::resolve_model_path("", "bundled:marblenet_vad", "");
+
+    check(ends_with(resolved, "/assets/marblenet_vad"),
+          "bundled: in ModelFile still resolves under the package assets dir (got \"" +
+              resolved + "\")");
+}
+
+// The ordinary case, and the one the bundled: lookup must not capture: a real
+// artifact path in ModelFile with a plain name in Model.
+static void test_a_plain_name_resolves_to_the_model_file() {
+    const std::string resolved = audiocpp_backend::resolve_model_path(
+        "/models", "/models/chatterbox-q8_0.gguf", "chatterbox-q8_0.gguf");
+
+    check_eq(resolved, "/models/chatterbox-q8_0.gguf",
+             "a plain name resolves to the absolute ModelFile");
+}
+
+// A relative ModelFile is still joined onto ModelPath. The fix does not touch
+// this branch, which is why it is pinned: the bundled: lookup now runs before it
+// and has to fall through for every non-bundled input.
+static void test_a_relative_model_file_joins_the_model_path() {
+    const std::string resolved = audiocpp_backend::resolve_model_path(
+        "/models", "sub/nemotron-asr-q8_0.gguf", "nemotron-asr");
+
+    check_eq(resolved, "/models/sub/nemotron-asr-q8_0.gguf",
+             "a relative ModelFile joins the models directory");
+}
+
+int main() {
+    test_bundled_in_model_survives_a_joined_model_file();
+    test_bundled_in_model_file_still_resolves();
+    test_a_plain_name_resolves_to_the_model_file();
+    test_a_relative_model_file_joins_the_model_path();
+    test_begin_stream_prepares_then_starts();
+    test_base_start_stream_resets();
+    test_begin_stream_refuses_a_non_streaming_session();
+    test_a_refetched_session_replays_identically();
+    test_run_streaming_audio_installs_and_clears_the_sink();
+    test_the_sink_is_cleared_when_the_stream_throws();
+    test_chunking_honours_the_policy_sample_count();
+    test_a_partial_trailing_frame_is_refused();
+    test_chunking_falls_back_to_the_policy_seconds();
+    test_chunking_falls_back_to_the_interface_default();
+    test_a_seconds_policy_with_no_rate_falls_through();
+    test_stereo_chunks_are_frame_aligned();
+    test_pull_drains_every_event_and_installs_no_sink();
+    test_pull_stops_on_a_final_event();
+    test_pull_refuses_a_non_streaming_session();
+    test_prepare_tracks_the_request_not_the_session();
+    test_live_buffers_wire_frames_into_policy_windows();
+    test_live_installs_and_clears_the_sink();
+    test_live_with_no_audio_never_finalizes();
+    test_live_feeds_a_short_tail();
+    test_live_prepares_at_the_contract_rate();
+    test_live_clears_the_sink_when_the_puller_throws();
+    test_live_replays_identically_on_a_refetched_session();
+    test_live_refuses_a_non_streaming_session();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all streaming driver checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/transcript_assembly.cpp

@@ -0,0 +1,215 @@
+#include "transcript_assembly.h"
+
+#include "audio_units.h"
+
+#include <algorithm>
+#include <cstdlib>
+#include <limits>
+
+namespace audiocpp_backend {
+namespace {
+
+std::int64_t midpoint(const Span &span) {
+    return span.start_sample + (span.end_sample - span.start_sample) / 2;
+}
+
+bool contains(const Span &span, std::int64_t sample) {
+    return sample >= span.start_sample && sample < span.end_sample;
+}
+
+std::int64_t overlap(const Span &a, const Span &b) {
+    const std::int64_t begin = std::max(a.start_sample, b.start_sample);
+    const std::int64_t end = std::min(a.end_sample, b.end_sample);
+    return end > begin ? end - begin : 0;
+}
+
+// Joins a segment's words into that segment's text.
+//
+// THE SEPARATOR IS NOT ALWAYS A SPACE, and getting it wrong is visible to every
+// caller rather than cosmetic: core/http/endpoints/openai/transcription.go
+// routes response_format text, srt, vtt and lrc through
+// schema.TranscriptionResponse, which builds the entire body out of
+// Segments[].Text and never reads the top-level text. For those four formats
+// the segment text IS the response.
+//
+// Two producer conventions have to be told apart:
+//
+//   whole words    "Some", "call", "me"   -> join with a space
+//   subword pieces "So", "me", " call"    -> concatenate
+//
+// The second is SentencePiece, where a word boundary is carried as a LEADING
+// SPACE on the piece; nemotron_asr emits one entry per token in exactly that
+// form. Space-joining those produced "So me  call   me  na ture ,", which is
+// what response_format=text returned while the correct sentence sat unread in
+// the top-level field. Concatenating them reproduces text_output exactly.
+//
+// The convention is read off the words themselves, because nothing else in the
+// result declares it. One leading space anywhere is enough to decide: a
+// whole-word producer has no reason to emit one, and a subword producer emits
+// one at every word boundary, so the two populations do not overlap. A producer
+// that mixed both conventions inside one segment could not be served correctly
+// by any single separator; this picks concatenation for it.
+//
+// This does NOT touch the top-level text, which stays text_output verbatim. The
+// rule that forbids deriving the transcript from the segments is about the
+// direction segments -> text. Segment text has no source other than its words
+// and is necessarily derived.
+std::string join_words(const std::vector<OutWord> &words) {
+    const bool subword_pieces =
+        std::any_of(words.begin(), words.end(), [](const OutWord &word) {
+            return !word.text.empty() && word.text.front() == ' ';
+        });
+    std::string out;
+    for (const auto &word : words) {
+        if (word.text.empty()) {
+            continue;
+        }
+        if (!subword_pieces && !out.empty()) {
+            out += " ";
+        }
+        out += word.text;
+    }
+    return out;
+}
+
+std::string speaker_for(const Span &segment,
+                        const std::vector<SpeakerSpan> &turns) {
+    std::string best;
+    std::int64_t best_overlap = 0;
+    for (const auto &turn : turns) {
+        const std::int64_t shared = overlap(segment, turn.span);
+        if (shared > best_overlap) {
+            best_overlap = shared;
+            best = turn.speaker;
+        }
+    }
+    return best;
+}
+
+// The chosen segmentation. labels is empty unless the spans were sourced from
+// the speaker turns themselves, in which case it is parallel to spans and holds
+// the label each span arrived with.
+struct SegmentSource {
+    std::vector<Span> spans;
+    std::vector<std::string> labels;
+};
+
+// Chooses the segment spans, per the documented precedence.
+SegmentSource choose_segment_spans(const std::string &text_output,
+                                   const std::vector<Span> &speech_segments,
+                                   const std::vector<SpeakerSpan> &speaker_turns,
+                                   const std::vector<WordSpan> &words) {
+    if (!speech_segments.empty()) {
+        return {speech_segments, {}};
+    }
+    if (!speaker_turns.empty()) {
+        // The labels are carried out rather than re-derived by overlap later. A
+        // turn wholly contained in another speaker's turn overlaps its own span
+        // completely, which is the largest overlap possible, so it can only tie
+        // with the containing turn and would then lose that tie on order.
+        // sortformer_diar binarizes each speaker's track independently and
+        // sorts the result by start sample, so the container always comes
+        // first, and the interjecting speaker would be silently relabelled to
+        // the speaker it interrupted.
+        SegmentSource source;
+        source.spans.reserve(speaker_turns.size());
+        source.labels.reserve(speaker_turns.size());
+        for (const auto &turn : speaker_turns) {
+            source.spans.push_back(turn.span);
+            source.labels.push_back(turn.speaker);
+        }
+        return source;
+    }
+    if (!words.empty()) {
+        Span covering = words.front().span;
+        for (const auto &word : words) {
+            covering.start_sample =
+                std::min(covering.start_sample, word.span.start_sample);
+            covering.end_sample = std::max(covering.end_sample, word.span.end_sample);
+        }
+        return {{covering}, {}};
+    }
+    if (!text_output.empty()) {
+        // A zero span rather than a fabricated duration: the model reported no
+        // timing, and inventing one would be a lie the caller cannot detect.
+        return {{Span{0, 0}}, {}};
+    }
+    return {};
+}
+
+// Returns the index of the segment a word belongs to, or the nearest segment
+// when the word falls outside all of them.
+size_t segment_index_for_word(const std::vector<Span> &spans, const Span &word) {
+    const std::int64_t centre = midpoint(word);
+    for (size_t i = 0; i < spans.size(); ++i) {
+        if (contains(spans[i], centre)) {
+            return i;
+        }
+    }
+    size_t nearest = 0;
+    std::int64_t best_distance = std::numeric_limits<std::int64_t>::max();
+    for (size_t i = 0; i < spans.size(); ++i) {
+        const std::int64_t distance = std::llabs(midpoint(spans[i]) - centre);
+        if (distance < best_distance) {
+            best_distance = distance;
+            nearest = i;
+        }
+    }
+    return nearest;
+}
+
+} // namespace
+
+AssembledTranscript assemble_transcript(const std::string &text_output,
+                                        const std::vector<Span> &speech_segments,
+                                        const std::vector<SpeakerSpan> &speaker_turns,
+                                        const std::vector<WordSpan> &words,
+                                        int sample_rate) {
+    AssembledTranscript assembled;
+    // THE RULE. Never derived from spans.
+    assembled.text = text_output;
+
+    const SegmentSource source =
+        choose_segment_spans(text_output, speech_segments, speaker_turns, words);
+    const std::vector<Span> &spans = source.spans;
+    if (spans.empty()) {
+        return assembled;
+    }
+
+    assembled.segments.resize(spans.size());
+    for (size_t i = 0; i < spans.size(); ++i) {
+        OutSegment &segment = assembled.segments[i];
+        segment.id = static_cast<int>(i);
+        segment.start_ns = samples_to_nanoseconds(spans[i].start_sample, sample_rate);
+        segment.end_ns = samples_to_nanoseconds(spans[i].end_sample, sample_rate);
+        // A segment that came from a speaker turn already knows its speaker.
+        // Only the other three sources have to look one up by overlap.
+        segment.speaker = source.labels.empty()
+                              ? speaker_for(spans[i], speaker_turns)
+                              : source.labels[i];
+    }
+
+    for (const auto &word : words) {
+        const size_t index = segment_index_for_word(spans, word.span);
+        OutWord out;
+        out.start_ns = samples_to_nanoseconds(word.span.start_sample, sample_rate);
+        out.end_ns = samples_to_nanoseconds(word.span.end_sample, sample_rate);
+        out.text = word.word;
+        assembled.segments[index].words.push_back(out);
+    }
+
+    for (auto &segment : assembled.segments) {
+        segment.text = join_words(segment.words);
+    }
+
+    // A single segment with no word timing carries the whole transcript. With
+    // several segments there is no defensible way to split the text, so their
+    // per-segment text stays empty and only the top-level text is authoritative.
+    if (assembled.segments.size() == 1 && assembled.segments[0].words.empty()) {
+        assembled.segments[0].text = text_output;
+    }
+
+    return assembled;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/transcript_assembly.h

@@ -0,0 +1,87 @@
+#pragma once
+
+// Builds LocalAI's TranscriptResult shape from audio.cpp's TaskResult spans.
+// Standard library only; result_map.cpp converts the engine types into these
+// PODs at the boundary.
+//
+// THE RULE: the top-level transcript text is text_output verbatim, always.
+// audio.cpp carries transcript text in exactly one place, TaskResult.text_output.
+// speech_segments, speaker_turns and word_timestamps carry spans and labels but
+// no text. Deriving the top-level text by concatenating per-segment text
+// therefore yields an empty transcript for every producer that reports segments
+// without word timestamps, which includes VibeVoice diarized ASR.
+
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Sample-index span, mirroring engine::runtime::TimeSpan.
+struct Span {
+    std::int64_t start_sample = 0;
+    std::int64_t end_sample = 0;
+};
+
+struct WordSpan {
+    Span span;
+    std::string word;
+};
+
+struct SpeakerSpan {
+    Span span;
+    std::string speaker;
+};
+
+struct OutWord {
+    std::int64_t start_ns = 0;
+    std::int64_t end_ns = 0;
+    std::string text;
+};
+
+struct OutSegment {
+    int id = 0;
+    std::int64_t start_ns = 0;
+    std::int64_t end_ns = 0;
+    std::string text;
+    std::string speaker;
+    std::vector<OutWord> words;
+};
+
+struct AssembledTranscript {
+    std::string text;
+    std::vector<OutSegment> segments;
+};
+
+// Segment source, first non-empty wins:
+//   1. speech_segments
+//   2. speaker_turns
+//   3. one segment spanning all words, when words are present
+//   4. one zero-span segment carrying the full text, when text is present
+//   5. no segments
+//
+// Words attach to the segment whose range contains their midpoint; a word
+// outside every segment attaches to the nearest one by midpoint distance so it
+// is never silently dropped. A lone segment with no words carries the full text.
+//
+// A segment's text is its words joined, and the separator depends on the
+// producer's convention: whole words ("Some", "call") are joined with a space,
+// while SentencePiece-style subword pieces, which carry the word boundary as a
+// LEADING SPACE (" call"), are concatenated. One leading space anywhere in the
+// segment selects concatenation. This matters beyond tidiness: response_format
+// text, srt, vtt and lrc build their entire body out of the segment text and
+// never read the top-level text.
+//
+// A segment's speaker is the speaker turn with the greatest overlap, except
+// when the segments came from the speaker turns themselves (source 2), where
+// each segment keeps its own turn's label. Re-deriving it there loses a turn
+// nested inside another speaker's turn: the nested turn overlaps its own span
+// completely, so it can only tie with the containing turn, which is listed
+// first and wins the tie.
+AssembledTranscript assemble_transcript(const std::string &text_output,
+                                        const std::vector<Span> &speech_segments,
+                                        const std::vector<SpeakerSpan> &speaker_turns,
+                                        const std::vector<WordSpan> &words,
+                                        int sample_rate);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/transcript_assembly_test.cpp

@@ -0,0 +1,532 @@
+// Unit tests for transcript_assembly. Standard library only. The harness
+// compiles this as a single translation unit, so both implementations are
+// included directly rather than linked.
+//
+// Every fixture below either mirrors a producer shape actually observed from
+// audio.cpp families, and names the families it was checked against, or says in
+// its own comment that it is defensive. Do not replace an observed shape with an
+// invented one and do not quietly promote a defensive fixture to an observed
+// one: an invented shape is what let the earlier attempt ship an empty
+// transcript.
+
+#include "audio_units.cpp"
+#include "transcript_assembly.cpp"
+
+#include <cstddef>
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// Indexed access that reports a named failure instead of running off the end.
+// std::vector::operator[] past the end is undefined behaviour, so a regression
+// that drops a segment would crash the process here and take every later check
+// with it. Returning a default element keeps the rest of the suite reporting.
+static const OutSegment &segment_at(const AssembledTranscript &out, size_t index,
+                                    const std::string &name) {
+    static const OutSegment missing;
+    if (index >= out.segments.size()) {
+        failures++;
+        fprintf(stderr, "FAIL: %s (segment %zu is missing)\n", name.c_str(), index);
+        return missing;
+    }
+    return out.segments[index];
+}
+
+static const OutWord &word_at(const OutSegment &segment, size_t index,
+                              const std::string &name) {
+    static const OutWord missing;
+    if (index >= segment.words.size()) {
+        failures++;
+        fprintf(stderr, "FAIL: %s (word %zu is missing)\n", name.c_str(), index);
+        return missing;
+    }
+    return segment.words[index];
+}
+
+static const int kRate = 16000;
+
+// Shape A: word timestamps only. Emitted by nemotron_asr, qwen3_asr and
+// qwen3_forced_aligner, all of which set text_output plus word_timestamps and
+// leave speech_segments empty.
+static void test_words_only() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "hello"},
+        {{8000, 16000}, "world"},
+    };
+    const auto out = assemble_transcript("hello world", {}, {}, words, kRate);
+
+    check(out.text == "hello world", "text is text_output verbatim");
+    check(out.segments.size() == 1, "words with no segments yield one segment");
+    const OutSegment &first = segment_at(out, 0, "words only segment");
+    check(first.start_ns == 0, "segment starts at the first word");
+    check(first.end_ns == 1000000000LL, "segment ends at the last word");
+    check(first.words.size() == 2, "both words attached");
+    check(word_at(first, 0, "first word").text == "hello", "first word text");
+    check(word_at(first, 1, "second word").start_ns == 500000000LL,
+          "second word start in ns");
+    check(first.text == "hello world", "segment text joins its words");
+    check(first.id == 0, "ids are zero based");
+}
+
+// Shape A-whole: the whole-word convention, stated explicitly rather than left
+// implicit in the shape A tests. qwen3_forced_aligner emits one entry per WORD
+// (processor.cpp parses per-word timestamp tokens), so its pieces carry no
+// leading space and must be joined with one.
+static void test_whole_words_are_space_joined() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "Some"},
+        {{8000, 16000}, "call"},
+        {{16000, 24000}, "me"},
+    };
+    const auto out = assemble_transcript("Some call me", {}, {}, words, kRate);
+
+    check(segment_at(out, 0, "whole words").text == "Some call me",
+          "whole words are joined with a single space");
+}
+
+// Shape A-subword: the SentencePiece convention, where the word boundary is a
+// LEADING SPACE on the piece. These are the first eleven word_timestamps
+// nemotron_asr actually returned for audio.cpp/assets/resources/sample_16k.wav
+// with the q8_0 GGUF, copied verbatim rather than invented, including the lone
+// " " piece at index 3.
+//
+// Space-joining these produced "So me  call   me  na ture ,  other s  call",
+// which is not a cosmetic problem: response_format text, srt, vtt and lrc build
+// their entire body from the segment text and never read the top-level text, so
+// that string WAS the transcription response for those formats.
+static void test_subword_pieces_are_concatenated() {
+    const std::vector<WordSpan> words = {
+        {{15360, 16640}, "So"},   {{15360, 16640}, "me"},
+        {{23040, 24320}, " call"}, {{28160, 29440}, " "},
+        {{28160, 29440}, "me"},   {{30720, 32000}, " na"},
+        {{33280, 34560}, "ture"}, {{35840, 37120}, ","},
+        {{38400, 39680}, " other"}, {{40960, 42240}, "s"},
+        {{43520, 44800}, " call"},
+    };
+    const auto out = assemble_transcript(
+        "Some call me nature, others call me mother nature.", {}, {}, words, kRate);
+
+    check(out.text == "Some call me nature, others call me mother nature.",
+          "the top-level text is still text_output verbatim");
+    check(segment_at(out, 0, "subword pieces").text ==
+              "Some call me nature, others call",
+          "subword pieces are concatenated, reproducing text_output");
+}
+
+// One leading space anywhere decides for the whole segment. A subword producer
+// emits a boundary space at every word start, so its first piece, which is
+// sentence-initial, does not have one; keying off the first piece alone would
+// therefore pick the wrong convention on every segment.
+static void test_a_single_leading_space_selects_concatenation() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "al"},
+        {{8000, 16000}, "pha"},
+        {{16000, 24000}, " beta"},
+    };
+    const auto out = assemble_transcript("alpha beta", {}, {}, words, kRate);
+
+    check(segment_at(out, 0, "mixed").text == "alpha beta",
+          "a leading space on a later piece selects concatenation");
+}
+
+// Shape A': the same producer, but text_output is punctuated and cased while
+// the word timestamps are not. qwen3_asr rebuilds text_output from its word
+// list only when timestamps are requested, so the two genuinely differ; this
+// pins the sole segment's text to its words rather than to the top-level text.
+static void test_words_only_with_punctuated_text_output() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "hello"},
+        {{8000, 16000}, "world"},
+    };
+    const auto out = assemble_transcript("Hello, world!", {}, {}, words, kRate);
+
+    check(out.text == "Hello, world!", "punctuated text_output is untouched");
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "punctuated segment").text == "hello world",
+          "a segment with words takes its text from the words, not text_output");
+}
+
+// Shape A'': a merged word list whose last word is not the one that ends
+// latest. audio.cpp concatenates per-chunk word lists in chunk order
+// (append_chunk_word_timestamps in framework/audio/chunking.cpp). It drops a
+// word whose global start falls before the chunk's keep span, but it never
+// clips a word's end to that boundary, so the last word kept from one chunk can
+// outlast the first word kept from the next. The covering span must therefore
+// be the extent of every word, not the span from the first to the last.
+static void test_covering_span_spans_every_word() {
+    const std::vector<WordSpan> words = {
+        {{0, 4000}, "a"},
+        // Kept from the earlier chunk, ending past the chunk boundary.
+        {{4000, 10000}, "b"},
+        // First word of the next chunk, shorter, so it ends earlier.
+        {{8000, 9000}, "c"},
+    };
+    const auto out = assemble_transcript("a b c", {}, {}, words, kRate);
+
+    check(out.segments.size() == 1, "one covering segment");
+    check(segment_at(out, 0, "covering segment").end_ns == 625000000LL,
+          "the covering span reaches the latest word end, not the last word's");
+}
+
+// Defensive, not observed: no pinned family emits a word with no text.
+// nemotron_asr's build_token_timestamps (models/nemotron_asr/decoder.cpp:97)
+// skips a token that decodes to an empty chunk before it ever becomes a
+// WordTimestamp. join_words guards against one anyway, and an unexercised guard
+// is a guard the next reader deletes as dead weight.
+static void test_empty_word_contributes_no_separator() {
+    const std::vector<WordSpan> words = {
+        {{0, 4000}, "alpha"},
+        {{4000, 8000}, ""},
+        {{8000, 12000}, "beta"},
+    };
+    const auto out = assemble_transcript("alpha beta", {}, {}, words, kRate);
+
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "sole segment").text == "alpha beta",
+          "an empty word adds no separator to the segment text");
+    check(segment_at(out, 0, "sole segment").words.size() == 3,
+          "the empty word still reports its span");
+}
+
+// Shape B: speech segments, no words. Emitted by ASR families that report
+// utterance boundaries without word-level timing.
+static void test_segments_without_words() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const auto out = assemble_transcript("one two three", segments, {}, {}, kRate);
+
+    // The regression: this must NOT be empty.
+    check(out.text == "one two three", "multi-segment text is not empty");
+    check(out.segments.size() == 2, "both segments survive");
+    check(segment_at(out, 0, "first segment").end_ns == 1000000000LL,
+          "first segment ends at 1s");
+    check(segment_at(out, 1, "second segment").start_ns == 1000000000LL,
+          "second segment starts at 1s");
+    check(segment_at(out, 0, "first segment").text.empty(),
+          "per-segment text stays empty when there are no words to split by");
+    check(segment_at(out, 1, "second segment").id == 1, "ids increment");
+}
+
+// Shape C: speech segments plus speaker turns, no words. This is the real
+// VibeVoice diarized ASR shape that broke the earlier attempt.
+static void test_segments_with_speaker_turns_no_words() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 16000}, "SPEAKER_00"},
+        {{16000, 32000}, "SPEAKER_01"},
+    };
+    const auto out = assemble_transcript("hi there", segments, turns, {}, kRate);
+
+    check(out.text == "hi there", "diarized multi-segment text is not empty");
+    check(out.segments.size() == 2, "two segments");
+    check(segment_at(out, 0, "first diarized segment").speaker == "SPEAKER_00",
+          "first speaker assigned");
+    check(segment_at(out, 1, "second diarized segment").speaker == "SPEAKER_01",
+          "second speaker assigned");
+}
+
+// Defensive, not observed: speech segments and speaker turns that disagree.
+// vibevoice_asr builds each SpeakerTurn with turn.span = speech_segment.span in
+// one loop (models/vibevoice_asr/session.cpp:965) and shifts and clips both
+// lists identically when merging chunks, so in practice the two lists are 1:1
+// with identical spans. That is exactly why the shape C fixture above cannot
+// show which list is the segment source: swapping the precedence there produces
+// byte-identical output. This fixture pins the precedence, and it is the shape
+// any future family that segments and diarizes separately would produce.
+static void test_speech_segments_outrank_speaker_turns() {
+    const std::vector<Span> segments = {{0, 32000}};
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 16000}, "SPEAKER_00"},
+        {{16000, 32000}, "SPEAKER_01"},
+    };
+    const auto out = assemble_transcript("hi there", segments, turns, {}, kRate);
+
+    check(out.segments.size() == 1,
+          "speech segments decide the segmentation, not speaker turns");
+    check(segment_at(out, 0, "single utterance").end_ns == 2000000000LL,
+          "the utterance keeps its own span");
+}
+
+// Defensive, not observed: no pinned family emits speaker turns and word
+// timestamps together. It pins rule 2 against rule 3, which nothing else does:
+// a diarized result is segmented by who spoke, and words only fill the turns in.
+static void test_speaker_turns_outrank_words() {
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 16000}, "SPEAKER_00"},
+        {{16000, 32000}, "SPEAKER_01"},
+    };
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "hi"},
+        {{16000, 24000}, "there"},
+    };
+    const auto out = assemble_transcript("hi there", {}, turns, words, kRate);
+
+    check(out.segments.size() == 2, "the two turns segment the result");
+    check(segment_at(out, 0, "turn 0").text == "hi", "first turn takes its word");
+    check(segment_at(out, 1, "turn 1").text == "there", "second turn takes its word");
+}
+
+// Shape D: text only. Emitted by ASR families that report no timing at all,
+// such as hviske_asr and citrinet_asr.
+static void test_text_only() {
+    const auto out = assemble_transcript("just text", {}, {}, {}, kRate);
+
+    check(out.text == "just text", "text survives");
+    check(out.segments.size() == 1, "a single synthetic segment is emitted");
+    const OutSegment &only = segment_at(out, 0, "synthetic segment");
+    check(only.start_ns == 0 && only.end_ns == 0,
+          "synthetic segment has zero span, not a fabricated duration");
+    check(only.text == "just text", "the sole segment carries the full text");
+}
+
+// Shape E: speaker turns only, no speech segments and no text. This is
+// sortformer_diar, reached through the Diarize RPC.
+static void test_speaker_turns_only() {
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 24000}, "0"},
+        {{24000, 48000}, "1"},
+    };
+    const auto out = assemble_transcript("", {}, turns, {}, kRate);
+
+    check(out.text.empty(), "no text is reported when the model produced none");
+    check(out.segments.size() == 2, "turns become segments");
+    check(segment_at(out, 0, "turn 0").speaker == "0",
+          "speaker label preserved verbatim");
+    check(segment_at(out, 1, "turn 1").start_ns == 1500000000LL,
+          "second turn starts at 1.5s");
+}
+
+// Shape E', the same producer with one speaker talking over another.
+// decode_sortformer_speaker_turns (models/sortformer_diar/postprocess.cpp)
+// binarizes each speaker's probability track independently, which is the whole
+// point of sortformer, then sorts the turns by start sample. So a turn can be
+// wholly contained in another speaker's turn, and the containing turn always
+// comes first. A segment sourced from a speaker turn must keep that turn's own
+// label: re-deriving it by overlap can only ever tie with the containing turn,
+// which then wins on order and silently erases the interjecting speaker.
+static void test_nested_speaker_turn_keeps_its_own_label() {
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 100000}, "speaker_0"},
+        {{10000, 20000}, "speaker_1"},
+    };
+    const auto out = assemble_transcript("", {}, turns, {}, kRate);
+
+    check(out.segments.size() == 2, "both turns become segments");
+    check(segment_at(out, 0, "containing turn").speaker == "speaker_0",
+          "the containing turn keeps its label");
+    check(segment_at(out, 1, "nested turn").speaker == "speaker_1",
+          "a turn nested inside another is not relabelled to the container");
+}
+
+// Shape F: nothing at all. A model that ran but produced no output must not
+// crash or fabricate a segment.
+static void test_empty() {
+    const auto out = assemble_transcript("", {}, {}, {}, kRate);
+    check(out.text.empty(), "empty stays empty");
+    check(out.segments.empty(), "no segments are invented");
+}
+
+// Shape H: speech segments with no text and no words at all. This is the VAD
+// path, silero_vad and marblenet_vad, which fill speech_segments and never
+// touch text_output. It reaches the lone-segment rule with nothing to carry.
+static void test_vad_segments_without_text() {
+    const std::vector<Span> segments = {{0, 16000}, {24000, 32000}};
+    const auto out = assemble_transcript("", segments, {}, {}, kRate);
+
+    check(out.text.empty(), "VAD reports no text");
+    check(out.segments.size() == 2, "both speech regions survive");
+    check(segment_at(out, 1, "second speech region").start_ns == 1500000000LL,
+          "second region starts at 1.5s");
+    check(segment_at(out, 0, "first speech region").text.empty(),
+          "a VAD segment carries no text");
+
+    const std::vector<Span> one = {{0, 16000}};
+    const auto single = assemble_transcript("", one, {}, {}, kRate);
+    check(single.segments.size() == 1, "a single speech region survives");
+    check(segment_at(single, 0, "lone speech region").text.empty(),
+          "a lone VAD segment does not fabricate text");
+}
+
+// Shape G: segments and words together. Words are assigned by midpoint so a
+// word straddling a boundary lands in exactly one segment.
+static void test_words_distributed_into_segments() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const std::vector<WordSpan> words = {
+        {{0, 4000}, "alpha"},
+        {{4000, 8000}, "beta"},
+        // Straddles the boundary; midpoint 16000 falls in the second segment.
+        {{12000, 20000}, "gamma"},
+        {{20000, 28000}, "delta"},
+    };
+    const auto out = assemble_transcript("alpha beta gamma delta", segments, {},
+                                         words, kRate);
+
+    check(out.text == "alpha beta gamma delta", "top level text unchanged");
+    check(out.segments.size() == 2, "two segments");
+    check(segment_at(out, 0, "first segment").words.size() == 2,
+          "first segment takes two words");
+    check(segment_at(out, 1, "second segment").words.size() == 2,
+          "second segment takes two words");
+    check(segment_at(out, 0, "first segment").text == "alpha beta",
+          "first segment text");
+    check(segment_at(out, 1, "second segment").text == "gamma delta",
+          "boundary-straddling word lands by midpoint");
+}
+
+// The midpoint rule is not the same as either endpoint rule. "early" starts in
+// the first segment but ends in the second, and "late" the other way round;
+// each must land where its midpoint says, which no start-only or end-only rule
+// reproduces.
+static void test_words_assigned_by_midpoint_not_endpoint() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const std::vector<WordSpan> words = {
+        // Midpoint 12000 -> first segment, although it ends in the second.
+        {{4000, 20000}, "early"},
+        // Midpoint 20000 -> second segment, although it starts in the first.
+        {{12000, 28000}, "late"},
+    };
+    const auto out = assemble_transcript("early late", segments, {}, words, kRate);
+
+    check(segment_at(out, 0, "first segment").text == "early",
+          "a word ending past the boundary stays where its midpoint is");
+    check(segment_at(out, 1, "second segment").text == "late",
+          "a word starting before the boundary follows its midpoint");
+}
+
+// A word outside every segment must still be reachable rather than dropped
+// silently, so it attaches to the nearest segment by midpoint distance.
+static void test_word_outside_all_segments() {
+    const std::vector<Span> segments = {{0, 16000}};
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "inside"},
+        {{40000, 48000}, "outside"},
+    };
+    const auto out = assemble_transcript("inside outside", segments, {}, words,
+                                         kRate);
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "sole segment").words.size() == 2,
+          "the stray word is not dropped");
+}
+
+// The fallback picks the nearest segment, which is not the same as picking the
+// first. With one segment the two are indistinguishable, so this uses three and
+// puts the stray word past the last one.
+static void test_stray_word_goes_to_the_nearest_segment() {
+    const std::vector<Span> segments = {{0, 8000}, {8000, 16000}, {16000, 24000}};
+    const std::vector<WordSpan> words = {
+        // Midpoint 44000, nearest the third segment.
+        {{40000, 48000}, "trailing"},
+    };
+    const auto out = assemble_transcript("trailing", segments, {}, words, kRate);
+
+    check(segment_at(out, 0, "first segment").words.empty(),
+          "the stray word does not fall back to the first segment");
+    check(segment_at(out, 2, "third segment").text == "trailing",
+          "the stray word attaches to the nearest segment");
+}
+
+// "Nearest" is measured from the segment's midpoint, and it is neither "the
+// first segment" nor "the last". A leading stray word is the case a
+// trailing-only fixture cannot reach: forced-aligner words scored against VAD
+// segments produce one, and with only trailing coverage it would land at the
+// end of the transcript with the suite green. Here the leading word's nearest
+// midpoint is the first segment while its nearest start is the second, and the
+// trailing word's nearest midpoint is the third while its nearest end is the
+// second, so no endpoint rule reproduces this assignment either.
+static void test_stray_word_distance_is_measured_from_the_midpoint() {
+    const std::vector<Span> segments = {{0, 2000}, {8000, 200000}, {300000, 302000}};
+    const std::vector<WordSpan> words = {
+        {{4000, 6000}, "lead"},
+        {{249000, 251000}, "trail"},
+    };
+    const auto out = assemble_transcript("lead trail", segments, {}, words, kRate);
+
+    check(out.segments.size() == 3, "three segments");
+    check(segment_at(out, 0, "first segment").text == "lead",
+          "the leading stray word goes to the nearest segment by midpoint");
+    check(segment_at(out, 2, "third segment").text == "trail",
+          "the trailing stray word goes to the nearest segment by midpoint");
+    check(segment_at(out, 1, "middle segment").words.empty(),
+          "the long middle segment claims neither stray word");
+}
+
+// Speaker assignment uses greatest overlap, not first match, so a turn that
+// barely touches a segment does not win over one that covers it.
+static void test_speaker_assigned_by_greatest_overlap() {
+    const std::vector<Span> segments = {{8000, 24000}};
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 9000}, "brief"},   // overlaps 1000 samples
+        {{9000, 24000}, "main"} // overlaps 15000 samples
+    };
+    const auto out = assemble_transcript("x", segments, turns, {}, kRate);
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "sole segment").speaker == "main",
+          "greatest overlap wins");
+}
+
+// A segment no turn touches gets no speaker rather than the label of whichever
+// turn happened to be listed first.
+static void test_segment_without_any_overlapping_turn_has_no_speaker() {
+    const std::vector<Span> segments = {{0, 8000}, {40000, 48000}};
+    const std::vector<SpeakerSpan> turns = {{{0, 8000}, "SPEAKER_00"}};
+    const auto out = assemble_transcript("x", segments, turns, {}, kRate);
+
+    check(segment_at(out, 0, "overlapped segment").speaker == "SPEAKER_00",
+          "the overlapped segment is labelled");
+    check(segment_at(out, 1, "unlabelled segment").speaker.empty(),
+          "a segment no turn overlaps is left unlabelled");
+}
+
+static void test_zero_sample_rate_is_safe() {
+    const std::vector<Span> segments = {{0, 16000}};
+    const auto out = assemble_transcript("x", segments, {}, {}, 0);
+    check(out.segments.size() == 1, "a zero sample rate still yields the segment");
+    const OutSegment &only = segment_at(out, 0, "sole segment");
+    check(only.start_ns == 0 && only.end_ns == 0,
+          "unknown sample rate yields zero timings rather than garbage");
+}
+
+int main() {
+    test_words_only();
+    test_whole_words_are_space_joined();
+    test_subword_pieces_are_concatenated();
+    test_a_single_leading_space_selects_concatenation();
+    test_words_only_with_punctuated_text_output();
+    test_covering_span_spans_every_word();
+    test_empty_word_contributes_no_separator();
+    test_segments_without_words();
+    test_segments_with_speaker_turns_no_words();
+    test_speech_segments_outrank_speaker_turns();
+    test_speaker_turns_outrank_words();
+    test_text_only();
+    test_speaker_turns_only();
+    test_nested_speaker_turn_keeps_its_own_label();
+    test_empty();
+    test_vad_segments_without_text();
+    test_words_distributed_into_segments();
+    test_words_assigned_by_midpoint_not_endpoint();
+    test_word_outside_all_segments();
+    test_stray_word_goes_to_the_nearest_segment();
+    test_stray_word_distance_is_measured_from_the_midpoint();
+    test_speaker_assigned_by_greatest_overlap();
+    test_segment_without_any_overlapping_turn_has_no_speaker();
+    test_zero_sample_rate_is_safe();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all transcript_assembly checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/upstream_absence_ctest.cpp

@@ -0,0 +1,283 @@
+// Asserts the ABSENCES that capability_routing.cpp's five refusal reasons rest
+// on, against the engine itself rather than against somebody's reading of it.
+//
+// Those reasons are prose making checkable claims about a pinned third-party
+// checkout: "VoiceTaskKind has no codec entry", "no family advertises spk",
+// "miocodec advertises only vc and s2s". Prose rots silently across an
+// AUDIO_CPP_VERSION bump, and it rots in the worst possible place, since a
+// refusal that states a false fact is worse than a bare UNIMPLEMENTED: it will
+// be believed. One of the four claims this backend was planned against was
+// already false when it was written ("streaming exists for tts and asr only" is
+// contradicted by silero_vad). This test is what turns the next such change
+// from a silent lie on the wire into a build failure.
+//
+// Pinned at audio.cpp e800d435d130dc776baf6f3e6129bb62b1495c89. What follows
+// was true of that commit; a bump is exactly when it needs to be re-run.
+//
+// It links engine_runtime and queries make_default_registry(), touching only
+// include/engine/framework/**, like every other unit in this backend. It loads
+// no model and reads no file: advertise_loaders() is the path-free catalog
+// upstream publishes for --list-loaders.
+//
+// FOUR CAVEATS, so nobody reads more into a green run than it earns:
+//
+//   1. It cannot assert the AudioToAudioStream reason. That one contrasts
+//      LocalAI's OpenAI-Realtime contract (conversation, system prompt, tool
+//      loop) with what audio.cpp's s2s families actually do, and semantics are
+//      not a queryable property. What IS asserted is the enumerable half: that
+//      s2s is advertised by exactly miocodec and vevo2, which is the clause the
+//      message names by hand.
+//   2. It queries the LOADER catalog, while grpc-server.cpp reads the LOADED
+//      model's own capabilities(). The two agree today, cross-checked on the
+//      wire: this test asserts miocodec advertises {vc/offline, s2s/offline},
+//      and a live LoadModel of miocodec-q8_0.gguf reports exactly
+//      "vc/offline, s2s/offline". A family whose loaded capabilities diverged
+//      from its advertisement would slip past, but nothing loads without a
+//      model file and a ctest cannot depend on one.
+//   3. Capabilities need not come from a loader at all.
+//      src/framework/model_spec/metadata.cpp's advertised_capabilities() builds
+//      a CapabilitySet from a spec's "capabilities"/"tasks"/"modes" keys, which
+//      is a second route by which a bump could falsify claim 3. Today no
+//      shipped model_specs/*.json carries a top-level "tasks" key and no loader
+//      calls that function, so the route is dead. It is covered anyway to the
+//      extent that a loader adopting it would surface through advertise_loaders
+//      like any other capability, which is why every assertion below queries
+//      advertised capabilities rather than loader source.
+//   4. An absence test passes trivially when the query is broken, so
+//      test_catalog_is_populated below is a POSITIVE control and is not
+//      optional. It proves the registry is non-empty and that the query does
+//      find a capability it should, before any absence is believed.
+
+#include "engine/framework/runtime/registry.h"
+#include "engine/framework/runtime/session.h"
+
+#include <algorithm>
+#include <cstdio>
+#include <set>
+#include <stdexcept>
+#include <string>
+#include <vector>
+
+namespace {
+
+int failures = 0;
+
+void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using engine::runtime::LoaderAdvertisement;
+using engine::runtime::RunMode;
+using engine::runtime::VoiceTaskKind;
+
+// Built once. make_default_registry constructs every loader, which is the
+// expensive part, and none of the assertions mutates it.
+const std::vector<LoaderAdvertisement> &catalog() {
+    static const std::vector<LoaderAdvertisement> kCatalog =
+        engine::runtime::make_default_registry().advertise_loaders();
+    return kCatalog;
+}
+
+bool advertises(const LoaderAdvertisement &loader, VoiceTaskKind task,
+                RunMode mode) {
+    for (const auto &capability : loader.capabilities.supported_tasks) {
+        if (capability.task != task) {
+            continue;
+        }
+        return std::find(capability.modes.begin(), capability.modes.end(), mode) !=
+               capability.modes.end();
+    }
+    return false;
+}
+
+bool advertises_any_mode(const LoaderAdvertisement &loader, VoiceTaskKind task) {
+    return advertises(loader, task, RunMode::Offline) ||
+           advertises(loader, task, RunMode::Streaming);
+}
+
+std::string describe(const LoaderAdvertisement &loader) {
+    std::string out;
+    for (const auto &capability : loader.capabilities.supported_tasks) {
+        for (const RunMode mode : capability.modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += engine::runtime::to_string(capability.task);
+            out += "/";
+            out += engine::runtime::to_string(mode);
+        }
+    }
+    return out.empty() ? "nothing" : out;
+}
+
+// THE POSITIVE CONTROL. Every other test here asserts an absence, and an
+// absence is what a broken query returns for everything. If make_default_registry
+// ever returns an empty registry, or advertise_loaders stops populating modes,
+// this is the test that fails instead of the suite going quietly green while
+// asserting nothing.
+void test_catalog_is_populated() {
+    check(catalog().size() >= 20,
+          "the default registry advertises a plausible number of families (" +
+              std::to_string(catalog().size()) + ")");
+
+    bool found_streaming_asr = false;
+    for (const auto &loader : catalog()) {
+        if (advertises(loader, VoiceTaskKind::Asr, RunMode::Streaming)) {
+            found_streaming_asr = true;
+            break;
+        }
+    }
+    check(found_streaming_asr,
+          "the query finds a capability that IS advertised (asr/streaming)");
+}
+
+// The AudioEncode and AudioDecode premise. Not "no family does codec" but the
+// stronger "the task kind does not exist", which is what makes those two RPCs
+// unroutable rather than merely unserved.
+//
+// Note what is NOT asserted: model_spec/schema.cpp's task whitelist DOES accept
+// the string "codec", so a spec declaring it validates and then fails here. That
+// is a hole in upstream's own validation, and it is deliberately kept off the
+// wire; the refusal rests on this parser, which is the thing routing would have
+// to go through.
+void test_no_codec_task_kind() {
+    bool threw = false;
+    try {
+        (void)engine::runtime::parse_voice_task_kind("codec");
+    } catch (const std::exception &) {
+        threw = true;
+    }
+    check(threw, "no codec task kind: parse_voice_task_kind(\"codec\") throws");
+
+    // The control for the line above: a real name must NOT throw, or the test
+    // would pass against a parser that rejected everything.
+    bool tts_threw = false;
+    try {
+        (void)engine::runtime::parse_voice_task_kind("tts");
+    } catch (const std::exception &) {
+        tts_threw = true;
+    }
+    check(!tts_threw, "parse_voice_task_kind accepts a real name (\"tts\")");
+}
+
+// The VoiceEmbed premise. SpeakerRecognition IS in the enum, and TitaNet and
+// ECAPA-TDNN exist as internal conditioning encoders; what is missing is any
+// registered family advertising the task, which is what the message says.
+void test_no_family_advertises_speaker_recognition() {
+    std::string offenders;
+    for (const auto &loader : catalog()) {
+        if (advertises_any_mode(loader, VoiceTaskKind::SpeakerRecognition)) {
+            if (!offenders.empty()) {
+                offenders += ", ";
+            }
+            offenders += loader.family;
+        }
+    }
+    check(offenders.empty(),
+          "no family advertises spk" +
+              (offenders.empty() ? std::string() : " (found: " + offenders + ")"));
+}
+
+// The AudioTransformStream premise, and the one that has to be scoped exactly.
+// The broad claim "upstream streams tts and asr only" is FALSE: silero_vad
+// advertises vad/streaming. The claim that holds is that none of the four tasks
+// AudioTransform routes to is advertised streaming by anybody.
+//
+// NEGATIVE CONTROL: add VoiceTaskKind::Tts to kTransformTasks and this test must
+// fail, naming the tts families. A run where that edit stays green means the
+// query is broken and every absence above is worthless.
+void test_no_streaming_for_the_transform_tasks() {
+    const VoiceTaskKind kTransformTasks[] = {
+        VoiceTaskKind::SourceSeparation,
+        VoiceTaskKind::VoiceConversion,
+        VoiceTaskKind::Svc,
+        VoiceTaskKind::SpeechToSpeech,
+    };
+
+    std::string offenders;
+    for (const VoiceTaskKind task : kTransformTasks) {
+        for (const auto &loader : catalog()) {
+            if (advertises(loader, task, RunMode::Streaming)) {
+                if (!offenders.empty()) {
+                    offenders += ", ";
+                }
+                offenders += loader.family;
+                offenders += "/";
+                offenders += engine::runtime::to_string(task);
+            }
+        }
+    }
+    check(offenders.empty(),
+          "no streaming for sep/vc/svc/s2s" +
+              (offenders.empty() ? std::string() : " (found: " + offenders + ")"));
+}
+
+// The clause the AudioEncode message names by hand: miocodec carries a Codec tag
+// in upstream's README, and its loader advertises only vc and s2s. Asserted
+// exactly, not as a subset, so a bump that ADDS a codec capability to miocodec
+// fails here rather than leaving the message stale.
+void test_miocodec_advertises_exactly_vc_and_s2s() {
+    const LoaderAdvertisement *miocodec = nullptr;
+    for (const auto &loader : catalog()) {
+        if (loader.family == "miocodec") {
+            miocodec = &loader;
+            break;
+        }
+    }
+    if (miocodec == nullptr) {
+        check(false, "miocodec is a registered family");
+        return;
+    }
+    check(describe(*miocodec) == "vc/offline, s2s/offline",
+          "miocodec advertises exactly vc/offline, s2s/offline (got: " +
+              describe(*miocodec) + ")");
+}
+
+// The "declared only by" clause in the AudioToAudioStream message. Exact set,
+// for the same reason as above: a third s2s family would make the message stale
+// without making it obviously wrong.
+void test_speech_to_speech_is_exactly_miocodec_and_vevo2() {
+    std::set<std::string> families;
+    for (const auto &loader : catalog()) {
+        if (advertises_any_mode(loader, VoiceTaskKind::SpeechToSpeech)) {
+            families.insert(loader.family);
+        }
+    }
+    const std::set<std::string> expected = {"miocodec", "vevo2"};
+    std::string found;
+    for (const auto &family : families) {
+        if (!found.empty()) {
+            found += ", ";
+        }
+        found += family;
+    }
+    check(families == expected,
+          "s2s is advertised by exactly miocodec and vevo2 (got: " +
+              (found.empty() ? "nothing" : found) + ")");
+}
+
+} // namespace
+
+int main() {
+    test_catalog_is_populated();
+    test_no_codec_task_kind();
+    test_no_family_advertises_speaker_recognition();
+    test_no_streaming_for_the_transform_tasks();
+    test_miocodec_advertises_exactly_vc_and_s2s();
+    test_speech_to_speech_is_exactly_miocodec_and_vevo2();
+    if (failures) {
+        fprintf(stderr,
+                "%d upstream absence check(s) failed. A refusal message in "
+                "capability_routing.cpp now states something that is not true "
+                "of the pinned audio.cpp; fix the message, not this test.\n",
+                failures);
+        return 1;
+    }
+    fprintf(stderr, "all upstream absence checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/wav_header.cpp

@@ -0,0 +1,63 @@
+#include "wav_header.h"
+
+#include <cstdint>
+#include <limits>
+
+namespace audiocpp_backend {
+namespace {
+
+void append_u32(std::string &out, std::uint32_t value) {
+    out.push_back(static_cast<char>(value & 0xFF));
+    out.push_back(static_cast<char>((value >> 8) & 0xFF));
+    out.push_back(static_cast<char>((value >> 16) & 0xFF));
+    out.push_back(static_cast<char>((value >> 24) & 0xFF));
+}
+
+void append_u16(std::string &out, std::uint16_t value) {
+    out.push_back(static_cast<char>(value & 0xFF));
+    out.push_back(static_cast<char>((value >> 8) & 0xFF));
+}
+
+// The unknown-length sentinel, in both the RIFF and the data chunk size.
+constexpr std::uint32_t kStreamingSize = 0xFFFFFFFFu;
+constexpr std::uint16_t kBitsPerSample = 16;
+constexpr std::uint32_t kPcmFmtChunkSize = 16;
+constexpr std::uint16_t kFormatTagPcm = 1;
+constexpr int kMaxChannels = 65535;
+
+} // namespace
+
+std::string streaming_wav_header(int sample_rate, int channels) {
+    int clamped_channels = channels > 0 ? channels : 1;
+    if (clamped_channels > kMaxChannels) {
+        clamped_channels = kMaxChannels;
+    }
+    const auto channel_count = static_cast<std::uint16_t>(clamped_channels);
+    const auto rate = static_cast<std::uint32_t>(sample_rate > 0 ? sample_rate : 0);
+    const auto block_align =
+        static_cast<std::uint16_t>(channel_count * (kBitsPerSample / 8));
+    // uint32 arithmetic on purpose: 384 kHz by 8 channels is 6.1 MB/s, which
+    // does not fit the uint16 block align it is derived from.
+    const std::uint32_t byte_rate = rate * static_cast<std::uint32_t>(block_align);
+    static_assert(std::numeric_limits<std::uint32_t>::max() >= 0xFFFFFFFFu,
+                  "the streaming sentinel must be representable");
+
+    std::string header;
+    header.reserve(44);
+    header += "RIFF";
+    append_u32(header, kStreamingSize); // unknown total length
+    header += "WAVE";
+    header += "fmt ";
+    append_u32(header, kPcmFmtChunkSize);
+    append_u16(header, kFormatTagPcm);
+    append_u16(header, channel_count);
+    append_u32(header, rate);
+    append_u32(header, byte_rate);
+    append_u16(header, block_align);
+    append_u16(header, kBitsPerSample);
+    header += "data";
+    append_u32(header, kStreamingSize); // unknown payload length
+    return header;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/wav_header.h

@@ -0,0 +1,39 @@
+#pragma once
+
+// Builds the 44 byte canonical WAV header that precedes a streamed PCM body.
+// Standard library only.
+//
+// TTSStream chunks travel in Reply.audio and the FIRST chunk must be this
+// header, or an HTTP client has no format to decode the PCM with. Because the
+// total length is unknown while the model is still generating, both size fields
+// carry 0xFFFFFFFF; that is the convention backend/go/vibevoice-cpp established
+// (govibevoicecpp.go, TTSStream) and the one core/backend/tts.go writes when it
+// synthesises a header itself.
+//
+// WHY THIS BACKEND SENDS THE HEADER RATHER THAN LETTING GO DO IT.
+// core/backend/tts.go's ModelTTSStream will emit a header of its own, but only
+// when the FIRST Reply carries a non-empty `message` field holding a JSON blob
+// with a sample_rate. This backend sends audio and never sets `message`, so
+// that branch never fires and there is exactly one header on the wire: this
+// one. Do not start setting Reply.message on this RPC without deleting the
+// header below, or every stream gains a second header 44 bytes into the PCM.
+//
+// The header this produces is byte-identical to the one pkg/audio.WAVHeader
+// serialises, which is what core/http/endpoints/openai/realtime_model.go
+// assumes when it reads the sample rate out of byte offset 24 of the first
+// callback.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+// 16-bit PCM, little endian, interleaved.
+//
+// `channels` is clamped to at least 1 and at most 65535, so a garbage channel
+// count can never write a zero block align, which is what a reader divides the
+// data size by. `sample_rate` is clamped to at least 0 rather than wrapped:
+// a zero rate is visibly wrong to whoever reads the header, whereas the
+// 4294967295 an unsigned conversion of -1 would write looks like a real field.
+std::string streaming_wav_header(int sample_rate, int channels);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/wav_header_test.cpp

@@ -0,0 +1,163 @@
+// Unit tests for wav_header. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+
+#include "wav_header.cpp"
+
+#include <cstdint>
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using audiocpp_backend::streaming_wav_header;
+
+static std::uint32_t read_u32(const std::string &data, size_t offset) {
+    return static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset])) |
+           (static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset + 1])) << 8) |
+           (static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset + 2])) << 16) |
+           (static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset + 3])) << 24);
+}
+
+static std::uint16_t read_u16(const std::string &data, size_t offset) {
+    return static_cast<std::uint16_t>(
+        static_cast<std::uint16_t>(static_cast<unsigned char>(data[offset])) |
+        (static_cast<std::uint16_t>(static_cast<unsigned char>(data[offset + 1])) << 8));
+}
+
+static std::string to_hex(const std::string &data) {
+    static const char *digits = "0123456789abcdef";
+    std::string out;
+    out.reserve(data.size() * 2);
+    for (const char byte : data) {
+        const auto value = static_cast<unsigned char>(byte);
+        out.push_back(digits[value >> 4]);
+        out.push_back(digits[value & 0x0F]);
+    }
+    return out;
+}
+
+static void test_header_layout() {
+    const std::string header = streaming_wav_header(24000, 1);
+
+    check(header.size() == 44, "canonical 44 byte header");
+    check(header.compare(0, 4, "RIFF") == 0, "RIFF magic");
+    check(header.compare(8, 4, "WAVE") == 0, "WAVE magic");
+    check(header.compare(12, 4, "fmt ") == 0, "fmt chunk id");
+    check(header.compare(36, 4, "data") == 0, "data chunk id");
+
+    check(read_u32(header, 16) == 16, "PCM fmt chunk is 16 bytes");
+    check(read_u16(header, 20) == 1, "format tag 1 is PCM");
+    check(read_u16(header, 22) == 1, "mono channel count");
+    check(read_u32(header, 24) == 24000, "sample rate");
+    // byte rate = rate * channels * bytes per sample
+    check(read_u32(header, 28) == 24000 * 1 * 2, "byte rate");
+    check(read_u16(header, 32) == 2, "block align for mono 16 bit");
+    check(read_u16(header, 34) == 16, "16 bits per sample");
+}
+
+// The field-wise checks above can all pass while the fields sit in the wrong
+// ORDER, since several of them hold the same value. This pins the whole 44 byte
+// string against a literal transcribed from the layout in pkg/audio/audio.go,
+// which is the struct binary.Write serializes for every Go LocalAI backend.
+// Independent of the implementation: it was written out by hand rather than
+// captured from a run.
+static void test_exact_bytes_match_the_go_layout() {
+    const std::string expected =
+        "52494646"          // "RIFF"
+        "ffffffff"          // chunk size: streaming sentinel
+        "57415645"          // "WAVE"
+        "666d7420"          // "fmt "
+        "10000000"          // subchunk1 size 16
+        "0100"              // audio format 1 (PCM)
+        "0100"              // channels 1
+        "c05d0000"          // sample rate 24000
+        "80bb0000"          // byte rate 48000
+        "0200"              // block align 2
+        "1000"              // bits per sample 16
+        "64617461"          // "data"
+        "ffffffff";         // subchunk2 size: streaming sentinel
+    check(to_hex(streaming_wav_header(24000, 1)) == expected,
+          "byte for byte match with the canonical mono 24 kHz header");
+}
+
+// The whole point: a streaming header cannot know the final length, so both
+// size fields are the sentinel. A client that sees a real size stops early.
+static void test_streaming_sentinels() {
+    const std::string header = streaming_wav_header(16000, 1);
+    check(read_u32(header, 4) == 0xFFFFFFFFu, "RIFF chunk size is the sentinel");
+    check(read_u32(header, 40) == 0xFFFFFFFFu, "data chunk size is the sentinel");
+}
+
+static void test_stereo() {
+    const std::string header = streaming_wav_header(44100, 2);
+    check(read_u16(header, 22) == 2, "stereo channel count");
+    check(read_u32(header, 28) == 44100 * 2 * 2, "stereo byte rate");
+    check(read_u16(header, 32) == 4, "block align for stereo 16 bit");
+    check(header.size() == 44, "stereo header is still 44 bytes");
+}
+
+static void test_degenerate_inputs() {
+    // A zero or negative channel count must not produce a header that divides
+    // by zero downstream; clamp to mono.
+    const std::string zero_channels = streaming_wav_header(16000, 0);
+    check(read_u16(zero_channels, 22) == 1, "zero channels clamps to mono");
+    check(read_u16(zero_channels, 32) == 2, "zero channels still block aligns as mono");
+    check(read_u32(zero_channels, 28) == 32000, "zero channels byte rate is the mono one");
+
+    const std::string negative_channels = streaming_wav_header(16000, -3);
+    check(read_u16(negative_channels, 22) == 1, "negative channels clamps to mono");
+
+    // A channel count past the field's range must saturate rather than wrap:
+    // 65536 truncated to uint16 is 0, and a zero channel count writes a zero
+    // block align, which is what a reader divides the data size by.
+    const std::string too_many = streaming_wav_header(16000, 65536);
+    check(read_u16(too_many, 22) == 65535, "an out of range channel count saturates");
+    check(read_u16(too_many, 32) != 0, "an out of range channel count never writes a zero block align");
+
+    // A non-positive rate is written as zero rather than wrapping through the
+    // unsigned conversion: 0 is visibly wrong to whoever reads the header,
+    // 4294967295 looks like a plausible field nobody checks.
+    const std::string zero_rate = streaming_wav_header(0, 1);
+    check(read_u32(zero_rate, 24) == 0, "zero sample rate stays zero");
+    check(read_u32(zero_rate, 28) == 0, "zero sample rate yields a zero byte rate");
+    const std::string negative_rate = streaming_wav_header(-48000, 1);
+    check(read_u32(negative_rate, 24) == 0, "negative sample rate is clamped to zero");
+    check(read_u32(negative_rate, 28) == 0, "negative sample rate yields a zero byte rate");
+
+    // The sentinels are unconditional. A degenerate rate must not turn the
+    // stream into one a client thinks it can measure.
+    check(read_u32(zero_rate, 4) == 0xFFFFFFFFu, "degenerate input keeps the RIFF sentinel");
+    check(read_u32(zero_rate, 40) == 0xFFFFFFFFu, "degenerate input keeps the data sentinel");
+}
+
+// Large but legal: 384 kHz 8 channel would overflow a 16 bit byte rate and
+// must not overflow the 32 bit one either.
+static void test_large_but_legal() {
+    const std::string header = streaming_wav_header(384000, 8);
+    check(read_u32(header, 28) == 384000u * 8u * 2u, "high rate multichannel byte rate");
+    check(read_u16(header, 32) == 16, "high channel count block align");
+}
+
+int main() {
+    test_header_layout();
+    test_exact_bytes_match_the_go_layout();
+    test_streaming_sentinels();
+    test_stereo();
+    test_degenerate_inputs();
+    test_large_but_legal();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all wav_header checks passed\n");
+    return 0;
+}

backend/cpp/bonsai/Makefile

@@ -1,7 +1,7 @@
 
 # Pinned to the HEAD of the `prism` branch on https://github.com/PrismML-Eng/llama.cpp.
 # Auto-bumped nightly by .github/workflows/bump_deps.yaml.
-BONSAI_VERSION?=7529fdaaf99ffdc5ca71ace9c7409a56b27ad92f
+BONSAI_VERSION?=4dd165625bb6c020285eec8b342af25cf60233dd
 LLAMA_REPO?=https://github.com/PrismML-Eng/llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/bonsai/package.sh

@@ -24,34 +24,7 @@ if [ -d "$CURDIR/ggml-shared-libs" ]; then
 fi
 
 # Detect architecture and copy appropriate libraries
-if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
-    # x86_64 architecture
-    echo "Detected x86_64 architecture, copying x86_64 libraries..."
-    cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so
-    cp -arfLv /lib/x86_64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
-    cp -arfLv /lib/x86_64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
-elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
-    # ARM64 architecture
-    echo "Detected ARM64 architecture, copying ARM64 libraries..."
-    cp -arfLv /lib/ld-linux-aarch64.so.1 $CURDIR/package/lib/ld.so
-    cp -arfLv /lib/aarch64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
-    cp -arfLv /lib/aarch64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
-else
-    echo "Error: Could not detect architecture"
-    exit 1
-fi
+source "$CURDIR/../../../scripts/build/package-system-libs.sh" "$CURDIR/package/lib" ""
 
 # Package GPU libraries based on BUILD_TYPE
 GPU_LIB_SCRIPT="${REPO_ROOT}/scripts/build/package-gpu-libs.sh"

backend/cpp/bonsai/run.sh

@@ -40,6 +40,27 @@ else
 	if [ -d "$CURDIR/lib/hipblaslt/library" ]; then
 		export HIPBLASLT_TENSILE_LIBPATH="$CURDIR"/lib/hipblaslt/library
 	fi
+	# Backends built for Intel GPUs carry a copy of the Intel graphics driver,
+	# and libze_loader is only there in those builds. Level Zero looks for a
+	# driver on its own, so point it at the copy that came with this backend: it
+	# was built against the same C library, while the machine's own driver may
+	# not have been, and loading that one can crash on start.
+	#
+	# Anything the user set is left alone, so a machine with a graphics card
+	# newer than the driver carried here can still be told to use its own.
+	# Nothing is said about OpenCL: no OpenCL driver is carried, so anything we
+	# set there would leave OpenCL worse off than the machine's own setup.
+	if [ -e "$CURDIR/lib/libze_loader.so.1" ]; then
+		if [ -e "$CURDIR/lib/libze_intel_gpu.so.1" ] && [ -z "${ZE_ENABLE_ALT_DRIVERS:-}" ]; then
+			export ZE_ENABLE_ALT_DRIVERS="$CURDIR"/lib/libze_intel_gpu.so.1
+		fi
+		# Ask the driver how much graphics memory is free. Without this, the
+		# backend reads zero on an integrated graphics chip, because such a chip
+		# shares the system memory instead of having its own.
+		if [ -z "${ZES_ENABLE_SYSMAN:-}" ]; then
+			export ZES_ENABLE_SYSMAN=1
+		fi
+	fi
 fi
 
 # If there is a lib/ld.so, use it

backend/cpp/ds4/package.sh

@@ -17,13 +17,7 @@ if [ "$UNAME_S" = "Darwin" ]; then
     exit 0
 fi
 
-if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
-    cp -arfLv /lib64/ld-linux-x86-64.so.2 "$PACKAGE_DIR/lib/ld.so"
-elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
-    cp -arfLv /lib/ld-linux-aarch64.so.1 "$PACKAGE_DIR/lib/ld.so"
-else
-    echo "package.sh: unknown architecture" >&2; exit 1
-fi
+source "$CURDIR/../../../scripts/build/package-system-libs.sh" "$CURDIR/package/lib" ""
 
 # Bundle the complete dependency closure for both executables. In particular,
 # grpc-server links the distro gRPC/protobuf/absl stack; copying only the core

backend/cpp/ik-llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-IK_LLAMA_VERSION?=b054a8b983827c01aec59d4dc273a27c492c51c4
+IK_LLAMA_VERSION?=0be97a7a5ad113f33e08729261649ccea2cdc5ff
 LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/ik-llama-cpp/package.sh

@@ -15,34 +15,7 @@ cp -avrf $CURDIR/ik-llama-cpp-* $CURDIR/package/
 cp -rfv $CURDIR/run.sh $CURDIR/package/
 
 # Detect architecture and copy appropriate libraries
-if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
-    # x86_64 architecture
-    echo "Detected x86_64 architecture, copying x86_64 libraries..."
-    cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so
-    cp -arfLv /lib/x86_64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
-    cp -arfLv /lib/x86_64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
-elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
-    # ARM64 architecture
-    echo "Detected ARM64 architecture, copying ARM64 libraries..."
-    cp -arfLv /lib/ld-linux-aarch64.so.1 $CURDIR/package/lib/ld.so
-    cp -arfLv /lib/aarch64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
-    cp -arfLv /lib/aarch64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
-else
-    echo "Error: Could not detect architecture"
-    exit 1
-fi
+source "$CURDIR/../../../scripts/build/package-system-libs.sh" "$CURDIR/package/lib" ""
 
 # Package GPU libraries based on BUILD_TYPE
 # The GPU library packaging script will detect BUILD_TYPE and copy appropriate GPU libraries

backend/cpp/llama-cpp/CMakeLists.txt

@@ -110,4 +110,9 @@ if(LLAMA_GRPC_BUILD_TESTS)
     target_link_libraries(parent_watch_test PRIVATE Threads::Threads)
     target_compile_features(parent_watch_test PRIVATE cxx_std_17)
     add_test(NAME parent_watch_test COMMAND parent_watch_test)
+
+    add_executable(passthrough_options_test passthrough_options_test.cpp passthrough_options.h)
+    target_include_directories(passthrough_options_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
+    target_compile_features(passthrough_options_test PRIVATE cxx_std_17)
+    add_test(NAME passthrough_options_test COMMAND passthrough_options_test)
 endif()

backend/cpp/llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-LLAMA_VERSION?=1cbfd1988311775425d36c0ce066590f7d3049cf
+LLAMA_VERSION?=876a4321163249c43ca4e986818fab5ab081f282
 LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/grpc-server.cpp

@@ -54,6 +54,7 @@
 #include "chat-auto-parser.h"
 #include "llama_compat.h" // fork-skew switches, generated by prepare.sh
 #include "message_content.h"
+#include "passthrough_options.h"
 #include <getopt.h>
 #include <grpcpp/ext/proto_server_reflection_plugin.h>
 #include <grpcpp/grpcpp.h>
@@ -579,6 +580,8 @@ static void params_parse(server_context& /*ctx_server*/, const backend::ModelOpt
     // Raw upstream llama-server flags collected from any option entry that
     // starts with '-'. Applied once after the loop via common_params_parse.
     std::vector<std::string> extra_argv;
+    bool passthrough_main_gpu_layers = false;
+    bool passthrough_draft_gpu_layers = false;
 
     // O_DIRECT intent from the `direct_io` option. Upstream folded
     // use_mmap/use_mlock/use_direct_io into a single common_params::load_mode
@@ -1186,6 +1189,17 @@ static void params_parse(server_context& /*ctx_server*/, const backend::ModelOpt
                     flag.c_str());
             } else {
                 extra_argv.push_back(flag);
+                passthrough_main_gpu_layers =
+                    passthrough_main_gpu_layers ||
+                    flag == "-ngl" ||
+                    flag == "--gpu-layers" ||
+                    flag == "--n-gpu-layers";
+                passthrough_draft_gpu_layers =
+                    passthrough_draft_gpu_layers ||
+                    flag == "--spec-draft-ngl" ||
+                    flag == "-ngld" ||
+                    flag == "--gpu-layers-draft" ||
+                    flag == "--n-gpu-layers-draft";
                 // Preserve the whole value after the first ':' so embedded
                 // colons (e.g. host:port) survive strtok's truncation of optval.
                 auto colon = opt.find(':');
@@ -1349,6 +1363,14 @@ static void params_parse(server_context& /*ctx_server*/, const backend::ModelOpt
         // (n_parallel -> -1, use_color). Snapshot n_parallel so an unrelated
         // passthrough flag can't silently clobber LocalAI's resolved value.
         const int saved_n_parallel = params.n_parallel;
+        // Newer upstream parsers assert that these fields still contain their
+        // negative initialization sentinels. LocalAI resolves them from the
+        // model request before applying passthrough options, so stage the
+        // sentinels and restore the values unless a raw flag overrides them.
+        const auto saved_gpu_layers =
+            llama_grpc::prepare_passthrough_gpu_layers(
+                params.n_gpu_layers,
+                params.speculative.draft.n_gpu_layers);
 
         std::vector<char *> argv;
         std::string prog = "llama-server";
@@ -1372,6 +1394,12 @@ static void params_parse(server_context& /*ctx_server*/, const backend::ModelOpt
         if (params.n_parallel == -1) {
             params.n_parallel = saved_n_parallel;
         }
+        llama_grpc::restore_passthrough_gpu_layers(
+            params.n_gpu_layers,
+            params.speculative.draft.n_gpu_layers,
+            saved_gpu_layers,
+            passthrough_main_gpu_layers,
+            passthrough_draft_gpu_layers);
     }
 
 #ifndef LOCALAI_LLAMA_CPP_NO_SCORE_TASK
@@ -3251,6 +3279,21 @@ public:
         return grpc::Status::OK;
     }
 
+    grpc::Status Detokenize(ServerContext* context, const backend::DetokenizeRequest* request, backend::DetokenizeResponse* response) override {
+        auto auth = checkAuth(context);
+        if (!auth.ok()) return auth;
+        if (params_base.model.path.empty()) {
+            return grpc::Status(grpc::StatusCode::FAILED_PRECONDITION, "Model not loaded");
+        }
+
+        std::string content;
+        for (const auto token : request->tokens()) {
+            content.append(common_token_to_piece(ctx_server.get_llama_context(), token));
+        }
+        response->set_content(content);
+        return grpc::Status::OK;
+    }
+
     grpc::Status GetMetrics(ServerContext* /*context*/, const backend::MetricsRequest* /*request*/, backend::MetricsResponse* response) override {
 
 

backend/cpp/llama-cpp/package.sh

@@ -31,34 +31,7 @@ if [ -d "$CURDIR/ggml-shared-libs" ]; then
 fi
 
 # Detect architecture and copy appropriate libraries
-if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
-    # x86_64 architecture
-    echo "Detected x86_64 architecture, copying x86_64 libraries..."
-    cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so
-    cp -arfLv /lib/x86_64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
-    cp -arfLv /lib/x86_64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
-    cp -arfLv /lib/x86_64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
-    cp -arfLv /lib/x86_64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
-elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
-    # ARM64 architecture
-    echo "Detected ARM64 architecture, copying ARM64 libraries..."
-    cp -arfLv /lib/ld-linux-aarch64.so.1 $CURDIR/package/lib/ld.so
-    cp -arfLv /lib/aarch64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
-    cp -arfLv /lib/aarch64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
-    cp -arfLv /lib/aarch64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
-    cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
-else
-    echo "Error: Could not detect architecture"
-    exit 1
-fi
+source "$CURDIR/../../../scripts/build/package-system-libs.sh" "$CURDIR/package/lib" ""
 
 # Package GPU libraries based on BUILD_TYPE
 # The GPU library packaging script will detect BUILD_TYPE and copy appropriate GPU libraries

backend/cpp/llama-cpp/passthrough_options.h

@@ -0,0 +1,38 @@
+// SPDX-License-Identifier: MIT
+
+#pragma once
+
+namespace llama_grpc {
+
+struct passthrough_gpu_layers_state {
+    int main;
+    int draft;
+};
+
+inline passthrough_gpu_layers_state prepare_passthrough_gpu_layers(
+    int & main_gpu_layers,
+    int & draft_gpu_layers) {
+    const passthrough_gpu_layers_state saved{
+        main_gpu_layers,
+        draft_gpu_layers,
+    };
+    main_gpu_layers = -1;
+    draft_gpu_layers = -1;
+    return saved;
+}
+
+inline void restore_passthrough_gpu_layers(
+    int & main_gpu_layers,
+    int & draft_gpu_layers,
+    passthrough_gpu_layers_state saved,
+    bool main_overridden = false,
+    bool draft_overridden = false) {
+    if (!main_overridden && main_gpu_layers == -1) {
+        main_gpu_layers = saved.main;
+    }
+    if (!draft_overridden && draft_gpu_layers == -1) {
+        draft_gpu_layers = saved.draft;
+    }
+}
+
+} // namespace llama_grpc

backend/cpp/llama-cpp/passthrough_options_test.cpp

@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: MIT
+
+#include <cstdio>
+
+#include "passthrough_options.h"
+
+static int failures = 0;
+
+static void check(bool ok, const char * name) {
+    if (!ok) {
+        ++failures;
+        std::fprintf(stderr, "FAIL: %s\n", name);
+    }
+}
+
+static void test_stages_resolved_gpu_layers_for_upstream_parser() {
+    int main_gpu_layers = 99;
+    int draft_gpu_layers = 12;
+
+    const auto saved = llama_grpc::prepare_passthrough_gpu_layers(
+        main_gpu_layers, draft_gpu_layers);
+
+    check(main_gpu_layers == -1, "main GPU layers use parser sentinel");
+    check(draft_gpu_layers == -1, "draft GPU layers use parser sentinel");
+
+    llama_grpc::restore_passthrough_gpu_layers(
+        main_gpu_layers, draft_gpu_layers, saved);
+
+    check(main_gpu_layers == 99, "main GPU layers restored");
+    check(draft_gpu_layers == 12, "draft GPU layers restored");
+}
+
+static void test_keeps_explicit_passthrough_overrides() {
+    int main_gpu_layers = 99;
+    int draft_gpu_layers = 12;
+
+    const auto saved = llama_grpc::prepare_passthrough_gpu_layers(
+        main_gpu_layers, draft_gpu_layers);
+
+    main_gpu_layers = 4;
+    draft_gpu_layers = 2;
+    llama_grpc::restore_passthrough_gpu_layers(
+        main_gpu_layers, draft_gpu_layers, saved);
+
+    check(main_gpu_layers == 4, "main passthrough override retained");
+    check(draft_gpu_layers == 2, "draft passthrough override retained");
+}
+
+static void test_keeps_explicit_auto_passthrough_overrides() {
+    int main_gpu_layers = 99;
+    int draft_gpu_layers = 12;
+
+    const auto saved = llama_grpc::prepare_passthrough_gpu_layers(
+        main_gpu_layers, draft_gpu_layers);
+
+    llama_grpc::restore_passthrough_gpu_layers(
+        main_gpu_layers, draft_gpu_layers, saved, true, true);
+
+    check(main_gpu_layers == -1, "main auto passthrough override retained");
+    check(draft_gpu_layers == -1, "draft auto passthrough override retained");
+}
+
+int main() {
+    test_stages_resolved_gpu_layers_for_upstream_parser();
+    test_keeps_explicit_passthrough_overrides();
+    test_keeps_explicit_auto_passthrough_overrides();
+    return failures == 0 ? 0 : 1;
+}

backend/cpp/llama-cpp/patches/0001-add-minimax-m3-chat-parser.patch

@@ -1,225 +0,0 @@
-# MiniMax-M3 chat-template parser, vendored from upstream llama.cpp PR #24523.
-#
-# Upstream has since merged the *model* half of #24523 (LLM_ARCH_MINIMAX_M3,
-# src/models/minimax-m3.cpp, the gguf-py constants and conversion/minimax.py), so
-# only the chat half is carried here: M3's namespace token "]<]minimax[>[" collides
-# with the autoparser's markup delimiters, so common/chat.cpp needs a dedicated
-# template detection + PEG parser that upstream does not have yet.
-#
-# Rebased against LLAMA_VERSION 0d47ea7427463093e69128bf2c2f9cd06b3ee5b3, which also
-# renamed common_chat_params::thinking_end_tag to thinking_end_tags (a vector).
-# LLAMA_VERSION is auto-bumped nightly; if a bump rejects this patch, re-vendor from
-# #24523 — or, once the chat half merges upstream, delete this file.
-# See https://github.com/mudler/LocalAI/issues/10820 and PR #10837.
-diff --git a/common/chat.cpp b/common/chat.cpp
-index 7a6e7238c..2dd015a2e 100644
---- a/common/chat.cpp
-+++ b/common/chat.cpp
-@@ -2121,6 +2121,191 @@ static common_chat_params common_chat_params_init_deepseek_v3_2(const common_cha
-     return data;
- }
- 
-+static common_chat_params common_chat_params_init_minimax_m3(const common_chat_template &    tmpl,
-+                                                             const autoparser::generation_params & inputs) {
-+    common_chat_params data;
-+
-+    data.prompt             = common_chat_template_direct_apply_impl(tmpl, inputs);
-+    data.generation_prompt  = common_chat_template_generation_prompt_impl(tmpl, inputs);
-+    data.format             = COMMON_CHAT_FORMAT_PEG_NATIVE;
-+    data.supports_thinking  = true;
-+    data.thinking_start_tag = "<mm:think>";
-+    data.thinking_end_tags  = {"</mm:think>"};
-+
-+    // M3 prefixes every tool tag with the namespace token "]<]minimax[>[";
-+    // params use the parameter name as the tag (<file_path>...</file_path>).
-+    const std::string NS           = "]<]minimax[>[";
-+    const std::string THINK_START  = "<mm:think>";
-+    const std::string THINK_END    = "</mm:think>";
-+    const std::string FC_START     = NS + "<tool_call>";
-+    const std::string FC_END       = NS + "</tool_call>";
-+    const std::string INVOKE_END   = NS + "</invoke>";
-+
-+    data.preserved_tokens   = {
-+        NS,
-+        "<tool_call>",
-+        "</tool_call>",
-+        THINK_START,
-+        THINK_END,
-+    };
-+
-+    auto has_tools           = inputs.tools.is_array() && !inputs.tools.empty();
-+    auto has_response_format = !inputs.json_schema.is_null() && inputs.json_schema.is_object();
-+    auto extract_reasoning   = inputs.reasoning_format != COMMON_REASONING_FORMAT_NONE;
-+    auto include_grammar     = has_response_format || (has_tools && inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_NONE);
-+
-+    const std::string GEN_PROMPT = data.generation_prompt;
-+
-+    if (inputs.has_continuation()) {
-+        const auto & msg = inputs.continue_msg;
-+
-+        data.generation_prompt = GEN_PROMPT + THINK_START + msg.reasoning_content;
-+        if (inputs.continue_final_message == COMMON_CHAT_CONTINUATION_CONTENT) {
-+            data.generation_prompt += THINK_END + msg.render_content();
-+        }
-+
-+        data.prompt += data.generation_prompt;
-+    }
-+
-+    auto parser = build_chat_peg_parser([&](common_chat_peg_builder & p) {
-+        auto generation_prompt = p.literal(GEN_PROMPT);
-+        auto end = p.end();
-+
-+        auto reasoning = p.eps();
-+        // M3 can emit a bare </mm:think> (no opener) after tool results; keep the opener optional.
-+        if (extract_reasoning && inputs.enable_thinking) {
-+            reasoning = p.optional(p.optional(p.literal(THINK_START)) + p.reasoning(p.until(THINK_END)) + THINK_END);
-+        } else if (extract_reasoning) {
-+            reasoning = p.optional(p.optional(p.literal(THINK_START)) + p.until(THINK_END) + p.literal(THINK_END));
-+        }
-+
-+        if (has_response_format) {
-+            auto response_format = p.rule("response-format",
-+                p.literal("```json") + p.space() +
-+                p.content(p.schema(p.json(), "response-format-schema", inputs.json_schema)) +
-+                p.space() + p.literal("```"));
-+            return generation_prompt + reasoning + response_format + end;
-+        }
-+
-+        if (!has_tools || inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_NONE) {
-+            return generation_prompt + reasoning + p.content(p.rest()) + end;
-+        }
-+
-+        auto tool_choice = p.choice();
-+        foreach_function(inputs.tools, [&](const json & tool) {
-+            const auto & function = tool.at("function");
-+            std::string  name     = function.at("name");
-+            auto params   = function.contains("parameters") ? function.at("parameters") : json::object();
-+            const auto & props    = params.contains("properties") ? params.at("properties") : json::object();
-+
-+            std::set<std::string> required;
-+            if (params.contains("required")) {
-+                params.at("required").get_to(required);
-+            }
-+
-+            auto schema_info = common_schema_info();
-+            schema_info.resolve_refs(params);
-+
-+            std::vector<common_peg_parser> required_parsers;
-+            std::vector<common_peg_parser> optional_parsers;
-+            for (const auto & [param_name, param_schema] : props.items()) {
-+                bool is_required = required.find(param_name) != required.end();
-+                bool is_string   = schema_info.resolves_to_string(param_schema);
-+
-+                const std::string p_close = NS + "</" + param_name + ">";
-+
-+                auto arg = p.tool_arg(
-+                    p.tool_arg_open(
-+                        p.literal(NS + "<") +
-+                        p.tool_arg_name(p.literal(param_name)) +
-+                        p.literal(">")) +
-+                    (is_string
-+                         ? p.ac(p.tool_arg_string_value(p.until(p_close)) +
-+                                p.tool_arg_close(p.literal(p_close)), p_close)
-+                         : p.tool_arg_json_value(p.schema(p.json(),
-+                                                          "tool-" + name + "-arg-" + param_name + "-schema",
-+                                                          param_schema, false)) +
-+                           p.tool_arg_close(p.literal(p_close))));
-+
-+                auto named_arg = p.rule("tool-" + name + "-arg-" + param_name, arg);
-+                if (is_required) {
-+                    required_parsers.push_back(named_arg);
-+                } else {
-+                    optional_parsers.push_back(named_arg);
-+                }
-+            }
-+
-+            common_peg_parser args_seq = p.eps();
-+            for (size_t i = 0; i < required_parsers.size(); i++) {
-+                if (i > 0) {
-+                    args_seq = args_seq + p.space();
-+                }
-+                args_seq = args_seq + required_parsers[i];
-+            }
-+
-+            if (!optional_parsers.empty()) {
-+                common_peg_parser any_opt = p.choice();
-+                for (const auto & opt : optional_parsers) {
-+                    any_opt |= opt;
-+                }
-+                args_seq = args_seq + p.repeat(p.space() + any_opt, 0, -1);
-+            }
-+
-+            common_peg_parser invoke_body = args_seq;
-+            auto func_parser = p.tool(
-+                p.tool_open(p.literal(NS + "<invoke name=\"") +
-+                            p.tool_name(p.literal(name)) + p.literal("\">")) +
-+                p.space() + invoke_body + p.space() +
-+                p.tool_close(p.literal(INVOKE_END)));
-+
-+            tool_choice |= p.rule("tool-" + name, func_parser);
-+        });
-+
-+        auto require_tools = inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED;
-+
-+        common_peg_parser tool_calls = p.eps();
-+        if (inputs.parallel_tool_calls) {
-+            tool_calls = p.trigger_rule("tool-call",
-+                p.literal(FC_START) + p.space() + tool_choice +
-+                p.zero_or_more(p.space() + tool_choice) + p.space() + p.literal(FC_END));
-+        } else {
-+            tool_calls = p.trigger_rule("tool-call",
-+                p.literal(FC_START) + p.space() + tool_choice + p.space() + p.literal(FC_END));
-+        }
-+
-+        if (!require_tools) {
-+            tool_calls = p.optional(tool_calls);
-+        }
-+
-+        auto content_before_tools = p.content(p.until(FC_START));
-+        return generation_prompt + reasoning + content_before_tools + tool_calls + end;
-+    });
-+
-+    data.parser = parser.save();
-+
-+    if (include_grammar) {
-+        data.grammar_lazy = !(has_response_format || (has_tools && inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED));
-+        data.grammar      = build_grammar([&](const common_grammar_builder & builder) {
-+            foreach_function(inputs.tools, [&](const json & tool) {
-+                const auto & function = tool.at("function");
-+                auto         schema   = function.contains("parameters") ? function.at("parameters") : json::object();
-+                builder.resolve_refs(schema);
-+            });
-+            if (has_response_format) {
-+                auto schema = inputs.json_schema;
-+                builder.resolve_refs(schema);
-+            }
-+            parser.build_grammar(builder, data.grammar_lazy);
-+        });
-+
-+        data.grammar_triggers = {
-+            { COMMON_GRAMMAR_TRIGGER_TYPE_WORD, FC_START },
-+        };
-+    }
-+
-+    return data;
-+}
-+
- // Cohere2 MoE (a.k.a. "North Code") parser.
- //
- // The assistant turn is fully marker-wrapped:
-@@ -2707,6 +2892,15 @@ std::optional<common_chat_params> common_chat_try_specialized_template(
-         return common_chat_params_init_gigachat_v3(tmpl, params);
-     }
- 
-+    // MiniMax-M3: the namespace token "]<]minimax[>[" collides with the autoparser's
-+    // markup delimiters, so detect the template and use a dedicated parser.
-+    if (src.find("]<]minimax[>[") != std::string::npos &&
-+        src.find("<tool_call>") != std::string::npos &&
-+        src.find("<invoke name=") != std::string::npos) {
-+        LOG_DBG("Using specialized template: MiniMax-M3\n");
-+        return common_chat_params_init_minimax_m3(tmpl, params);
-+    }
-+
-     // DeepSeek V3.2/V4 format detection: template defines dsml_token and uses it for tool calls.
-     // The template source contains the token as a variable assignment, not as a literal in markup.
-     // V3.2 names the tool call block "function_calls", V4 names it "tool_calls".

backend/cpp/llama-cpp/prepare.sh

@@ -2,6 +2,7 @@
 
 set -e
 
+
 ## Patches
 
 ## Apply patches from the `patches` directory. Runs under set -e so a
@@ -24,6 +25,9 @@ cp -r grpc-server.cpp llama.cpp/tools/grpc-server/
 # unit test (compiled only when -DLLAMA_GRPC_BUILD_TESTS=ON).
 cp -r message_content.h llama.cpp/tools/grpc-server/
 cp -r message_content_test.cpp llama.cpp/tools/grpc-server/
+# Generic passthrough parser staging and its standalone regression test.
+cp -r passthrough_options.h llama.cpp/tools/grpc-server/
+cp -r passthrough_options_test.cpp llama.cpp/tools/grpc-server/
 # Parent-death watcher (included by grpc-server.cpp) and its standalone unit
 # test (run via backend/cpp/run-unit-tests.sh; also buildable under ctest).
 cp -r parent_watch.h llama.cpp/tools/grpc-server/
@@ -58,4 +62,3 @@ else
     echo "add_subdirectory(grpc-server)" >> llama.cpp/tools/CMakeLists.txt
 fi
 set -e
-

backend/cpp/llama-cpp/run.sh

@@ -12,10 +12,10 @@ grep -e "flags" /proc/cpuinfo | head -1
 
 BINARY=llama-cpp-fallback
 
-# CPU images (x86, arm64, darwin) ship a single llama-cpp-cpu-all built with ggml
+# CPU images and x86 GPU images ship a single llama-cpp-cpu-all built with ggml
 # CPU_ALL_VARIANTS: ggml's backend registry dlopens the best libggml-cpu-*.so for this
-# host, so no shell-side AVX probing. GPU images (cublas/sycl/vulkan/hipblas) ship only
-# llama-cpp-fallback (the accelerator does the compute), so fall back to it when absent.
+# host, so no shell-side AVX probing. GPU arm64 images still ship llama-cpp-fallback
+# until their builder toolchains support ggml's complete arm variant matrix.
 if [ -e "$CURDIR"/llama-cpp-cpu-all ]; then
 	BINARY=llama-cpp-cpu-all
 fi
@@ -42,6 +42,27 @@ else
 	if [ -d "$CURDIR/lib/hipblaslt/library" ]; then
 		export HIPBLASLT_TENSILE_LIBPATH="$CURDIR"/lib/hipblaslt/library
 	fi
+	# Backends built for Intel GPUs carry a copy of the Intel graphics driver,
+	# and libze_loader is only there in those builds. Level Zero looks for a
+	# driver on its own, so point it at the copy that came with this backend: it
+	# was built against the same C library, while the machine's own driver may
+	# not have been, and loading that one can crash on start.
+	#
+	# Anything the user set is left alone, so a machine with a graphics card
+	# newer than the driver carried here can still be told to use its own.
+	# Nothing is said about OpenCL: no OpenCL driver is carried, so anything we
+	# set there would leave OpenCL worse off than the machine's own setup.
+	if [ -e "$CURDIR/lib/libze_loader.so.1" ]; then
+		if [ -e "$CURDIR/lib/libze_intel_gpu.so.1" ] && [ -z "${ZE_ENABLE_ALT_DRIVERS:-}" ]; then
+			export ZE_ENABLE_ALT_DRIVERS="$CURDIR"/lib/libze_intel_gpu.so.1
+		fi
+		# Ask the driver how much graphics memory is free. Without this,
+		# llama.cpp reads zero on an integrated graphics chip, because such a
+		# chip shares the system memory instead of having its own.
+		if [ -z "${ZES_ENABLE_SYSMAN:-}" ]; then
+			export ZES_ENABLE_SYSMAN=1
+		fi
+	fi
 fi
 
 # If there is a lib/ld.so, use it
@@ -55,4 +76,4 @@ echo "Using binary: $BINARY"
 exec "$CURDIR"/$BINARY "$@"
 
 # We should never reach this point, however just in case we do, run fallback
-exec "$CURDIR"/llama-cpp-fallback "$@"
+exec "$CURDIR"/llama-cpp-fallback "$@"

backend/cpp/privacy-filter/package.sh

@@ -11,13 +11,7 @@ cp -rfv "$CURDIR/run.sh"      "$CURDIR/package/"
 
 # The dynamic loader, renamed to lib/ld.so so run.sh can invoke it explicitly
 # (makes the image independent of the host's glibc layout).
-if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
-    cp -arfLv /lib64/ld-linux-x86-64.so.2 "$CURDIR/package/lib/ld.so"
-elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
-    cp -arfLv /lib/ld-linux-aarch64.so.1 "$CURDIR/package/lib/ld.so"
-else
-    echo "package.sh: unknown architecture" >&2; exit 1
-fi
+source "$CURDIR/../../../scripts/build/package-system-libs.sh" "$CURDIR/package/lib" ""
 
 # Bundle the binary's transitive shared deps (libstdc++, libgomp, and the apt
 # grpc++/protobuf/absl stack) by walking ldd — robust to whichever of those are

backend/cpp/run-unit-tests.sh

@@ -37,9 +37,19 @@ if [ -z "$JSON_INC" ]; then
 fi
 
 # Active source dirs only - exclude per-variant build copies, dev snapshots and
-# the vendored upstream llama.cpp tree.
+# the vendored upstream checkouts.
+#
+# Every upstream checkout needs its own -not -path, and audio.cpp is safe TODAY
+# only by luck: its 44 tests all put "test" at the FRONT (17 test-*.cpp, 27
+# test_*.cpp, zero *_test.cpp), so the glob below misses every one of them.
+# Nothing enforces that. This gate runs on every PR for every backend, and it
+# compiles each match as a STANDALONE translation unit with nothing but
+# nlohmann/json on the include path, so the day upstream adds or renames one
+# test to *_test.cpp the whole gate goes red repo-wide, on an Apache-2.0 file
+# nobody here wrote. Exclude it now rather than diagnose that later.
 mapfile -t tests < <(find "$ROOT" -name '*_test.cpp' \
     -not -path '*/llama.cpp/*' \
+    -not -path '*/audio.cpp/*' \
     -not -path '*-build/*' \
     -not -path '*-dev/*' \
     -not -path '*fallback*' | sort)