feat(audio-cpp): add the audio.cpp native backend (#11141)

* backend(audio-cpp): add the native build scaffold

Links 0xShug0/audio.cpp engine_runtime through its public framework headers
and serves Health/Status. Model loading and the audio RPCs follow.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): keep the build-tree rpath at $ORIGIN

Upstream sets CMAKE_BUILD_WITH_INSTALL_RPATH in its own directory scope, so
CMake was appending its build-tree library dir to our target and baking an
absolute build-host path into the shipped binary. Set BUILD_WITH_INSTALL_RPATH
on the target so a package that forgets to bundle libggml*.so fails on the
build machine too, instead of only on a user's box.

Also document why EXCLUDE_FROM_ALL must stay on the add_subdirectory call,
correct the claim that Ubuntu ships no gRPC CMake config, stop the pin comment
from repeating the assignment token that bump_deps.sh rewrites, and make
test-engine fail rather than pass when no test is registered.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): parse namespaced model options

Splits option entries on the first colon so path values survive, and routes
load./session. prefixes to the upstream load and session option maps.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): reject out-of-range numeric model options

std::atoi is undefined once the digits exceed long and in practice wraps, so
device:2147483648 was accepted and handed the ggml backend selector a device
index of -2147483648 from a function whose error text promises a non-negative
integer. Parse with strtol and reject on ERANGE, on a value above INT_MAX, and
on any unconsumed trailing input. The error strings are unchanged.

Name the whole entry in the unknown-key error too: an entry like ':value' has
an empty key and left the user nothing to grep for in their YAML.

Tests look keys up through a helper instead of map::at, so a prefix off-by-one
fails one named check rather than aborting the binary and skipping the rest of
the suite, and cover the overflow, negative and non-numeric paths.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): route LocalAI RPCs onto audio.cpp tasks

Task-major resolution over the family's advertised capability set, with the
voice-reference and instructions signals selecting cloning and voice design,
and a streaming-to-offline fallback for server-streaming transcription only.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): use upstream's 'spk' task name and pin the preference order

The SpeakerRecognition short name was 'spkrec', which audio.cpp neither prints
nor parses; a name copied out of audio.cpp was rejected and a pinned 'spkrec'
would not survive the engine boundary. Emit 'spk', keep 'spkrec' as an
input-only alias, and correct the known-tasks lists.

Three assertions were vacuous because their fixtures advertised a single task,
so reversing a preference order or dropping the RPC name and the attempted
pairs from the capability error all passed. Give them fixtures that can tell
the orderings apart.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): convert sample, time and PCM units

Integer nanosecond conversion so 44.1 kHz stays exact, float seconds for the
VAD and diarization messages, and saturating s16le encode so an overshooting
sample cannot wrap to the opposite sign.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): harden seconds_to_samples against NaN and overflow

seconds_to_samples is the one entry point fed by untrusted-shaped input: a
float-seconds timestamp off the wire, or a boundary from a model that diverged.
Its guard covered only the low side, so NaN and out-of-range values fell through
to an undefined double-to-int64 cast and came back as INT64_MIN. A hugely
negative sample index used later as an offset or a length is a wild pointer
rather than merely a wrong timestamp. Reject NaN with the !(x > 0) form and
saturate before the cast.

Also round instead of truncating there. These functions exist to cross the float
seconds boundary the VAD and diarize messages use, and truncation lost a sample
about half the time on the samples-to-seconds-and-back round trip, starting at
n=1.

Pin the decode scale at INT16_MIN, pin nanosecond truncation on a nonzero
fraction, and record why the clamp argument order in f32_to_s16le is
load-bearing for NaN.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): map NaN PCM samples to silence explicitly

f32_to_s16le relied on std::min argument order to keep a NaN sample away from
std::lround, whose result is unspecified for NaN. That was too subtle to rest on
a comment, and the comment was itself wrong: it warned against a spelling that
the outer std::max already catches, while three real spellings leak, including
std::clamp, which is the idiomatic C++17 way to write the same clamp and so the
likeliest future edit.

Divert NaN before the clamp and encode it as 0. A NaN sample rendered as a
full-scale click is worse audio than a dropped one, and this unit converts audio
that may have originated off the wire.

Pin it with an exact-value check rather than a range check, since all three
outcomes the plausible spellings produce are finite and inside full scale, plus
an invalid-operation check that fails unless the NaN is diverted before any
ordered comparison. That second check is what catches modernizing the clamp and
dropping the guard together.

Also bound the seconds round-trip comment, which claimed unconditionally what
holds only below roughly 2^23 samples, and document NaN, saturation and that
bound in the header.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): assemble transcripts from runtime spans

The top-level transcript text is TaskResult.text_output verbatim. audio.cpp
carries text nowhere else: speech_segments, speaker_turns and word_timestamps
hold spans and labels only, so deriving the text from them empties the
transcript for any producer that omits word timing, VibeVoice diarized ASR
included. Fixtures cover every observed producer shape.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): keep a nested speaker turn's own label

A segment sourced from speaker_turns re-derived its speaker by greatest
overlap. A turn's overlap with its own span is the largest possible, so a turn
nested inside another speaker's turn could only tie with the container, and the
tie went to whichever came first. sortformer_diar binarizes each speaker
independently and sorts by start sample, so the container always comes first
and the interjecting speaker was silently erased from DiarizeSegment.speaker.
choose_segment_spans now carries the label out with the span.

Also pins the nearest-segment fallback against measuring from either endpoint
or from segment position, which a trailing-only stray word could not do, and
exercises the empty-word guard in join_words. Two fixtures that pin a rule but
do not mirror any pinned family are relabelled defensive.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serialize runs with a wedge-aware guard

audio.cpp sessions are not reentrant and a wedged CUDA call cannot be
cancelled, so a plain mutex would pile every worker thread behind a stuck GPU.
Callers waiting past the configured bound, or arriving while the holder has
already overrun it, fail fast instead.

A caller that queues behind a healthy run deliberately does not stamp the
clock: only the thread that takes the lock does. Stamping on arrival would
restart the wedge clock on every request and hide a stuck run from everyone
behind it, which is the pile-up this guard exists to prevent.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serialize inference through an InferenceLane

One audio.cpp model is loaded per backend process and its sessions are not
reentrant, so concurrent gRPC handlers have to take turns. Serialization alone
is not enough: a wedged GPU call cannot be cancelled from userspace, so an
unbounded queue behind one stuck run would swallow every gRPC worker thread
until the process is useless.

InferenceLane gives handlers a lane with room for one runner. LaneEntry occupies
it for a scope and gives it back on every exit, including an exception, and is
the only way to take the lane at all: occupy/vacate are private with LaneEntry
as the sole friend, so a caller cannot acquire without holding something that
releases. LaneEntry is immovable on purpose, because a moved-from entry would
have to stop releasing while the lane still recorded it as occupied.

A caller either waits indefinitely or brings a millisecond budget. A bounded
caller that cannot get in fails instead of waiting on, and a bounded caller
whose budget is already shorter than the age of the run in the lane fails
immediately, which is what stops a queue forming behind a wedged run. The two
failures carry different text: one names the wait it exhausted, the other states
the measured age of the run without claiming to know why it is long, since a
short budget meeting a legitimately long run lands there too.

The run's age is stamped only after acquisition. A waiter that published itself
as holder would restart the measurement and hide a genuinely stuck holder from
every caller behind it.

Budget negotiation and the overrun decision are pure functions taking their
inputs explicitly, so both are covered without threads or sleeping. The
per-model ceiling arrives as an int of milliseconds; a request may tighten it
and may never loosen it.

Replaces the previous run_guard unit, which was a derivative of an
Apache-2.0 file upstream and could not stay in an MIT tree. Written from a
behaviour contract with no reference to the removed code.

Tests: 65 checks, standard library only, single translation unit, clean under
-Wall -Wextra. Mutation tested at 23/23 killed; two of those mutants exposed
missing coverage and the tests were extended until they died. ThreadSanitizer
clean.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): make the B10 test able to fail, and document LaneEntry

Review of the previous commit found the B10 test could not fail for the reason
it was named. It aged the in-flight run to about 120 ms and then tried two
budgets, 30 ms and 60 ms, both under that age, so both callers took the
fail-fast path. "The two failure modes do not share one message" was comparing
two fail-fast messages that differ only in the budget they print, and the
timeout path was never reached. The second budget is now 400 ms, well over the
run's age, so that caller queues and times out, and a new check asserts which
path each caller took instead of inferring it from inequality. A mutant that
makes the fail-fast path emit the timeout message previously died only on B4 and
B8 checks; it now also dies on B10.

Comment-only changes elsewhere. LaneEntry now says it is not reentrant and does
not detect reentrancy: a second entry on a thread that already holds the lane
surfaces as LaneUnavailable with a positive budget, but parks silently in
unbounded mode, which matters because a handler may hold one across a whole
stream. The immovability note now names the shapes that work, an optional
emplaced in place or a unique_ptr, rather than saying to hold the entry
indirectly without saying how; all three documented forms were compiled before
being written down, which is how the note came to say that an optional of an
immovable type cannot itself be returned.

The header's explanation of why fail-fast exists is reworded. Two clauses traced
back to a specification written after reading the Apache-2.0 upstream header,
and while that was judged de minimis, this unit was rewritten precisely to carry
no upstream expression at all.

The margin table in the report was also wrong about which wall-clock margins are
load-sensitive: there are four, not one, and the tightest is the B3 arrival
check, which is now flagged at the call site. No margin value changed and none
moved across 65 runs.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): gate model loading on the audio.cpp family

Refuses any GGUF without an audiocpp.model_spec.family key and any non-GGUF
path without an explicit family option, so the model loader's greedy backend
probe cannot bind an unrelated llama.cpp GGUF to this backend (#9287).

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): load models and cache sessions per task

Loads one ILoadedVoiceModel and creates an IVoiceTaskSession lazily per
(task, mode), so the same model serves both the unary and streaming RPCs.
LoadModel derives the family from GGUF metadata or an explicit option and
fails with INVALID_ARGUMENT otherwise, so a failed load is a gRPC error the
backend probe can see.

audiocpp_backend::Task mirrors engine::runtime::VoiceTaskKind positionally,
and drift there is silent: every unit still compiles and every test still
passes while the backend runs a different task. Two mechanisms pin it. The
static_asserts in loaded_model.cpp catch an insertion or a reorder, and
-Werror=switch on that one file turns an appended upstream enumerator into a
build failure rather than a warning in a 600 file log.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): stop aborting the process on SIGTERM

The signal handler called grpc::Server::Shutdown directly. Shutdown takes an
absl::Mutex, which is not async-signal-safe: the handler can interrupt a thread
already holding that mutex, and abseil's deadlock detector responds by aborting.
Every SIGTERM therefore ended in exit 134 and a 'dying due to potential
deadlock' stack rather than a drained shutdown.

The handler now sets a lock-free atomic and returns. Server::Wait moves to a
helper thread so the main thread can poll that flag and call Shutdown itself,
outside any signal context. A condition variable would not have helped, because
notifying one from a handler is not async-signal-safe either.

SIGTERM and SIGINT both exit 0 with no stack trace, where both previously
exited 134.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): correct the status, lifetime and state contracts of LoadedModel

An environment fault during session creation was reported as UNIMPLEMENTED. A
missing libggml-cpu-*.so surfaced to the client as 'family silero_vad advertises
vad/offline but refused to create the session: Failed to initialize CPU
backend', which tells LocalAI the model cannot do this and must never be
retried, and sends an operator hunting a capability bug instead of a packaging
one. A throw from create_task_session is now a plain runtime_error, so it maps
to INTERNAL. Only a null return, where the family genuinely declined, stays a
CapabilityError.

The model.'s task: option was parsed and then dropped: it lived in a local that
died at the end of LoadModel and had no route to RequestShape::pinned_task.
LoadedModel now keeps it and exposes pinned_task().

The global model becomes a shared_ptr reached through snapshot(). An audio RPC
runs for seconds and cannot hold g_model_mu for its duration, so under a
unique_ptr a Free arriving mid-request would destroy the model underneath it.
Handlers now take a counted reference and whichever finishes last does the
teardown, outside the lock.

session_for documents the streaming state contract rather than resetting the
session itself. Resetting on a cache hit was tried first and is not possible:
silero_vad throws 'session prepare() must be called before Silero VAD reset()',
so it would turn an ordinary second fetch into a hard error. start_stream's base
implementation is already a reset, so a caller that runs prepare then
start_stream per stream gets a clean session; a probe against the bundled
silero_vad confirms an identical replay when it does and a carried-over stream
when it does not.

Also: an unknown backend: name is rejected before the model loads rather than
after; MainGPU is parsed instead of passed through std::atoi, which turned
'gpu1' into device 0 silently; and device carries a device_set flag, because 0
is both the default and a real device index, so MainGPU was overriding an
explicit device:0 that the neighbouring threads: handling promises will win.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serve the VAD and Diarize RPCs

Both emit float seconds, converted from the runtime's sample-index spans, and
both take a counted reference to the loaded model through snapshot() and hold it
for the whole call: a Free arriving mid-request drops only the global's
reference, so whichever request finishes last destroys the model instead of one
of them running on freed weights. An AddressSanitizer build reproduces exactly
that heap-use-after-free inside ggml_vec_dot_f32 when the handler keeps a raw
pointer instead, which is why the shape is what it is.

The inference lane is taken before session_for, not after. session_for reads and
writes an unsynchronised session cache and the offline run calls prepare(),
which mutates the session, so both belong inside the lane.

Diarize routes before it reads the input file, so a family that cannot diarize
at all says so rather than complaining about the audio first. Its per-segment
text stays empty because audio.cpp's SpeakerTurn carries a span and a speaker
label only, and nested or overlapping turns are passed through untouched: a
sortformer turn inside another speaker's turn is correct output for overlapped
speech, and LocalAI is overlap-tolerant downstream. Duration counts frames
rather than floats, so a stereo input does not report twice its length.

Verified end to end against upstream's bundled silero_vad, which needs no
download, using the bundled 16 kHz speech asset: a synthetic tone returns
nothing, correctly, because silero detects speech and a sine is not speech.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): enforce ModelIdentity on VAD and Diarize

audio-cpp was the only C++ backend without the model-identity guard, and no
later task in the plan added it. pkg/grpc/server.go enforces checkModelIdentity
on exactly these two RPCs, for the reason #10952 records: in distributed mode a
worker can recycle a stopped backend's gRPC port for another model's backend,
and the controller's liveness-only probe cannot tell a stale cached route from a
live one. Without this guard a stale route gets a different model's VAD or
diarization answer back with a 200.

The loaded identity lives on LoadedModel rather than in a separate global, which
is where this differs from llama-cpp. A handler holding the model through
snapshot() then necessarily judges against the identity that model was loaded
with, and a concurrent reload cannot swap one without the other. The refusal is
NOT_FOUND carrying the verbatim grpcerrors.ModelMismatchSentinel substring.

session_for and run_offline now take a const LaneEntry & proof-of-holding
parameter. The rule that both must run under the inference lane was prose, which
is exactly how the plan came to specify the inverted order; it is now a compile
error. Restoring the inverted order fails to build rather than racing on an
unsynchronised session map with a mutating prepare().

Diarize's speaker-hint comment claimed the dropped hints were "not a silent
failure". From the caller's side that is what they are, and backend.proto
documents num_speakers as forcing, so the comment now says plainly that the
forwarding is dead for sortformer and that the family which lands must either
honour num_speakers or refuse it. read_audio_file inspects the error_code from
exists(), so an unsearchable parent directory no longer reports as a missing
file. The VAD handler records the stimulus that actually works, since silero
correctly ignores synthetic tones and the next task would otherwise rediscover
that.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): make the lane and identity guards structural

Two hardenings ahead of the eleven handlers still to be written, both of which
get harder to retrofit later.

The lane proof-of-holding parameter was a const reference, which binds to a
temporary, so session_for(rpc, shape, model->acquire(0)) compiled. Each such
temporary dies at the end of its own full-expression, releasing the lane between
two calls that must share one: precisely the split the parameter exists to
prevent, and the form a future author is most likely to reach for because it
reads as tidy. A non-const reference requires an lvalue, so the temporary form
now fails to compile while the named-local handlers build unchanged. The header
comment no longer implies the check is total either: it proves a lane was taken,
not that it is this model's lane.

The identity check was two lines each handler had to remember, with nothing
failing if a new one forgot them and no C++ equivalent of
model_identity_modalities_test.go to notice. snapshot() becomes
snapshot_unchecked(), whose only legitimate caller is Status, since HealthMessage
carries no ModelIdentity. Handlers go through snapshot_for(), which takes the
counted reference, refuses when nothing is loaded, and runs the identity check
before anything can route. Every handler already has to call something to obtain
the model, so the guarded call is now the shortest path and skipping it means
deliberately typing snapshot_unchecked. A convention that has to be remembered
can rot; this cannot.

Verified: the temporary-argument and inverted-order forms each fail to compile
with the expected diagnostic, the real handlers build, and bypassing the guard in
Diarize alone turns the identity test red on that RPC while VAD stays green.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serve the AudioTranscription RPC

Adds result_map, the engine-to-proto boundary, and wires the offline
transcription RPC.

The handler branches on the ROUTED task: for Asr the request's prompt is
whisper-style decoding context and becomes a request option, for Alignment
the same field IS the transcript to align and becomes the text input.
Routing has already decided which.

The result text is TaskResult.text_output verbatim and is never derived
from the segments. audio.cpp carries transcript text in text_output and
nowhere else, so deriving it returns an empty transcript for every
producer that reports segments without word timing. transcript_assembly
already enforces that; this commit's job is not to undo it at the proto
boundary, and result_map_ctest pins it there.

read_audio_file now takes the sample rate the caller needs. Both file-fed
speech handlers ask for 16 kHz mono, for two reasons: silero_vad and
sortformer_diar refuse anything else outright, which turned an ordinary
44.1 kHz upload into INTERNAL, and nemotron_asr emits word timestamps in
its own 16 kHz feature domain whatever the input was, so only a 16 kHz
buffer makes the emitted nanoseconds right. Zero keeps the file's native
rate and channels, which is what source separation will need.

LoadedModel::check_can_serve answers a capability refusal before the lane
is taken and before the input file is read. Routing is a pure read of the
immutable capabilities, so a model that cannot serve an RPC no longer
waits out somebody else's run to say so. VAD and Diarize use it too.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): stop linking sentencepiece's vendored protobuf

engine_runtime links sentencepiece, whose default SPM_PROTOBUF_PROVIDER
builds the protobuf-lite 3.14.0 sources it vendors. The generated
backend.pb.cc is built against the toolchain's protobuf 3.21.12. Both
ended up in the binary: 476 google::protobuf:: symbols came from the
archive, 278 of them also defined by libprotobuf.so, and the archive won,
because once ld pulls a member in for sentencepiece's own code every
reference binds to the definitions that member carries.

The visible symptom is one function.
ParseContext::ParseMessage(MessageLite*, const char*) is what a generated
_InternalParse calls for a submessage field and for nothing else, so flat
messages parsed and nested ones did not: a TranscriptResult carrying
segments serialized to correct bytes that the same process could not read
back, and TranscriptLiveRequest, a oneof of submessages, could not have
been parsed at all. Underneath that, 3.21 generated code was running 3.14
arena, ArenaStringPtr and ExtensionSet code.

-Wl,--exclude-libs does not fix it. It makes those symbols LOCAL in
.dynsym and the parse still fails, because the binding was decided at
static link time and no visibility flag revisits it.

Setting SPM_PROTOBUF_PROVIDER to "package" before add_subdirectory points
sentencepiece at the protobuf the generated code was already built
against. Zero google::protobuf:: definitions remain in the executable
afterwards, every nested message round trips, and citrinet_asr, which
parses a SentencePiece ModelProto at load time and would break first if
this were wrong, still tokenizes and transcribes correctly.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): fix the segment text a transcription response is built from

Segment text is not decoration. core/http/endpoints/openai/transcription.go
routes response_format text, srt, vtt and lrc through
schema.TranscriptionResponse, which builds the entire body out of
Segments[].Text and never reads the top-level text. So for those four
formats the segment text IS the response.

nemotron_asr emits one word_timestamp per SentencePiece token, and the
word boundary is carried as a LEADING SPACE on the piece ("So", "me",
" call"). join_words inserted a space unconditionally, so
response_format=text returned "So me  call   me  na ture ," while the
correct sentence sat unread in the top-level field. The separator is now
chosen from the words themselves: whole words are space-joined, subword
pieces are concatenated, and one leading space anywhere selects the
latter. Concatenating the real nemotron pieces reproduces text_output
exactly, verified end to end.

This does not touch the top-level text, which is still text_output
verbatim. The rule that forbids deriving the transcript from the segments
is about the direction segments -> text; segment text has no source other
than its words.

Two smaller corrections in the same area:

timestamp_granularities ["word"] set only "word_timestamps", a key no
family in the pinned upstream reads. It now sets "return_timestamps",
which qwen3_asr does read and which both runs its forced aligner and
shortens its chunk window, so asking for word granularity no longer
silently returns nothing.

The request-option comment claimed more than it delivered. prompt,
translate and temperature are read by no ASR family, and are forwarded
only so a family adopting them works unchanged; the comment now says so
per key, and gives TranscriptRequest.diarize the same explicit treatment
threads already had.

Also: the shipping target now carries -Wall -Wextra -Wpedantic, which it
never did, so "the build is clean" starts meaning something; and
fill_transcript_result no longer swallows a null response pointer, since
answering OK with an empty transcript is the one failure mode this unit
exists to prevent.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serve the AudioTransform RPC

Covers voice conversion, singing voice conversion, speech to speech and source
separation, the four tasks LocalAI's AudioTransform can represent.

AudioTransformResult carries one dst while htdemucs and mel_band_roformer
produce several named stems from a single run, so inference runs ONCE, every
stem is written as a sibling file <dst-stem>.<name>.<ext>, and params[stem]
selects which one dst receives, defaulting to vocals and falling back to the
first output. An unknown stem name is INVALID_ARGUMENT listing the real stem
names rather than a silent substitution, and the selection happens before the
first write so a refused request leaves no files behind. params[stem] is
consumed here and is not forwarded into the engine's request options.

The stem decision lives in stem_selection, which is stdlib only and therefore
tested by backend/cpp/run-unit-tests.sh. It also validates the names, because
they come from the model (htdemucs reads them from the GGUF's config.sources)
and each becomes a component of a path this backend writes: a name carrying a
path separator would escape the caller's output directory, and two stems
sharing a name would silently overwrite one another.

Both files are read at their native rate and channel count. Separation forces
it, since demucs and roformer refuse any rate but 44.1 kHz and lose the stereo
image that separates a centred vocal from a wide mix. The conversion families
all resample internally (seed_vc, vevo2, miocodec, chatterbox were each
checked), so passing the file through unchanged is also strictly better than
band limiting it to 16 kHz first.

Verified end to end against htdemucs f16 on a 44.1 kHz stereo mix: four stems
plus dst, dst byte identical to the selected stem, params[stem] selecting a
different one, an unknown stem refused with no files written, and mono input
preserved as mono output. Also against miocodec for the single output path,
where params[stem] is refused rather than ignored.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): refuse an impossible stem early, and stop blaming the caller for a failed write

Four fixes from the first review of the AudioTransform RPC.

check_can_serve now returns the resolved route, so params[stem] on a route that
is not source separation is refused from the route instead of after a full
inference: 11 ms rather than the 4.5 s a miocodec conversion costs, and far
worse on seed_vc or vevo2. The post-run refusal stays as the backstop for a
separation-routed family that returns no stems anyway. The typo'd-stem-name
case still needs the run, since no framework header publishes the stem names
before one.

Stem names carrying control bytes are refused. GGUF strings are length prefixed
and demucs reads its sources from JSON, so an embedded NUL survives to here:
two names differing only after the NUL are distinct std::strings, so the
duplicate check passes them, and then path::c_str() truncates both and they
open the same file. That is exactly the silent overwrite the duplicate check
exists to prevent, with the .wav lost as well.

A failed write is now INTERNAL rather than INVALID_ARGUMENT. The destination is
LocalAI's own generated-content directory, not anything the caller named, so a
full disk or a permission fault there is a server fault and is worth retrying,
which is the opposite of what INVALID_ARGUMENT tells a client. An empty output
path stays INVALID_ARGUMENT.

Two comment corrections and one clarification: the separators' required rate is
their checkpoint's declared samplerate rather than a hardcoded 44100, seed_vc
resamples with soxr and falls back to sinc-hann, and the "no files left behind"
guarantee covers a refused request, not a write that fails partway through the
loop.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(audio-transform): stop folding every upload to 16 kHz mono, and name the separation stems

Two defects that made source separation unusable through LocalAI's own API,
even though the backend served it correctly over gRPC.

/audio/transform normalized every upload to 16 kHz mono s16 through
utils.AudioToWav, with no way past it. htdemucs and mel_band_roformer refuse
any rate but their checkpoint's own and separate a centred vocal from a wide
mix using the stereo image, so every separation request through the HTTP API
died with "HTDemucs prepare() sample rate mismatch: expected 44100, got 16000"
while the same call over gRPC worked. The fold is not wrong, it is
backend-specific: LocalVQE's echo cancellation genuinely wants 16 kHz mono and
needs the reference in the same shape. So it becomes a declaration,
BackendCapability.AudioTransformInputMono16k, set for localvqe and for nothing
else. A backend that declares nothing gets its upload unchanged, which means no
backend has to opt in to work. utils.AudioToWavPreservingShape is the
non-folding conversion: a 16-bit PCM WAV passes through byte for byte at any
rate and channel count, anything else is transcoded to WAV with its rate and
channel layout kept.

The other defect is that the run-once stem design bought nothing. A separation
backend writes every stem beside dst from one inference, but AudioTransformResult
carried only dst, so the other three were files no caller could find and a
caller wanting all four had to run four separations. AudioTransformResult grows
a repeated AudioTransformStem, the backend fills it, core/backend validates that
each path really is inside the generated-content directory it handed over, and
the endpoint publishes them as an X-Audio-Stems JSON header beside the existing
X-Audio-Input-Url. JSON because a stem name is the model's own string and could
contain any separator a hand-rolled format would use.

Verified end to end through the HTTP endpoint with htdemucs f16 on a 44.1 kHz
stereo file: 200 with a 44.1 kHz stereo body, all four stems named and fetchable
through /generated-audio/, body byte identical to the selected stem, and
params[stem]=drums returning a different one. The same upload sent to a model
whose backend is localvqe still reaches the backend as 16 kHz mono, confirmed
both by the engine's own rate refusal and by the persisted input file.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(audio-transform): reject extensible WAV from the passthrough, escape stem URLs, convert stems with dst

Four fixes from the second review, plus one bug they made visible.

isPCM16Wav tested only the bit depth, and go-audio's IsValidFile never looks at
the format tag, so a 16-bit WAVE_FORMAT_EXTENSIBLE (0xFFFE) upload was passed
through untouched where the old fold would have transcoded it. audio.cpp's WAV
reader accepts 16-bit only when the tag is 1, so such a file died with
"unsupported WAV encoding". Extensible is what many DAWs and Windows tools
write and music files are this endpoint's new headline input, so it is a
first-contact failure rather than a corner. The check now requires tag 1, with a
spec that fails against the old implementation.

Stem URLs are percent-escaped. A stem name is the model's own string and legally
contains a space, a '#', a '?' or a '%'; an unescaped '#' truncates the URL
before the request is even sent. The name field keeps the raw name.

sample_rate and response_format are applied to the stems as well as to dst.
Applying beat documenting: dst IS one of those stems, so leaving them alone
broke the "dst duplicates the selected stem" invariant the whole design rests
on, and both conversions are no-ops when unset. A stem whose conversion fails is
dropped from the header rather than advertised in the wrong shape.

Verifying that turned up why it had never been noticed: the two fields were
never bound at all. The request arrives as multipart/form-data and echo's binder
falls back to the FIELD NAME without a form tag, matching only
case-insensitively, so "SampleRate" never matched "sample_rate" and "Format"
never matched "response_format". Both were documented in the endpoint table and
silently ignored. Two form tags fix it, and with them the conversion is
observable end to end.

Docs: audio-transform.md now documents what LocalAI does to an upload before the
backend sees it, which backend gets the 16 kHz mono fold and why, params[stem],
and the X-Audio-Stems header with a worked example.

Also records the known limitation that the fold lookup is on the bare backend
name, so pinned variants (vulkan-localvqe) do not match, and points at
IsLlamaCppBackend as the suffix-tolerant precedent.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serve the TTS and SoundGeneration RPCs

TTSRequest.voice is treated as a speaker reference clip when it names an
existing regular file, which makes routing prefer VoiceCloning, and as a named
preset otherwise, in which case it travels as VoiceReference::cached_voice_id.
Both the clip and SoundGenerationRequest.src are read at the file's own rate and
channel count: upstream's own CLI and server do exactly that, every consuming
family resamples internally and mostly with a better resampler than ours, and
ace_step and stable_audio resample their input per channel, so a downmix here
would delete the stereo image they are built to consume.

The request builders live in their own unit rather than in grpc-server.cpp's
anonymous namespace so they can be tested; grpc-server.cpp has a main() and
cannot be linked into a test binary. The option keys are the whole point of
these functions, so each one was grepped against the pinned upstream and the
accounting is written down beside it. instructions maps to "instruct", which is
what upstream's own server maps the OpenAI field to and what qwen3_tts and
omnivoice read, and to "caption" for irodori_tts; the style tag is spelled
"instruct" too, because "instructions" is looked up nowhere. duration maps to
"duration_seconds", read by all three generation families, with the proto's own
name kept only as a forward-tolerant alias. Keys that no family reads say so.

Both handlers answer a capability refusal before taking the lane and before any
file read, so a model that cannot synthesise does not queue behind somebody
else's run to be told no.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): stop emitting an empty style language, and name the missing clip

StyleCondition::language was set whenever has_language() was true, with no
!empty() guard, while the language option twelve lines below had one.
core/backend/tts.go sets Language unconditionally, so has_language() is true on
every request LocalAI sends and carries "" when the caller named none. An
engaged-but-empty style language is worse than an absent one: supertonic reads
text_input->language behind its own !empty() guard and then overrides it from
style->language with no guard at all, so "" replaced its "en" default and its
tokenizer threw "invalid Supertonic language: ". Every /v1/audio/speech request
that set instructions and no language would have been an INTERNAL against a
supertonic model. A plain request never saw it, because the style condition only
exists when instructions are non-empty, which is why the chatterbox end to end
run did not catch it.

TTS also stops discarding the Route that check_can_serve already returns. A
family routed to voice cloning without a reference clip used to be refused from
inside its own prepare(), which meant an INTERNAL naming neither the RPC nor the
field to set; chatterbox advertises clon and no tts, so that was every
preset-only request to it. It is now an INVALID_ARGUMENT naming
TTSRequest.voice, answered in about 4 ms, and it cannot misfire because
has_voice_reference is what selected cloning in the first place. Reading
CapabilitySet::supports_speaker_reference to generalise this stays a follow-up.

The src read carries a written caveat rather than a family blocklist, because
ace_step's editing routes legitimately need src: setting src on a stable_audio
model corrupts the heap and aborts the process in the pinned upstream, and the
only thing keeping that off the network is that
schema.ElevenLabsSoundGenerationRequest has no field for it. Nobody reading that
Go schema would know why, so the reason is recorded where the field is read.

build_tts_shape is extracted so TTSStream cannot describe the same request
differently, and it arrived untested: two mutations of it survived until a
test_tts_shape case was added.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serve the TTSStream and AudioTranscriptionStream RPCs

TTSStream leads with a streaming WAV header carrying 0xFFFFFFFF sizes, matching
the convention backend/go/vibevoice-cpp established, so an HTTP client can start
playback before the full PCM exists. Its chunks are read from
StreamEvent::named_audio_outputs and not audio_output: supertonic, omnivoice and
voxcpm2 all put their streamed audio there and leave audio_output empty until
the very end, so reading the obvious field yields a stream with no audio in it.
The finish_stream result is the family's own merged whole rather than a tail, so
it is emitted only when nothing was streamed.

Streaming transcription sends incremental deltas and degrades to a single delta
plus the final result on families that offer no streaming ASR, which is the same
message sequence with fewer deltas. The four streaming ASR families disagree on
what partial_text means: nemotron_asr, vibevoice_asr and higgs_audio_stt report
incremental fragments while voxtral_realtime reports the whole hypothesis and
reports it twice, so the reconciliation lives in one tested unit rather than in
the handler. nemotron_asr reports only through the stream event sink, and only
from inside finalize, so the audio driver installs one and clears it again
before returning: the session is cached and a sink left holding the caller's
frame is a use after free waiting for the next stream.

begin_stream is now the only implementation of the streaming state obligation,
prepare then start_stream. Streaming sessions are cached, and what clears the
previous stream is start_stream's reset; a family override that dropped it would
break every call site with no compile error, so there is one call site.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): keep streaming deltas on UTF-8 boundaries, refuse dtypes that abort

TranscriptStreamResponse.delta is a proto3 string, whose wire format requires
valid UTF-8. voxtral_realtime reports its hypothesis as a concatenation of raw
token BYTES (tokenizer_text.cpp:171-183), so the cumulative difference between
two consecutive reports is eventually a lone continuation byte, and the C++
runtime serializes that with only a logged warning while the Go runtime refuses
to unmarshal it: the client loses the remaining deltas AND the final_result.
Measured on a trace of a non-ASCII sentence, 11 of 31 messages failed to
unmarshal and every accented character was lost. TranscriptDeltaTracker now
holds back an incomplete trailing sequence and merges it into the next fragment;
reconcile flushes it, which it always can because the final text is complete.
The same trace now unmarshals in full with zero failures.

A streaming buffer whose float count is not a whole number of frames is refused
rather than truncated. The integer division dropped the tail floats from the fed
audio and therefore from the transcript, with no diagnostic; vibevoice_asr
refuses the same thing from the other side of the call.

A supertonic GGUF whose weights are not f32 is refused at load. It reaches
ggml_concat with mismatched operand types and ggml_abort takes the whole backend
process down on the first request, so nothing downstream can report it: the
model loads, then every request kills the process. Attributed rather than
assumed, the unary TTS path aborts identically, and upstream records that
package as untested. The refusal names the orig package and says what to run
before deleting the guard.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): stop a repeated lead byte from orphaning the next delta

The first UTF-8 fix closed the cumulative half only. Rule 2 discards a fragment
the known text already starts with, and when that fragment is the LEAD BYTE of a
new character it looks exactly like a repeat of an older character beginning
with the same byte. It was discarded rather than held, its continuation bytes
then arrived alone and began the next delta, and utf8_complete_prefix_length
only ever inspected the trailing sequence, so a delta invalid at the FRONT went
out whole. Through a real Go proto.Unmarshal the review's four-character repro
gave 3 deltas, 2 unmarshal failures and a lost transcript.

Reachable from the incremental families, not only from voxtral: nemotron_asr's
decoder cuts at a byte offset and vibevoice_asr's common_prefix_size compares
bytes, so both split characters. Measured over 30,000 randomized incremental
traces, 53.28% of Japanese traces and 9.52% of French ones carried at least one
delta the Go runtime refuses.

Two changes. Rule 2 no longer judges a fragment that ends mid-character, so the
lead byte is held instead of swallowed and the character survives intact; the
cost is a few duplicated bytes in a shrinking cumulative report, which no pinned
family produces. release() additionally drops leading orphan continuation bytes,
so no delta can begin mid-character whatever the rules above it decide. Losing a
byte keeps the stream alive; emitting one ends the RPC and takes the
final_result with it.

Post-fix all 60,000 traces produce zero unmarshal failures, and the cumulative
streams plus both pure-ASCII incremental streams are byte-identical to the
previous commit, so nothing changed for the families already working.

The weight-dtype allow list moves to family_gate, where it is stdlib-only and
pinned by a test rather than only by a comment. Two comment citations corrected.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): read only an exact repeat as a repeat, not any prefix

Rule 2 discarded any partial the known text merely started with. For a cumulative
family that is a duplicate; for an incremental family it is an ordinary short
fragment that happens to coincide with the start of the transcript, and it was
dropped, silently corrupting the text. Pure ASCII, no multi-byte character
anywhere: the fragments "pure ", "ascii ", "trans", "c", "ri", "p", "t" left the
client holding "pure ascii transcrit". Over 5,000 randomized traces per
transcript, 9.50% of pure-ASCII and 29.12% of French traces ended with the client
holding something other than final_result.text, with a 200 and no diagnostic.
Both incremental families emit fragments that small routinely, since nemotron_asr
cuts at a byte offset and vibevoice_asr at a common prefix.

Narrowing rule 2 to an exact repeat drives that to zero on all six transcripts
and changes no cumulative stream at all: 30,000 randomized cumulative traces are
byte-identical to the previous commit.

What rule 2 guarded was established from upstream rather than from its own
comment. The only duplicate any pinned family produces is voxtral_realtime's,
where process_available_stream_chunks feeds each event to the sink from inside
its loop and returns the last of the batch, so that event arrives twice with
byte-equal text. A duplicate is an exact repeat, so equality still covers it. The
case given up is a cumulative report that SHRINKS, which no pinned family can
produce: voxtral decodes a token vector that is only push_back'ed and cleared by
reset(), so within a stream it can only grow.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): serve the AudioTranscriptionLive RPC

The one bidirectional stream this backend serves. The client sends a
TranscriptLiveConfig, then TranscriptLiveAudio frames; the server acknowledges
with ready, emits deltas as the audio arrives, and sends final_result once the
read side closes. There is no offline fallback: live transcription has to
consume audio incrementally, so a family with no streaming ASR is refused
rather than served a batch run, which is what this RPC's Streaming-only
mode_candidates list already says.

The driver is a new sibling of run_streaming_audio, run_streaming_live, because
the audio does not exist yet: instead of slicing a buffer it pulls frames from
the caller until the read side closes. It installs the same ScopedStreamSink in
the same order, which is not optional, since nemotron_asr returns a bare event
from process_audio_chunk and reports every partial through the sink from inside
finalize(). It buffers the wire's frames up to the family's own preferred window
rather than feeding whatever size the client's audio callback produced, and it
does not call finish_stream at all when no audio arrived, because nemotron_asr
throws "finalize requires streamed audio" and an empty transcript is the
truthful answer to transcribing nothing.

Three things the handler had to get right and one it cannot:

  - The audio contract. A live request carries no samples, but nemotron_asr's
    streaming prepare() throws without an audio contract, and
    build_preparation_request derives it from TaskRequest::audio_input, so that
    field is an EMPTY buffer holding only the rate and the channel count.
  - 16 kHz or a refusal. The families express their spans in their own 16 kHz
    feature domain whatever the input was, and live frames cannot be resampled
    on the way in the way a file can, so an 8 kHz session would return
    timestamps 2x off with a 200. core/backend hardcodes 16000 anyway.
  - A mid-stream Config is refused. backend.proto calls it a decoder reset, but
    deltas already on the wire cannot be retracted, so a reset would leave the
    final text contradicting the transcript the client assembled. Ignoring the
    message would hand a client that believes it reset the decoder a transcript
    that silently continues the audio it thought it discarded.
  - The stale-route identity check cannot run here: TranscriptLiveRequest
    carries no ModelIdentity in either arm of its oneof, so snapshot_for does
    not instantiate for it. snapshot_unchecked's comment now names that as a
    second legitimate class of caller and says the fix is a proto change.

eou and eob stay false. They exist for cache-aware models that emit
end-of-utterance and end-of-backchannel tokens; audio.cpp's StreamEvent has no
equivalent signal, and a client uses eou to decide the speaker yielded the turn,
so a guess inferred from silence cuts people off mid-sentence.

The lane is held for the whole stream, which is as long as the user keeps
talking: the streaming session is stateful and cached, so a concurrent run would
interleave two callers' audio and corrupt both transcripts.

Verified against nemotron_asr over a real connection with a 14 s WAV in
512-sample frames: ready first, 59 incremental deltas with no repeated prefix,
concat(deltas) equal to final_result.text, word timestamps in nanoseconds, eou
and eob false. citrinet_asr answers UNIMPLEMENTED naming the family and listing
asr/offline. A config followed by a close returns an empty final_result rather
than hanging, and a first message that is not a config is INVALID_ARGUMENT. Two
concurrent streams both return the complete transcript.

Two cleanups on lines Task 12 touched, folded in. The DtypeAllowList terminator
is now asserted at compile time: the reported out-of-bounds read did not exist,
the single entry does terminate, but the loops have no other bound and any edit
that widened an entry would walk off the end. And the dtype guard now
short-circuits on "is there a table entry" through a new predicate rather than
on the emptiness of the description string, which would have skipped the check
on an entry with an empty allow list, i.e. on precisely the entry that refuses
every dtype.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): bound the lane a live stream can hold

AudioTranscriptionLive holds the model's inference lane for the whole stream,
which is correct (the streaming session is stateful and a concurrent run would
interleave two callers' audio) and newly dangerous. Every other RPC holds the
lane across compute, or across a write to a slow reader, and both of those
terminate on their own. A live stream instead blocks in a client-driven read,
and a peer that goes silent WITHOUT closing the stream never terminates
anything: the lane stays taken and every other request against that model queues
behind a client that stopped speaking.

live_watchdog is a one-shot idle timer that ends the stream when no frame has
arrived inside a window. It is standard library only, so it is unit tested
without an engine. gRPC's synchronous Read has no timeout and cannot be given
one, so the only way to unblock it is ServerContext::TryCancel, which decides
the wire status itself: the client sees CANCELLED rather than the
DEADLINE_EXCEEDED the handler returns, the reason is logged, and the lane coming
back is the point. When it fires the read loop throws rather than reporting
end-of-input, so the driver does not go on to finalize a decode nobody is
waiting for.

It is armed only after the lane is taken and disarmed as soon as the read side
closes, and both ends matter. Arming earlier would cover acquire(), which
legitimately blocks while another live stream runs, so a queued caller would be
cancelled for waiting its turn. Disarming later would cover our own decode,
where a window overrun is not a peer going quiet and cancelling would throw away
the transcript the client is waiting for.

The window is the new live_idle_timeout_ms option, 30 s by default, 0 meaning no
limit. core/http/endpoints/openai/realtime.go drives a 300 ms ticker and feeds
every tick that produced new audio while a turn is open, so 30 s of silence is a
hundred ticks that delivered nothing. It is also longer than any pause a speaker
takes mid-utterance, which is the case that must never be cut off, and
backend.proto lets one stream span many utterances, so a client that pauses
longer between them raises the option rather than discovering it.

Two smaller corrections in the same handler:

  - check_can_serve now runs BEFORE the sample rate check.
    pkg/grpc/grpcerrors/errors.go degrades to the file path on UNIMPLEMENTED and
    on nothing else, so a live-incapable model asked at a wrong rate was
    answering INVALID_ARGUMENT and costing the caller its fallback.
  - a negative sample rate is refused instead of silently becoming 16000. Zero
    still means 16000, which is what the proto documents; -1 is malformed rather
    than absent and gets the same refusal every other bad rate gets.

And one thing recorded rather than changed, at the handler: "live" here means
incremental INPUT, not low latency, and with the pinned families it does not yet
mean incremental OUTPUT either. nemotron_asr's process_audio_chunk only appends
to its buffer, so its whole decode and every delta happen inside finalize(),
after the client closes its send side. The policy-window buffering is inert for
that family and matters only for vibevoice_asr and higgs_audio_stt.

Verified on the wire with live_idle_timeout_ms:3000. A silent client acked at
371 ms and was cancelled at 3.371 s; a second live stream opened one second
later received its ack 2.37 s in, i.e. at the instant the first was cancelled,
and then transcribed successfully on the same cached session. Without the
watchdog it would still be waiting. Re-ran the live transcription (ready first,
59 incremental deltas, concat equal to the final text, word timestamps in
nanoseconds, eou and eob false), the citrinet refusal at both a right and a
wrong rate (UNIMPLEMENTED either way now), and Task 12's AudioTranscriptionStream
on nemotron_asr, which is unchanged.

Mutation testing the watchdog found a weakness in its own test: the destructor
test slept past the window inside the watched scope, so a destructor that
DETACHED the thread instead of joining it passed unnoticed. The test now uses a
window longer than the scope, which kills that mutant, and says why.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): refuse the unsupported RPCs with a reason

AudioEncode, AudioDecode, AudioTransformStream, AudioToAudioStream and
VoiceEmbed have no counterpart in audio.cpp's VoiceTaskKind. Each now returns
UNIMPLEMENTED naming the loaded family, what that family does support, and the
upstream limitation, instead of the generated base class's bare status. The
reasons live in a table in capability_routing.cpp so they are data rather than
literals copied into five handlers, and so a test can assert every one of them.

The five claims this was planned against were re-read at the pinned upstream
e800d435d130dc776baf6f3e6129bb62b1495c89, and one did not hold. "audio.cpp
streams tts and asr only" is false: silero_vad advertises vad with
RunMode::Streaming. The refusal stands on the narrower claim that survives, that
no family advertises streaming for any task AudioTransform routes to, and a test
asserts the refuted wording does not come back.

VoiceEmbed is the one refusal whose request carries a ModelIdentity, so it runs
the #10952 check before answering: a stale route must get NOT_FOUND and the
router's sentinel, not "audio.cpp cannot embed speakers" about a model that is
not loaded here. It cannot use snapshot_for, whose no-model branch would tell
the caller to load a model when no model can help, so it takes the reference
through snapshot_unchecked and checks identity itself. That function's comment
now names three classes of caller instead of two.

The two bidirectional surfaces refuse without reading their stream, verified
with a client that writes a config and eight frames first and gets the status
rather than hanging.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): correct the vevo2 clause, and assert the absences

Review found a false clause in the AudioToAudioStream refusal. It said s2s is
"offline voice conversion ... which converts one clip into another speaker's
voice", which is true of miocodec and false of vevo2: vevo2's s2s route is
`editing` and only `editing` (default_route_for_task and route_matches_task in
src/models/vevo2/session.cpp), documented as "Edit source speech into new target
text while using the target voice" and requiring --target-text, so it rewrites
what was said. vevo2's voice conversion is its separate vc task. It now reads
"offline clip-to-clip processing against a target voice, declared only by
miocodec (voice conversion) and vevo2 (speech editing)", and a test asserts the
miscast cannot come back. The conclusion is unchanged: neither family converses.

That defect was undetectable on the wire, since vevo2 does not load here, which
is the argument for upstream_absence_ctest.cpp. It links engine_runtime purely
to interrogate make_default_registry() and asserts the five premises the refusal
reasons rest on: no codec task kind, no family advertising spk, no streaming for
sep/vc/svc/s2s, miocodec advertising exactly vc and s2s, and s2s advertised by
exactly miocodec and vevo2. The last two are exact sets, so an addition fails
here rather than leaving a message stale. A positive control proves the registry
is populated and the query works before any absence is believed, and every
assertion has a reproduced negative control. This turns an AUDIO_CPP_VERSION
bump from "remember to re-read five prose paragraphs" into a test failure.

unsupported_surface now switches over UnsupportedRpc with no default label, so
-Wswitch reports a sixth enumerator added without a row at build time; the
runtime bounds guard it replaces is deleted.

The AudioTransformStream reason had a true premise and an overreaching
conclusion: an offline sep family could be buffered into a stream, as other
LocalAI backends do. It now says this backend declines to offer a buffered
offline call in disguise, rather than implying impossibility.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): make the missing-switch-case diagnostic fatal

unsupported_surface() switches UnsupportedRpc onto the table row that explains
it, with no default label, so -Wswitch reports an enumerator nobody handled. As
a warning that is not enough: adding a sixth enumerator and building the shipping
target gives exit 0, a binary and one warning, and the trailing
`return surfaces[0];` then answers the new RPC with AudioEncode's codec reason.
That is a confident, specific and false statement about audio.cpp on the wire, on
the one code path whose entire job is to be truthful about what this backend
cannot do, and it is worse than the runtime fallback it replaced, which at least
named itself as a bug in this file.

capability_routing.cpp therefore joins loaded_model.cpp on the existing
-Werror=switch pin, whose comment already made this argument for the engine enum.
The comment now covers both files. The pin stays per-file rather than
project-wide because upstream's own ace_step/vae_decoder.cpp has unhandled
-Wswitch cases of its own.

Verified: a sixth enumerator now fails `make grpc-server` with exit 2 and no
binary; appending a 14th VoiceTaskKind upstream still fails loaded_model.cpp, so
the two pins fire independently; both reverted clean.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): package the backend image

Bundles the dependency closure for the from-scratch image, the dlopened ggml
CPU-variant shared objects that ldd cannot see, and upstream's bundled
silero_vad and marblenet_vad assets so VAD works with no download.

The bundled loader sits in the package ROOT rather than at lib/ld.so. run.sh
execs it, which makes /proc/self/exe name the loader, and this backend has two
consumers of that path: ggml discovers the libggml-cpu-*.so by listing
dirname(/proc/self/exe), and resolve_model_path expands bundled:<name> under the
same directory. Rooting the loader makes the binary, the ggml objects and
assets/ share the one directory all three resolution mechanisms agree on.
llama-cpp's lib/ld.so layout would need assets/ moved into lib/ as well.

The image builds against apt gRPC and protobuf, like Dockerfile.ds4 and unlike
Dockerfile.privacy-filter. The from-source gRPC that install-base-deps.sh and
the base-grpc-* images supply vendors protobuf 26, which pulls abseil into
message_lite.h; with SPM_PROTOBUF_PROVIDER=package that collides with
sentencepiece's vendored mini-abseil and every absl::internal reference becomes
ambiguous. Noble's protobuf 3.21.12 predates the abseil dependency and is the
pair every earlier verification of this backend ran against.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): exempt the driver libraries from the packaging gate

package.sh already left libcuda.so* and libnvidia-* to the host when copying,
because the driver has to match the kernel module on whatever host runs the
image, but the validation gate had no matching exemption. With BUILD_TYPE=cublas
ggml is static and links CUDA::cuda_driver, so grpc-server carries DT_NEEDED
libcuda.so.1 and the gate would have rejected the very absence the copy loop
created, failing every cublas build in CI. One regex now feeds both.

Building a control for that found a second defect: ld.so --list refuses to trace
an object with an unresolvable dependency at all, exiting 127 without emitting a
per-library line, so the "=> not found" rule was dead code and no exemption could
have applied to it. The gate now traces with LD_TRACE_LOADED_OBJECTS and
LD_LIBRARY_PATH, which reports the missing name and exits 0, and which is also
what run.sh does at run time.

Adds a layout assertion so a future move of the loader into lib/ fails the build
instead of shipping a package that resolves bundled: models into lib/assets and
finds no ggml CPU backend, and records for Task 16 that the Darwin script must
not be a straight copy of privacy-filter-darwin.sh, which never calls package.sh
and would silently drop assets/.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): register the backend with CI and the gallery

Adds the five Linux matrix entries (cpu amd64/arm64 sharing a tag-suffix so the
manifest merge fires, cuda 12, cuda 13, vulkan), the path-filter case that keeps
later PRs touching backend/cpp/audio-cpp/ from getting zero CI jobs, the
bump-bot entry pointing at the AUDIO_CPP_VERSION pin in the backend Makefile,
the gallery meta plus its -development variant and the image entries for every
variant, and the Makefile docker-build wiring.

The matrix entries carry base-image only, with no builder-base-image, unlike
the llama-cpp and privacy-filter blocks they sit next to. The prebuilt
quay.io/go-skynet/ci-cache:base-grpc-* images ship a from-source gRPC whose
protobuf v26 depends on abseil, and this backend's sentencepiece is built with
SPM_PROTOBUF_PROVIDER=package, so it sees real abseil's absl::lts_20240116::
internal alongside its own vendored plain absl::internal and every
absl::internal:: reference becomes ambiguous. Building against base-grpc-amd64
fails at sentencepiece-static.dir/error.cc.o with "reference to 'internal' is
ambiguous". Dockerfile.audio-cpp installs apt's gRPC/protobuf 3.21.12 itself,
which is also the pair every unit and end-to-end run of this backend has been
verified against, and the CUDA toolkit therefore has to come from base-image.

No Darwin matrix entry and no metal gallery entries: the Metal build needs
scripts/build/audio-cpp-darwin.sh, a backends/audio-cpp-darwin make target and
a routing step in backend_build_darwin.yml, none of which exist yet, so an
entry added now would be routed to build-darwin-go-backend and look for
backend/go/audio-cpp/. The inferBackendPathDarwin case and the
DARWIN_BESPOKE_BUILDERS membership are in place, inert, so that adding the
entry later is a one-line change that cannot be claimed by the generic Go path.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): pin the CUDA architectures, drop the vulkan variant

Upstream sets CUDA_ARCHITECTURES to `native` on the engine_runtime target
whenever CMAKE_CUDA_ARCHITECTURES is unset at root scope, and docs/build/
linux.md says so outright. ggml's own default does not rescue it: it
list(APPEND)s in the ggml subdirectory scope, which never reaches the root
scope where the engine_runtime property is decided. No CI runner has a GPU for
`native` to enumerate, so both cublas entries would have gone red on the very
commit that first turns a CUDA build on.

Pin the list in backend/cpp/audio-cpp/Makefile, selected by CUDA_MAJOR_VERSION,
which Dockerfile.audio-cpp now forwards from the CI build-arg it was previously
discarding. The values are copied from ggml's own version guards rather than
invented, so engine_runtime and ggml compile for the same set: CUDA 12 keeps the
Maxwell/Pascal/Volta virtual archs and stops at 120a-real, CUDA 13 drops them
and adds 121a-real. The `a` suffix is used rather than `f` because the latter
needs CMake 3.31.8 and Ubuntu Noble ships 3.28.3. Verified by driving CMake
3.28.3's own CUDA architecture validator over both lists, with 120f-virtual as
the rejected control.

Drop the vulkan matrix entry, its two gallery entries, the vulkan capability
key on both metas and the Vulkan tag. Every other vulkan backend gets its Mesa
ICD drivers from .docker/install-base-deps.sh, which package-gpu-libs.sh then
bundles; Dockerfile.audio-cpp calls neither and installs only libvulkan-dev and
glslc, so the image would ship a Vulkan loader that finds no GPU. No CI job runs
a vulkan image against real hardware, so that would have passed green and failed
in users' hands. BUILD_TYPE=vulkan stays supported for local builds.

Also note on the cublas entries that cuda-major-version now selects the
architecture list and that cuda-minor-version and the base-image tag encode the
same toolkit, and correct the stale entry counts on matrixEntryKey.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): build for Darwin Metal

Bespoke C++ Darwin path like ds4 and privacy-filter: an includeDarwin matrix
entry, a backends/audio-cpp-darwin make target, a gated workflow step, and the
metal image entries plus metal/metal-darwin-arm64 capability keys in the
backend gallery.

The build script deliberately does NOT reassemble the package the way
privacy-filter-darwin.sh does. It runs the backend's own `make package` and
copies the result, so the Darwin package keeps the root-level layout the Linux
one has: grpc-server, run.sh, the ggml objects and assets/ in one directory,
with lib/ for the dylib closure. Hand-assembling would drop assets/, and
assets/ is what makes the bundled: model paths resolve with nothing downloaded.
The dylib walk is a full transitive closure rather than the single level ds4
and llama-cpp do, because Homebrew's grpc++ pulls libgrpc, abseil, upb, cares
and OpenSSL that grpc-server does not link itself, and a level-1 walk ships a
package that only works on a machine that already has Homebrew grpc.

Two fixes folded in, both in the backend Makefile:

  - an EMPTY CUDA_MAJOR_VERSION fell through to the CUDA 12 architecture list,
    which contains 120a-real and so needs nvcc >= 12.8. A local
    BUILD_TYPE=cublas build on a 12.0-12.7 host failed to compile where
    upstream's documented default (native) worked. EMPTY now maps to native,
    12 and 13 keep their lists, and any other non-empty value is an error on
    cublas builds. CI always passes a major, so CI is unaffected.

  - the Darwin branch now points CMake at Homebrew's keg-only libomp. AppleClang
    ships no OpenMP runtime and nothing is symlinked into /opt/homebrew, so
    FindOpenMP finds neither the library nor the header, and audio.cpp calls
    find_package(OpenMP REQUIRED) whenever ENGINE_ENABLE_OPENMP is on. Without
    the hint the macOS build would have died at configure time. If the keg is
    absent the build disables OpenMP instead of failing.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): make the Darwin fallbacks loud and the rpath walk complete

Review follow-up on the Darwin Metal build.

The OpenMP fallback was silent. If brew --prefix libomp ever comes back empty,
CI produced a green Metal package with 108 #pragma omp directives across ~30
files compiled out, and clang says nothing about an ignored omp pragma without
-Wsource-uses-openmp, so the only trace was one absent flag inside a set -x
cmake line. That regression would have been blamed on Metal. It now warns.

The @rpath arm of the dylib walk had no live candidate when it was written, on
the reasoning that a Metal build links ggml statically. The OpenMP fix in the
same commit made libomp.dylib one, and whether Homebrew records it as an
absolute opt path or as @rpath/libomp.dylib is not observable from Linux. The
walk now expands @rpath, @loader_path and @executable_path against the object's
own LC_RPATH entries, and only fails when nothing on disk answers, printing the
rpath list with the error so a failure on a machine nobody can attach to
explains itself.

Also: ADDITIONAL_LIBS now go through the closure rather than a bare cp, so they
are deduplicated and their own dependencies bundled; build/darwin/lib is
created explicitly instead of relying on package.sh pre-creating it; the libomp
probe uses nested ifneq rather than $(and ...), which needs GNU make 3.81 and
would otherwise expand empty and take the OFF branch on an older make; and
-DOpenMP_ROOT is quoted like its CUDA sibling.

Verified with a Linux harness that runs the script verbatim against a stubbed
otool: a level-2 transitive dep, an @rpath dep reachable only through LC_RPATH,
and an ADDITIONAL_LIBS dep are all bundled, a dependency cycle terminates,
system libraries are skipped, the packaged tree has assets/ at the root beside
grpc-server with the dylibs in lib/, and both failure paths exit non-zero.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): make bundled: reachable from a model YAML

resolve_model_path() tested the bundled: prefix on `candidate`, which prefers
ModelFile and falls back to Model. LocalAI fills ModelFile by joining ModelPath
onto the configured model string (pkg/model/loader.go, LoadModelWithFile), and
only sets it from a managed artifact otherwise, so a model YAML saying
`model: bundled:silero_vad` arrives as ModelFile "/models/bundled:silero_vad"
and Model "bundled:silero_vad". The prefix therefore never matched through the
normal load path: it matched only for a hand-written LoadModel call that left
ModelFile empty, which is exactly how task 15 verified it, and every model YAML
using the form failed with "model path does not exist:
/models/bundled:silero_vad".

Both fields are now checked, Model first, so the zero-download VAD path the
package ships assets for is reachable the way it is documented. A caller that
puts the form in ModelFile still works, so task 15's verification stands.

Compiled clean; the runtime check could not run on this host, whose system
libprotobuf/libre2 have gone missing (the pre-existing grpc-server binary no
longer resolves its libraries either), so it wants a container run.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): advertise the backend and document its options

Registers audio-cpp as preference-only in /backends/known: the family lives in
GGUF metadata that an importer cannot read from a remote repo, and one repo
hosts thirty families, so there is no honest auto-detect signal. Modality is a
single string and the import form chips on a fixed key set, so it registers as
tts with the other modalities named in the description rather than under an
invented key the UI would bucket as "other".

Adds a features page covering the option namespacing, the routing table per
endpoint, the RPCs this backend declines and why, the bundled VAD path, the
separation stem behaviour, and the family gotchas (supertonic needs the orig
package; chatterbox advertises cloning and no plain tts; nemotron_asr defers
its whole decode to finalize so live transcription emits nothing until the
client half-closes, unlike higgs_audio_stt and voxtral_realtime). Every option
name and family capability in it was read off the pinned upstream checkout.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(audio-cpp): test resolve_model_path, and correct the family names

The bundled: fix in 842443cd7 shipped without a test, which is how the bug got
there: task 15 verified the form with a hand-written LoadModel that left
ModelFile empty, and that is the one shape the server never produces. Four cases
in streaming_driver_ctest, which already links loaded_model.cpp, pin the
PRODUCTION shapes instead. The first fails against the pre-fix source (returns
the joined /models/bundled:silero_vad); the other three are the branches the
bundled: lookup now runs in front of and must fall through for.

Three family names in the docs were the source directory rather than the
registered family, on pages whose whole argument is that these names cannot be
guessed: demucs is htdemucs (demucs/loader.cpp:22), roformer is
mel_band_roformer (roformer/assets.h:15), and moss is TWO families,
moss_tts_local and moss_tts_nano. The hyphenated ASR names are underscored to
match, here and in the compatibility table.

The supertonic dtype note claimed more than the evidence carries. The f16 abort
is a local observation, identical through TTS and TTSStream; upstream's
docs/gguf.md leaves the 16-bit column untested and records q8_0 as "No
(unsupported weight dtype)", which says unusable rather than fatal. Both are
still refused, because the allow list is what the family can run. Corrected in
family_gate.h, family_gate.cpp and the docs together, since the docs inherited
the wording from the code.

The importers tripwire says in the file that it is a tripwire: it exercises no
audio-cpp behaviour, and the registration assertion lives in backend_test.go.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: add audio.cpp models covering every served RPC

One representative model per RPC group of the audio-cpp backend, plus the two
bundled VAD models, which need no download at all because the assets ship
inside the backend package.

Every hash was computed with sha256sum on the downloaded file. Quantizations
come from upstream's tested-status table in docs/gguf.md rather than a default
of q8_0: supertonic ships the orig package (its q8_0 is recorded as an
unsupported weight dtype and its f16 aborts in ggml_concat), and nemotron_asr
and htdemucs ship f16 because their q8_0 builds are recorded with drift while
16-bit is a clean pass.

Diarization and separation use the diarization and audio_transform usecases,
not transcript: /v1/audio/diarization and /audio/transform filter the default
model on FLAG_DIARIZATION and FLAG_AUDIO_TRANSFORM respectively, so a
transcript flag would have hidden both models from their own endpoints. The
forced aligner sets parameters.language, which the transcription endpoint uses
as the fallback when no language form field is sent, because the family
requires both a transcript and a language.

All ten entries were run twice: once against the raw gRPC server, and once
installed with local-ai models install and called through the HTTP endpoint.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: correct the audio.cpp entries' licenses

Swept all ten entries against the real upstream named in audio.cpp's
tools/model_manager.py rather than against the audio.cpp repo's own license.
Three were wrong:

  supertonic   apache-2.0 -> openrail      weights come from
                                           mlx-community/supertonic-3-mlx, and
                                           both it and Supertone/supertonic are
                                           openrail
  citrinet     apache-2.0 -> other         pulled from NGC
                                           nvidia/nemo/stt_en_citrinet_256,
                                           governed by the NGC Terms of Use
  sortformer   other -> cc-by-nc-4.0       nvidia/diar_sortformer_4spk-v1 is
                                           CC BY-NC 4.0, and the gallery already
                                           uses that exact string, so there is no
                                           reason to obscure a non-commercial bar

The license field is one word, so citrinet and sortformer also gained a
sentence saying why they are restricted. The other seven were confirmed
correct against their sources.

Also drops an unverified claim from the nemotron description. It said the
model drives the realtime transcription session; that endpoint actually calls
TranscribeStream, and the live RPC reaches LocalAI only through
realtime_semantic_vad.go. Neither path was exercised here, so the description
now states only the two calls that were.

MarbleNet gains the NeMo upstream under urls: for parity with silero.

No sha256, quantization, usecase or model choice changed.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(audio-transform): bound sample_rate, keep same-named uploads apart

Four defects the whole-branch review found on the Go side, plus two comment
corrections.

sample_rate is a disk-exhaustion hazard. The branch added the `form:` tag that
makes the field bind for the first time, so the resample path went from dead to
live, and utils.AudioResample interpolates the int straight into ffmpeg's -ar
with no bound. Measured with ffmpeg 7: -ar 999999999 on a 0.01 s clip writes
20 MB and exits 0, which scales linearly to the reported 3.9 GB for one second,
into a GeneratedContentDir nothing sweeps, and convertStems repeats it once per
separation stem. Clamped to 8000..192000 in the handler, before the temp dir and
before the model is touched, and rejected with a 400 outside it.

The low end was reported as "a 0-byte file". It is not: -ar 1 writes a 78-byte
header with no audio behind it, whose declared data size still claims 70 bytes,
so go-audio parses it as a 35 SECOND file and a size check does not see it. The
guard therefore compares the declared data chunk against the bytes actually on
disk, and AudioResample now fails rather than returning a WAV carrying nothing.

Both parts of a transform request land in one temp dir, and the raw copy was
named only after the client's basename, so `-F audio=@mic/clip.wav
-F reference=@loopback/clip.wav` wrote "raw-clip.wav" twice. Since
AudioToWavPreservingShape hardlinks an already-PCM16 WAV rather than copying it,
the reference part's os.Create truncated the inode audio.wav pointed at: mic and
reference came out identical, which makes an echo canceller null everything and
return near-silence with a 200. The raw copy now carries the form field name.

audio-cpp had no BackendCapabilities entry, so VoiceCloningForModel returned nil
before it ever consulted the model's tts.voice_cloning override and every
`voice: "profile:<id>"` request was refused with a 400, on a backend that ships
audio-cpp-chatterbox whose family serves cloning and not plain TTS. Registered
with its RPCs, usecases and the reference-audio contract, and deliberately
without the 16 kHz mono fold, which its separation families cannot survive.

GetBackendCapability was exact-match only, so every pinned gallery variant read
as an unknown backend: vulkan-localvqe lost the 16 kHz mono fold that used to be
unconditional and started failing inside LocalVQE, and the usecase gate does not
stand in for it because BuildFilteredFirstAvailableDefaultModel returns early
once the client names a model. Lookup now falls back to the meta name by
stripping the gallery's hardware prefix and release-channel suffix, exact match
first so nothing can be shadowed. Same class as #10945.

Also corrected: the AudioTransformRequest comment claimed echo's binder falls
back to the field name, which it does not in either direction (bindData binds
ONLY tagged fields and `continue`s otherwise; `model` arrives from
setModelNameFromRequest's c.FormValue). And the stable_audio `src` heap
corruption caveat now lives on ElevenLabsSoundGenerationRequest, where the Go
developer who would add the field can see it, instead of only in C++.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(audio-cpp): refuse a task pin the RPC cannot serve, and stop empty frames holding the lane

The model's `task:` option is copied into the request shape by all nine
handlers, which is correct, but resolve_route then replaced the RPC's candidate
list with the pin WHOLESALE and never asked whether the pin was something that
RPC routes to. One pin therefore bled across all nine surfaces, and because the
family still supported the pinned task the result was a wrong 200 rather than an
error. Reproduced live: nemotron with task:asr made Vad return 200 with zero
segments after a full ASR decode, so 14 seconds of speech was reported as
silence, and Diarize did the same; silero_vad with task:vad made
AudioTranscription return 200 with empty text and four segments whose spans were
VAD segments, which combined with response_format in {text,srt,vtt,lrc} building
the body solely from Segments[].Text yields a well formed SRT of four timed
EMPTY cues. It also contradicted the documented contract, that a family which
cannot serve a request is refused rather than rerouted.

A pin is now checked against the RPC's admissible task set before it is adopted,
and the refusal names both the pin and the RPC. The set is derived from
task_candidates with every shape flag set rather than restated, so a task added
to an RPC's candidates cannot become inadmissible by omission. Every legitimate
pin survives, and the test asserts all fifteen of them alongside the eight
crossings that must not.

The live watchdog was defeated by empty frames. idle.touch() ran on ANY message,
before the has_audio and pcm.empty() filters, so a peer writing unset-oneof or
zero-length frames faster than the window held the lane indefinitely while
feeding the decoder nothing. There is one lane per model and one model per
process, so that is a single client denying the whole backend, which is what the
watchdog exists to prevent, and the thrown text already said "no audio frame
arrived". The touch moved below the filters, which are now a named predicate so
the distinction is testable rather than a call order nobody can see.

Three comments corrected against measurement rather than reasoning:

- CMakeLists claimed zero google::protobuf:: definitions remain in the
  executable. nm -C --defined-only reports 2515, and that is expected: they are
  generated code, sentencepiece::ModelProto's own _InternalParse among them. The
  claim that holds, and the one the ABI fix is actually about, is that no
  vendored protobuf RUNTIME is linked and ParseContext::ParseMessage is
  UNDEFINED in the executable, resolving to libprotobuf.so.
- refuse_cloning_without_a_clip's "cannot misfire" paragraph had its reasoning
  backwards. Routing picks VoiceCloning as the FALLBACK when there is no clip,
  which is the case being caught; chatterbox, which ships in the gallery,
  advertises clon and no tts at all, so every voice-less request lands there.
- audio_units read "2.1 min at 96 kHz" for index 11289602, which is 1.96 min.
  2.1 min is 96 kHz's OWN first failure at 12288002. Both were remeasured and
  the note is now a per-rate table.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* build(audio-cpp): exclude the upstream checkout from the C++ gate, harden the darwin walk

run-unit-tests.sh pruned */llama.cpp/* but not */audio.cpp/*. It is safe today
only by luck: upstream's 44 tests all put "test" at the FRONT of the filename
(17 test-*.cpp, 27 test_*.cpp, zero *_test.cpp), so the glob misses every one of
them, and nothing enforces that. This gate runs on every PR for every backend
and compiles each match as a standalone translation unit with nothing but
nlohmann/json on the include path, so the day upstream adds or renames one test
the gate goes red repo-wide on an Apache-2.0 file nobody here wrote.

audio-cpp-darwin.sh now logs the raw otool -L output and the parsed LC_RPATH
list unconditionally, before the walk. Both awk filters in that script assume a
column layout nobody working on this can observe, since it runs only on the CI
Mac, and a green first Darwin run proves nothing about the assumption: an awk
that silently matched nothing yields an empty dependency list, which reads
exactly like "no non-system dependencies" and packages happily. Both filters
otherwise feed process substitutions, so their input never reached the log.

It also lists every symlink in the package and fails on one that cannot resolve
inside the image. A dangling link does not fail anything else here, because
every assertion tests with -e, which follows links; it fails at dlopen on a
user's Mac. Links are NOT banned outright, which the review suggested but which
would break the libggml.dylib -> libggml.0.dylib chain the `cp -a` above exists
to preserve. What is banned is a link that resolves on the build host and will
not resolve in the image: a broken one, or an absolute one pointing outside the
package.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* style(audio-cpp): drop em dashes from the audio-cpp capability entry

Follow-up to a84b3c4b9, no behaviour change.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(config): key the voice-cloning model rule on the resolved backend

Making GetBackendCapability strip the gallery hardware prefix and release
channel fixed pinned variants of /audio/transform, but VoiceCloningForModel
kept keying its per-backend switch on the caller's spelling. A pinned name
therefore resolved the capability by stripping and then missed every case in
the switch, falling through to the permissive default: cuda12-vibevoice-cpp
advertised voice cloning for the realtime 0.5B model, metal-coqui for
tacotron2, cuda12-crispasr for a pure ASR model, cpu-qwen3-tts-cpp for
CustomVoice. Each of those is a model that cannot clone, so /v1/audio/speech
accepted a profile: voice it had to fail on inside the backend rather than
rejecting it with a 400, and the UI advertised the capability too.

resolveBackendCapability now returns the key the entry was found under, and
callers that branch on backend identity use that key instead of the name they
were handed. The exact-match-first order is unchanged, so a backend genuinely
registered under a variant-looking name still keys on its own name.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* gallery(audio-cpp): declare audio_transform on the chatterbox entry

Chatterbox advertises VoiceCloning AND VoiceConversion
(src/models/chatterbox), and the entry's own description already said so, but
known_usecases listed only tts. /audio/transform selects its default model by
FLAG_AUDIO_TRANSFORM, so voice conversion was reachable only by naming the
model explicitly and was invisible to every usecase-driven surface. It is the
one audio.cpp task with a shipped gallery model and no way to find it.

Verified against the real model rather than inferred from the capability list:
AudioTransform with chatterbox-q8_0, speech as audio_path and a speaker clip
as reference_path, returns a 5.08 s 24 kHz mono WAV at -25.5 dB mean and zero
stems, which is the single-output shape voice conversion should have.

The description now says which endpoint reaches that half and warns that
installing this next to a source-separation model gives /audio/transform two
candidates, so the model should be named rather than defaulted.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* gallery(audio-cpp): add voice-design and singing-voice-conversion entries

Two of the three audio.cpp task kinds that had no gallery model now have one.
Both were driven end to end against the real weights through the backend
before being written, not inferred from the capability tables.

audio-cpp-irodori-voicedesign covers vdes. TTS carrying `instructions` routes
to the vdes task, so the voice is described in words rather than supplied as a
clip. Verified: "a calm elderly woman speaking slowly with a warm, gentle
tone" over an 8.76 s 48 kHz mono render at -16.8 dB mean, and a closed-loop
citrinet pass recovers the sentence with the accent drift expected from a
Japanese-first model read by an English recogniser.

audio-cpp-seedvc-singing covers svc, and pins task:svc because nothing else
can reach it. seed_vc advertises svc and ordinary voice conversion, no request
signal means "this input is singing", and auto-routing resolves the tie to
voice conversion every time. Verified with the pin: 5.04 s 44.1 kHz output
whose closed-loop citrinet transcription is exact.

s2s deliberately has no entry, and the reason is not effort. miocodec is the
only upstream family whose speech-to-speech route needs no text, and it
returned audio with correct duration and level but no recoverable speech in
four independent attempts: the stale build, v2 q8_0, v2 orig (the variant
upstream records as a clean Pass), both tasks, and matched 44.1 kHz inputs on
both sides. vevo2's route refuses with "Vevo2 text/prosody route requires
text_input or target_text", and session.cpp:897 fills target_text only from
request.text_input, which AudioTransform has no field to carry. The same
vevo2 weights convert voice correctly through the default route with an exact
ASR round trip, so the model and the plumbing are both healthy; it is the s2s
route specifically that this RPC cannot express.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* backend(audio-cpp): carry transform text through params, add the s2s entry

AudioTransform is audio-in / audio-out and its proto message has no text
field, but not every task it routes to is audio-only. vevo2's speech-to-speech
route is a text and prosody route: session.cpp:897 fills refs.target_text from
request.text_input and nowhere else, and the run refuses without one with
"Vevo2 text/prosody route requires text_input or target_text". The params map
is the only channel this RPC has that reaches the engine, so the text travels
through it and apply_transform_text_input unpacks it after the params have
been copied into task.options.

Before this, s2s was not awkward to reach through /audio/transform, it was
unreachable, and it was the last audio.cpp task kind with a real model and no
way to get to it.

target_text is canonical and text is its alias, the order vevo2's own option
table declares them in, so a request setting both gets the canonical one
rather than whichever the map happened to store first. An empty value falls
through to the next candidate instead of ending the search. language rides
along only when a text was found: on its own it conditions nothing, and
manufacturing a text_input for it would route a plain separation request
carrying a language hint through the text path. The keys are left in
task.options rather than erased, because vevo2's loader advertises target_text
as a request option and a family reading it there keeps working.

Nine tests, all confirmed failing on behaviour against a stub that returned
false before the implementation was written. Verified end to end afterwards:
vevo2-q8_0 with task:s2s and params[text] returns a 5.12 s 24 kHz output whose
closed-loop citrinet transcription is exact, and htdemucs separation with no
text param still returns its four stems, with and without params[stem].

audio-cpp-vevo2-speech-to-speech ships that route. Every audio.cpp task kind
with a loadable family now has a gallery entry; spk remains the only gap and
has no family upstream at all.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* docs(audio-cpp): document params[text] and the pinned transform tasks

The text channel and the two task pins are both invisible from the endpoint
contract alone: nothing in the AudioTransform form tells a reader that a
speech-to-speech model needs the line it is resynthesising, and nothing says
that asking for singing voice conversion without task:svc silently gets plain
voice conversion instead. Both are the kind of thing a user only discovers
from a refusal or, worse, from output that looks right and is not.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* fix(utils): annotate the two G304 sites this branch introduced

gosec flags os.Open on a variable path, and both new call sites in ffmpeg.go
are its alerts on this PR. Neither is reachable by an outside caller: isPCM16Wav
opens the exact path it is about to hand ffmpeg as input, which in the upload
path is a server-created temp file named from path.Base of the client name so
no traversal survives, and wavAudioBytes opens AudioResample's own dst, a name
this package derives from src and has just had ffmpeg write.

Annotated in the repo's existing style rather than restructured, with the
reason spelled out, because a bare suppression is worth nothing to the next
reader. The three other G304 sites in this file, in passthroughWAV and
isTargetWav, predate the branch and are left untouched.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* fix(audio-cpp): build arm64 with gcc-14 for the armv9.2 SME variants

The arm64 CPU image failed to build:

  cc1: error: invalid feature modifier 'sme' in
       '-march=armv9.2-a+dotprod+fp16+sve+i8mm+sve2+sme'

ggml's CPU_ALL_VARIANTS table includes armv9.2 variants compiled with +sme, and
Ubuntu Noble's default gcc-13 rejects that feature modifier. Every entry in the
table has to compile even though a host only ever dlopens the one its own CPU
supports, so a single unbuildable variant fails the whole image. gcc-14 accepts
it, which is exactly the fix llama-cpp already carries in
.docker/llama-cpp-compile.sh; this is the same problem reached by a different
Dockerfile.

Applied to every arm64 BUILD_TYPE rather than to the CPU one alone, and that
differs from llama-cpp on purpose. llama-cpp needs it only for its pure-CPU
image because its GPU builds run llama-cpp-fallback, which builds no variant
table. This backend's Makefile turns ENGINE_ENABLE_CPU_ALL_VARIANTS on for
every non-Darwin build, GPU included, so an arm64 GPU image would hit the
identical error. The matrix has no arm64 GPU entry today, which is precisely
why gating on an empty BUILD_TYPE would leave the trap armed for whoever adds
the first one.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

backend/cpp/audio-cpp/.gitignore

@@ -0,0 +1,8 @@
+audio.cpp/
+build/
+package/
+grpc-server
+backend.pb.cc
+backend.pb.h
+backend.grpc.pb.cc
+backend.grpc.pb.h

backend/cpp/audio-cpp/CMakeLists.txt

@@ -0,0 +1,331 @@
+cmake_minimum_required(VERSION 3.20)
+project(audio-cpp-grpc-server LANGUAGES C CXX)
+
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(TARGET grpc-server)
+
+set(AUDIO_CPP_DIR "${CMAKE_CURRENT_SOURCE_DIR}/audio.cpp"
+    CACHE PATH "Path to the pinned audio.cpp checkout")
+option(AUDIO_CPP_GRPC_BUILD_TESTS "Build engine-linked ctest binaries" OFF)
+
+if(NOT EXISTS "${AUDIO_CPP_DIR}/CMakeLists.txt")
+    message(FATAL_ERROR
+        "AUDIO_CPP_DIR does not contain an audio.cpp checkout: ${AUDIO_CPP_DIR}. "
+        "Run 'make audio.cpp' first.")
+endif()
+
+if(APPLE)
+    # Homebrew installs protobuf/grpc under a non-default prefix.
+    if(CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "arm64")
+        set(HOMEBREW_DEFAULT_PREFIX "/opt/homebrew")
+    else()
+        set(HOMEBREW_DEFAULT_PREFIX "/usr/local")
+    endif()
+    link_directories("${HOMEBREW_DEFAULT_PREFIX}/lib")
+    include_directories("${HOMEBREW_DEFAULT_PREFIX}/include")
+endif()
+
+find_package(Threads REQUIRED)
+find_package(Protobuf CONFIG QUIET)
+if(NOT Protobuf_FOUND)
+    find_package(Protobuf REQUIRED)
+endif()
+find_package(gRPC CONFIG QUIET)
+if(NOT gRPC_FOUND)
+    # Reached only on distros whose grpc++ packaging ships no CMake config.
+    # Ubuntu's libgrpc-dev does ship one, so this is dead code on LocalAI's own
+    # build distro. Kept for the distros that do not.
+    find_library(GRPCPP_LIB grpc++ REQUIRED)
+    find_library(GRPCPP_REFLECTION_LIB grpc++_reflection REQUIRED)
+    add_library(gRPC::grpc++ INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++ PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_LIB}")
+    add_library(gRPC::grpc++_reflection INTERFACE IMPORTED)
+    set_target_properties(gRPC::grpc++_reflection PROPERTIES
+        INTERFACE_LINK_LIBRARIES "${GRPCPP_REFLECTION_LIB}")
+endif()
+
+find_program(_PROTOC NAMES protoc REQUIRED)
+find_program(_GRPC_CPP_PLUGIN NAMES grpc_cpp_plugin REQUIRED)
+
+get_filename_component(HW_PROTO "${CMAKE_CURRENT_SOURCE_DIR}/../../backend.proto" ABSOLUTE)
+get_filename_component(HW_PROTO_PATH "${HW_PROTO}" PATH)
+
+set(HW_PROTO_SRCS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.cc")
+set(HW_PROTO_HDRS "${CMAKE_CURRENT_BINARY_DIR}/backend.pb.h")
+set(HW_GRPC_SRCS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.cc")
+set(HW_GRPC_HDRS  "${CMAKE_CURRENT_BINARY_DIR}/backend.grpc.pb.h")
+
+add_custom_command(
+    OUTPUT "${HW_PROTO_SRCS}" "${HW_PROTO_HDRS}" "${HW_GRPC_SRCS}" "${HW_GRPC_HDRS}"
+    COMMAND ${_PROTOC}
+    ARGS --grpc_out "${CMAKE_CURRENT_BINARY_DIR}"
+         --cpp_out  "${CMAKE_CURRENT_BINARY_DIR}"
+         -I "${HW_PROTO_PATH}"
+         --plugin=protoc-gen-grpc="${_GRPC_CPP_PLUGIN}"
+         "${HW_PROTO}"
+    DEPENDS "${HW_PROTO}")
+
+add_library(hw_grpc_proto STATIC
+    ${HW_GRPC_SRCS} ${HW_GRPC_HDRS}
+    ${HW_PROTO_SRCS} ${HW_PROTO_HDRS})
+target_include_directories(hw_grpc_proto PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
+# Required on macOS: without these the Homebrew protobuf/grpc include dirs never
+# reach this target and google/protobuf/runtime_version.h is not found.
+target_link_libraries(hw_grpc_proto PUBLIC protobuf::libprotobuf gRPC::grpc++)
+
+# TWO PROTOBUF RUNTIMES IN ONE BINARY, AND THE ONE THAT WON WAS THE WRONG ONE.
+#
+# engine_runtime links sentencepiece, whose default SPM_PROTOBUF_PROVIDER
+# ("internal") builds the protobuf-lite 3.14.0 sources vendored under
+# external/sentencepiece/third_party/protobuf-lite. Our generated backend.pb.cc
+# is compiled against the toolchain's protobuf 3.21.12 headers and links
+# libprotobuf.so 3.21.12. Both used to end up in the executable: 476
+# google::protobuf:: symbols from that archive, 278 of them also defined by
+# libprotobuf.so.
+#
+# The binding is decided at STATIC LINK time. Once ld pulls a sentencepiece
+# member in for sentencepiece's own code, that member's protobuf definitions are
+# in the executable and references from libhw_grpc_proto.a bind to them. Do NOT
+# reach for -Wl,--exclude-libs: it flips those symbols to LOCAL in .dynsym and
+# the breakage is unchanged, because no visibility flag revisits a static
+# binding already made.
+#
+# What broke, measured rather than assumed:
+# google::protobuf::internal::ParseContext::ParseMessage(MessageLite*, const char*)
+# is what every generated _InternalParse calls for a SUBMESSAGE field and for
+# nothing else. Bound to the 3.14 definition it fails, so a flat message parsed
+# and every nested one did not: a TranscriptResult carrying segments serialized
+# to correct bytes that the same process could not read back, and
+# TranscriptLiveRequest, a oneof of submessages, could not have been parsed at
+# all. 3.21 generated code was also running 3.14 arena, ArenaStringPtr and
+# ExtensionSet code, which is an ABI mismatch rather than a missing feature, so
+# "not observed to bite yet" was never a reason to leave it.
+#
+# "package" makes sentencepiece use the protobuf found above, which is the one
+# the generated code was built against. It must be set before add_subdirectory,
+# since that is when sentencepiece's own cache entry is created.
+#
+# WHAT THAT BUYS IS ONE PROTOBUF RUNTIME, not an executable free of protobuf
+# symbols, and the difference matters to whoever checks this next. Measured with
+# nm -C --defined-only on the linked grpc-server, 2515 google::protobuf::
+# symbols are still DEFINED in it, and that is what should be there: they are
+# generated code, sentencepiece::ModelProto's own _InternalParse and
+# CheckTypeAndMergeFrom among them, which name protobuf types in their
+# signatures and are compiled into every user of a .proto. Expecting zero would
+# send a reader looking for a regression that is not one.
+#
+# The claim that decides whether the ABI mismatch above is gone is the RUNTIME
+# one, and it holds: google::protobuf::internal::ParseContext::ParseMessage is
+# UNDEFINED in the executable, so every generated _InternalParse resolves it to
+# libprotobuf.so at load instead of to a vendored 3.14 copy. No vendored
+# protobuf-lite archive is pulled in at all, and citrinet_asr, which parses a
+# SentencePiece ModelProto at load, tokenizes correctly as a result.
+set(SPM_PROTOBUF_PROVIDER "package" CACHE STRING
+    "Make sentencepiece use the found protobuf, not its vendored 3.14 copy" FORCE)
+
+# Upstream's global add_compile_options(-Wall -Wextra -Wpedantic -pedantic-errors)
+# is a directory property of the subdirectory and does not reach our targets.
+#
+# EXCLUDE_FROM_ALL is load-bearing, do not drop it: upstream's default target set
+# includes its CLI, server, converter and test binaries, none of which we ship.
+# Without it every build would compile all of them. The targets we do name in
+# target_link_libraries below are still built on demand, so nothing is lost.
+add_subdirectory("${AUDIO_CPP_DIR}" "${CMAKE_CURRENT_BINARY_DIR}/audio-cpp" EXCLUDE_FROM_ALL)
+
+add_executable(${TARGET}
+    grpc-server.cpp
+    model_options.cpp
+    capability_routing.cpp
+    family_gate.cpp
+    loaded_model.cpp
+    audio_io.cpp
+    audio_units.cpp
+    transcript_assembly.cpp
+    result_map.cpp
+    stem_selection.cpp
+    generation_request.cpp
+    stream_delta.cpp
+    wav_header.cpp
+    inference_lane.cpp
+    live_watchdog.cpp
+)
+
+# Two files carry a switch over an enum with no `default:` label, deliberately,
+# so that -Wswitch reports an enumerator nobody handled. -Wswitch is only a
+# warning by default, and a warning in a 600-file build log is a warning nobody
+# reads, so it is promoted to an error on exactly these two translation units.
+# Not project-wide: upstream's own sources are not held to this, and they are
+# where the churn is.
+#
+# loaded_model.cpp mirrors engine::runtime::VoiceTaskKind onto its own Task enum.
+# Its static_asserts catch an insertion or a reorder, but an enumerator APPENDED
+# after the last one shifts no value, so no assertion can see it. What does see
+# it is from_engine_task's switch over the engine enum. This is the difference
+# between a build failure and a backend that silently runs the wrong task.
+#
+# capability_routing.cpp's unsupported_surface() switches UnsupportedRpc onto the
+# row of unsupported_surfaces() that explains it. Left as a warning, a sixth
+# enumerator added without a row BUILDS AND SHIPS, and its trailing
+# `return surfaces[0];` then answers the new RPC with AudioEncode's codec reason:
+# a confident, specific and false statement about audio.cpp, on the wire, on the
+# one code path whose entire job is to be truthful about what this backend
+# cannot do. Verified rather than assumed: adding a sixth enumerator and building
+# the shipping target produced exit 0, a binary, and one warning. A compile-time
+# check is the better trade than the runtime fallback it replaced only if it is
+# fatal, so here it is fatal.
+if(NOT MSVC)
+    set_source_files_properties(loaded_model.cpp capability_routing.cpp
+        PROPERTIES COMPILE_OPTIONS "-Werror=switch")
+endif()
+
+target_include_directories(${TARGET} PRIVATE
+    "${AUDIO_CPP_DIR}/include"
+    "${CMAKE_CURRENT_SOURCE_DIR}")
+
+# The shipping binary is held to the same bar as the tests below. Upstream's own
+# add_compile_options is a property of its directory and never reached this
+# target, so until now "the build was clean" meant only that nothing was being
+# checked.
+if(NOT MSVC)
+    target_compile_options(${TARGET} PRIVATE -Wall -Wextra -Wpedantic)
+endif()
+
+target_link_libraries(${TARGET} PRIVATE
+    hw_grpc_proto
+    engine_runtime
+    ggml
+    gRPC::grpc++
+    gRPC::grpc++_reflection
+    protobuf::libprotobuf
+    Threads::Threads)
+
+# ENGINE_ENABLE_CPU_ALL_VARIANTS builds ggml backends as shared objects that sit
+# next to the binary in the package, so the binary must search its own directory.
+# BUILD_WITH_INSTALL_RPATH keeps the build-tree binary at exactly "$ORIGIN".
+# Upstream sets CMAKE_BUILD_WITH_INSTALL_RPATH in its own directory scope, which
+# does not reach ours, so without this CMake also appends its build-tree library
+# directory. That absolute build-host path would survive into the copied binary
+# and let a package.sh that forgot to bundle libggml*.so still pass on the build
+# machine while failing everywhere else.
+set_target_properties(${TARGET} PROPERTIES
+    BUILD_RPATH "$ORIGIN"
+    INSTALL_RPATH "$ORIGIN"
+    BUILD_WITH_INSTALL_RPATH TRUE)
+
+if(AUDIO_CPP_GRPC_BUILD_TESTS)
+    enable_testing()
+
+    # These are the units whose tests CANNOT run under
+    # backend/cpp/run-unit-tests.sh, because that script compiles each
+    # *_test.cpp standalone with no protobuf and no audio.cpp include path.
+    # They are named *_ctest.cpp so the script's glob does not pick them up and
+    # fail every backend's suite; everything that can be stdlib-only still is,
+    # and still lives in a *_test.cpp beside its unit.
+    add_executable(result_map_ctest
+        result_map_ctest.cpp
+        result_map.cpp
+        transcript_assembly.cpp
+        audio_units.cpp)
+    target_include_directories(result_map_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h. Every other target
+        # here inherits that directory from the ggml target it links; this one
+        # links no ggml, so it has to name it.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    # No engine_runtime: result_map touches only the plain structs in
+    # engine/framework/runtime/session.h, so the header is all it needs.
+    target_link_libraries(result_map_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(result_map_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME result_map COMMAND result_map_ctest)
+
+    # Same shape as result_map_ctest: generation_request touches only the plain
+    # structs in engine/framework/runtime/session.h plus the generated protobuf
+    # messages, so the headers are all it needs and no engine_runtime is linked.
+    add_executable(generation_request_ctest
+        generation_request_ctest.cpp
+        generation_request.cpp)
+    target_include_directories(generation_request_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}"
+        # session.h reaches ggml.h through core/backend.h, and this target links
+        # no ggml, so it has to name the include directory itself.
+        "${AUDIO_CPP_DIR}/external/ggml/include")
+    target_link_libraries(generation_request_ctest PRIVATE
+        hw_grpc_proto
+        protobuf::libprotobuf
+        Threads::Threads)
+    target_compile_options(generation_request_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME generation_request COMMAND generation_request_ctest)
+
+    add_executable(audio_io_ctest
+        audio_io_ctest.cpp
+        audio_io.cpp)
+    target_include_directories(audio_io_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(audio_io_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(audio_io_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    set_target_properties(audio_io_ctest PROPERTIES
+        BUILD_RPATH "$ORIGIN"
+        INSTALL_RPATH "$ORIGIN"
+        BUILD_WITH_INSTALL_RPATH TRUE)
+    add_test(NAME audio_io COMMAND audio_io_ctest)
+
+    # The streaming drivers live in loaded_model.cpp, which links the engine, so
+    # this cannot be a standalone *_test.cpp. It builds no model and reads no
+    # file: LoadedModel::Session is a plain struct holding a pointer to an
+    # engine interface, so the drivers are exercised against fake sessions.
+    add_executable(streaming_driver_ctest
+        streaming_driver_ctest.cpp
+        loaded_model.cpp
+        capability_routing.cpp
+        family_gate.cpp
+        model_options.cpp
+        inference_lane.cpp)
+    target_include_directories(streaming_driver_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(streaming_driver_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(streaming_driver_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    # No "$ORIGIN" rpath override here, unlike the shipping target and unlike
+    # audio_io_ctest. loaded_model.cpp reaches make_default_registry, so this
+    # binary genuinely links libggml, and CMake's own build-tree rpath is what
+    # finds it: the ggml shared objects land in ${CMAKE_CURRENT_BINARY_DIR}/bin
+    # while the test binary sits one directory up. A build-host absolute path in
+    # a test binary is harmless, since package.sh ships only grpc-server, and
+    # forcing "$ORIGIN" here means ctest cannot start the binary at all.
+    add_test(NAME streaming_driver COMMAND streaming_driver_ctest)
+
+    # Asserts that the upstream ABSENCES capability_routing.cpp's refusal
+    # messages rest on are still absences, by querying make_default_registry()
+    # rather than by re-reading upstream. This is what makes an AUDIO_CPP_VERSION
+    # bump that adds a codec task kind, an spk family or a streaming converter
+    # fail the build instead of leaving a false statement on the wire.
+    #
+    # It links engine_runtime purely to run that query, which is why it lives
+    # here rather than with the standalone *_test.cpp files, and it needs no
+    # "$ORIGIN" rpath override for the same reason streaming_driver_ctest does
+    # not: see the note above.
+    add_executable(upstream_absence_ctest upstream_absence_ctest.cpp)
+    target_include_directories(upstream_absence_ctest PRIVATE
+        "${AUDIO_CPP_DIR}/include"
+        "${CMAKE_CURRENT_SOURCE_DIR}")
+    target_link_libraries(upstream_absence_ctest PRIVATE
+        engine_runtime
+        ggml
+        Threads::Threads)
+    target_compile_options(upstream_absence_ctest PRIVATE -Wall -Wextra -Wpedantic)
+    add_test(NAME upstream_absence COMMAND upstream_absence_ctest)
+endif()

backend/cpp/audio-cpp/Makefile

@@ -0,0 +1,172 @@
+# audio.cpp backend Makefile.
+#
+# Upstream pin lives below in the AUDIO_CPP_VERSION variable, so
+# .github/bump_deps.sh can find and update it, matching the llama-cpp / ds4
+# convention. That script seds every line matching the variable name followed by
+# an assignment, so this comment deliberately spells the name on its own: a
+# comment repeating the full assignment token gets rewritten and mangled by the
+# first auto-bump (backend/cpp/ds4/Makefile shows the damage). The clone
+# recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
+# rebuild and so the bump bot can see the pin.
+
+AUDIO_CPP_VERSION?=e800d435d130dc776baf6f3e6129bb62b1495c89
+AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
+
+CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+BUILD_DIR := build
+
+BUILD_TYPE ?=
+NATIVE ?= false
+JOBS ?= $(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
+
+UNAME_S := $(shell uname -s)
+
+# AUDIOCPP_DEPLOYMENT_BUILD compiles the model_specs/*.json catalog into
+# engine_runtime, so the shipped package needs no model_specs directory and a
+# safetensors model tree still resolves its family spec.
+CMAKE_ARGS ?= -DCMAKE_BUILD_TYPE=Release -DAUDIOCPP_DEPLOYMENT_BUILD=ON
+
+# CMAKE_CUDA_ARCHITECTURES must be set explicitly for a cublas build, and this
+# is not a tuning knob: upstream's CMakeLists sets CUDA_ARCHITECTURES to
+# `native` on the engine_runtime target whenever the root-scope variable is
+# unset (audio.cpp/CMakeLists.txt, the `if (CMAKE_CUDA_ARCHITECTURES)` branch
+# next to the istft/torch_random .cu sources), and docs/build/linux.md says so
+# outright: "Leave CMAKE_CUDA_ARCHITECTURES unset to build for the GPUs present
+# at build time (native)". No CI runner has a GPU, so `native` has nothing to
+# enumerate. ggml's own default (external/ggml/src/ggml-cuda/CMakeLists.txt)
+# does not rescue this: it list(APPEND)s in the ggml subdirectory scope, which
+# never reaches the root scope where the engine_runtime property is decided.
+#
+# The values below are ggml's list for the matching toolkit, copied rather than
+# invented, so the two targets compile for exactly the same set:
+#   - CUDA 13 drops the Maxwell/Pascal/Volta virtual archs (50/61/70).
+#   - 121a-real needs CUDA >= 12.9, so the CUDA 12 list (built against 12.8)
+#     stops at 120a-real.
+#   - `a`-suffixed archs are used rather than ggml's rejected 120f-virtual: the
+#     `f` suffix needs CMake >= 3.31.8, and Ubuntu Noble ships 3.28.3. The
+#     3.28 validator (Modules/Internal/CMakeCUDAArchitecturesValidate.cmake)
+#     accepts `[0-9]+a?(-real|-virtual)?`.
+#
+# Setting it here also pins ggml's copy, since its default is guarded by
+# `if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES)`. CUDA_MAJOR_VERSION is the CI
+# build-arg, forwarded by Dockerfile.audio-cpp.
+#
+# An EMPTY major maps to `native`, NOT to the CUDA 12 list. Only CI declares a
+# major; a developer running `BUILD_TYPE=cublas make` locally declares none, and
+# the CUDA 12 list contains 120a-real, which needs nvcc >= 12.8. Falling through
+# to it turned every local build on a CUDA 12.0-12.7 host into a compile error,
+# where upstream's documented behaviour ("Leave CMAKE_CUDA_ARCHITECTURES unset
+# to build for the GPUs present at build time") worked. `native` restores that.
+# It does require a GPU to enumerate, so the escape hatch for a GPU-less local
+# cross-build is to set CUDA_ARCHITECTURES on the command line, which the ?=
+# assignments below leave untouched.
+CUDA_MAJOR_VERSION ?=
+ifeq ($(CUDA_MAJOR_VERSION),13)
+    CUDA_ARCHITECTURES ?= 75-virtual;80-virtual;86-real;89-real;120a-real;121a-real
+else ifeq ($(CUDA_MAJOR_VERSION),12)
+    CUDA_ARCHITECTURES ?= 50-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-real;89-real;120a-real
+else ifeq ($(CUDA_MAJOR_VERSION),)
+    CUDA_ARCHITECTURES ?= native
+else ifeq ($(BUILD_TYPE),cublas)
+    # Gated on cublas because the variable means nothing to any other build, so a
+    # stray CUDA_MAJOR_VERSION in the environment must not break `make clean` or
+    # a CPU build. It does still error for `BUILD_TYPE=cublas make clean`, which
+    # is the right trade: that invocation is asking about a CUDA build tree.
+    $(error CUDA_MAJOR_VERSION=$(CUDA_MAJOR_VERSION) has no architecture list here (12 and 13 do). Leave it empty for a native build, or pass CUDA_ARCHITECTURES explicitly.)
+endif
+
+ifeq ($(BUILD_TYPE),cublas)
+    CMAKE_ARGS += -DENGINE_ENABLE_CUDA=ON "-DCMAKE_CUDA_ARCHITECTURES=$(CUDA_ARCHITECTURES)"
+else ifeq ($(BUILD_TYPE),vulkan)
+    CMAKE_ARGS += -DENGINE_ENABLE_VULKAN=ON
+else ifeq ($(UNAME_S),Darwin)
+    # Metal. ggml embeds the shader library by default (GGML_METAL_EMBED_LIBRARY
+    # defaults to GGML_METAL), so the package needs no .metallib beside the
+    # binary. Darwin builds go through scripts/build/audio-cpp-darwin.sh.
+    CMAKE_ARGS += -DENGINE_ENABLE_METAL=ON
+    # AppleClang ships no OpenMP runtime and Homebrew's libomp is keg-only, so
+    # neither libomp.dylib nor omp.h is symlinked into /opt/homebrew and CMake's
+    # FindOpenMP cannot find them on its own (the workflow's `brew link libomp`
+    # is a no-op for a keg-only formula, and its failure is swallowed).
+    # audio.cpp calls find_package(OpenMP REQUIRED COMPONENTS CXX) whenever
+    # ENGINE_ENABLE_OPENMP is ON, so with no hint the macOS build dies at
+    # configure time before compiling anything. OpenMP_ROOT is honoured by the
+    # find_library/find_path calls inside FindOpenMP under CMP0074, which is NEW
+    # here because audio.cpp requires CMake 3.20.
+    #
+    # If the keg is absent, turn OpenMP off rather than fail: the tree's only
+    # <omp.h> include is guarded by #ifdef _OPENMP and a #pragma omp without
+    # -fopenmp is simply ignored, so an OpenMP-less build is CORRECT. It is not
+    # cheap, though: 108 `#pragma omp` directives across ~30 files (roformer,
+    # demucs, chatterbox, moss, supertonic, seed_vc, framework/audio/dsp) are
+    # compiled out, and clang says nothing about an ignored omp pragma unless
+    # -Wsource-uses-openmp is on. A green package that is quietly single-threaded
+    # in every host DSP loop gets blamed on Metal, not on packaging, so the
+    # fallback announces itself.
+    ifeq ($(origin LIBOMP_PREFIX),undefined)
+        LIBOMP_PREFIX := $(shell brew --prefix libomp 2>/dev/null)
+    endif
+    # Nested ifneq rather than $(and ...): $(and) needs GNU make 3.81, and while
+    # that is what Apple ships, an older make expands it to empty and would take
+    # the OpenMP-OFF branch with no way to tell that from a genuinely missing
+    # keg. Two plain conditionals cannot fail that way.
+    LIBOMP_USABLE :=
+    ifneq ($(wildcard $(LIBOMP_PREFIX)/lib/libomp.dylib),)
+        ifneq ($(wildcard $(LIBOMP_PREFIX)/include/omp.h),)
+            LIBOMP_USABLE := yes
+        endif
+    endif
+    ifeq ($(LIBOMP_USABLE),yes)
+        CMAKE_ARGS += "-DOpenMP_ROOT=$(LIBOMP_PREFIX)"
+    else
+        $(warning audio-cpp: libomp not found at '$(LIBOMP_PREFIX)'; building without OpenMP (single-threaded host DSP). Install it with `brew install libomp`, or set LIBOMP_PREFIX.)
+        CMAKE_ARGS += -DENGINE_ENABLE_OPENMP=OFF
+    endif
+else
+    # Portable Linux CPU. Upstream wires this to GGML_BACKEND_DL +
+    # GGML_CPU_ALL_VARIANTS + $ORIGIN rpath, so one build serves every CPU
+    # tier instead of an AVX-tier image fan-out.
+    CMAKE_ARGS += -DENGINE_ENABLE_CPU_ALL_VARIANTS=ON
+endif
+
+ifneq ($(NATIVE),true)
+    CMAKE_ARGS += -DENGINE_ENABLE_NATIVE_CPU=OFF
+endif
+
+.PHONY: all grpc-server package test test-engine clean purge
+all: grpc-server
+
+# Clone the upstream source at the pinned commit. The directory is the target
+# so make only re-clones when it is missing. After bumping AUDIO_CPP_VERSION,
+# run 'make purge && make' to refetch.
+audio.cpp:
+	mkdir -p audio.cpp
+	cd audio.cpp && \
+	git init -q && \
+	git remote add origin $(AUDIO_CPP_REPO) && \
+	git fetch --depth 1 origin $(AUDIO_CPP_VERSION) && \
+	git checkout FETCH_HEAD
+
+grpc-server: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS)
+	cp $(BUILD_DIR)/grpc-server grpc-server
+
+package: grpc-server
+	bash package.sh
+
+test:
+	@echo "audio-cpp: standalone unit tests run from the repo root via 'make test-backend-cpp'"
+
+# Engine-linked tests. Needs the upstream checkout and a full engine build.
+test-engine: audio.cpp
+	mkdir -p $(BUILD_DIR)
+	cd $(BUILD_DIR) && cmake $(CMAKE_ARGS) -DAUDIO_CPP_GRPC_BUILD_TESTS=ON $(CURRENT_MAKEFILE_DIR) && \
+	cmake --build . --config Release -j $(JOBS) && ctest --output-on-failure --no-tests=error
+
+clean:
+	rm -rf $(BUILD_DIR) grpc-server package
+
+purge: clean
+	rm -rf audio.cpp

backend/cpp/audio-cpp/audio_io.cpp

@@ -0,0 +1,124 @@
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include "engine/framework/audio/conversion.h"
+#include "engine/framework/audio/wav_reader.h"
+#include "engine/framework/audio/wav_writer.h"
+
+#include <filesystem>
+#include <string>
+#include <utility>
+
+namespace audiocpp_backend {
+
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no input audio path was supplied");
+    }
+    std::error_code ec;
+    const bool present = std::filesystem::exists(std::filesystem::path(path), ec);
+    if (ec) {
+        // exists() returning false with ec set does NOT mean the file is
+        // absent, it means the question could not be answered: most often a
+        // parent directory is not searchable. Reporting that as "does not
+        // exist" sends the operator after the file when the fault is the
+        // permissions on the directory above it.
+        throw ConfigError("audio-cpp: cannot stat input audio " + path + ": " +
+                          ec.message());
+    }
+    if (!present) {
+        throw ConfigError("audio-cpp: input audio does not exist: " + path);
+    }
+    engine::audio::WavData wav;
+    try {
+        wav = engine::audio::read_wav_f32(std::filesystem::path(path));
+    } catch (const std::exception &err) {
+        throw ConfigError("audio-cpp: cannot read " + path +
+                          " as WAV: " + err.what());
+    }
+    if (wav.sample_rate <= 0) {
+        throw ConfigError("audio-cpp: " + path +
+                          " declares a non-positive sample rate; every "
+                          "timestamp derived from it would be zero");
+    }
+    // AudioBuffer's own default is 1, and a reader that reports 0 channels
+    // still gave us an interleaving of one. Normalised before the conversion
+    // below rather than after, because mixdown_interleaved_to_mono_average
+    // throws on a non-positive channel count.
+    if (wav.channels <= 0) {
+        wav.channels = 1;
+    }
+
+    engine::runtime::AudioBuffer buffer;
+    if (target_sample_rate <= 0) {
+        buffer.sample_rate = wav.sample_rate;
+        buffer.channels = wav.channels;
+        buffer.samples = std::move(wav.samples);
+        return buffer;
+    }
+
+    buffer.sample_rate = target_sample_rate;
+    buffer.channels = 1;
+    try {
+        // A no-op copy when the rates already match, so the common 16 kHz
+        // upload pays only the mono mixdown it would have paid inside the
+        // family anyway.
+        buffer.samples =
+            engine::audio::convert_wav_to_mono_linear_resampled(wav, target_sample_rate);
+    } catch (const std::exception &err) {
+        // ConfigError, so this is INVALID_ARGUMENT rather than INTERNAL. What
+        // reaches here is a malformed input: a sample count that is not a whole
+        // number of frames is the realistic one, and it is the uploader's file
+        // that is truncated, not this backend that is broken.
+        throw ConfigError("audio-cpp: cannot resample " + path + " from " +
+                          std::to_string(wav.sample_rate) + " Hz to " +
+                          std::to_string(target_sample_rate) +
+                          " Hz: " + err.what());
+    }
+    return buffer;
+}
+
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio) {
+    if (path.empty()) {
+        throw ConfigError("audio-cpp: no output path was supplied");
+    }
+    const std::filesystem::path destination(path);
+    if (destination.has_parent_path()) {
+        // Best effort: a failure here shows up as a write failure below, with a
+        // message naming the file the caller actually asked for.
+        std::error_code ec;
+        std::filesystem::create_directories(destination.parent_path(), ec);
+    }
+    try {
+        engine::audio::write_pcm16_wav(destination, audio.sample_rate,
+                                       audio.channels > 0 ? audio.channels : 1,
+                                       audio.samples);
+    } catch (const std::exception &err) {
+        // NOT a ConfigError, and the distinction is not cosmetic. The
+        // destination is chosen by LocalAI rather than by the caller: it is a
+        // unique name inside GeneratedContentDir. A failure to write it is a
+        // full disk, a permission fault on the server's own directory, or a bad
+        // mount, none of which the caller can fix or is to blame for. As a
+        // ConfigError this surfaced as INVALID_ARGUMENT, which tells a client
+        // its request was wrong and not to retry; a plain runtime_error maps to
+        // INTERNAL, which is both true and retryable. The empty path above
+        // stays INVALID_ARGUMENT, because that one really is a malformed
+        // request.
+        throw std::runtime_error("audio-cpp: cannot write " + path + ": " +
+                                 err.what());
+    }
+}
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = 1;
+    buffer.samples = std::move(samples);
+    return buffer;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io.h

@@ -0,0 +1,61 @@
+#pragma once
+
+// Thin wrappers over the framework's public audio IO. Engine-linked, so this
+// unit is built and tested through the CMake target rather than by
+// backend/cpp/run-unit-tests.sh. The pure part of the arithmetic these
+// wrappers feed lives in audio_units, which is stdlib-only and does have a
+// standalone test.
+
+#include "engine/framework/runtime/session.h"
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Reads a WAV file. Throws ConfigError when the file is missing, is not
+// readable as WAV, or declares a non-positive sample rate: all three are
+// user-fixable input problems rather than backend faults.
+//
+// A declared sample rate of zero is refused rather than passed on, because
+// every downstream conversion in audio_units answers 0 for a non-positive rate.
+// Accepting it would turn a corrupt header into a response full of zero
+// timestamps, which reads as a real answer.
+//
+// `target_sample_rate` is the rate the CALLER needs, in Hz:
+//
+//   0 (or negative)  keep the file's own rate and channel count.
+//   positive         downmix to mono and resample to that rate. Resampling is
+//                    skipped when the file already declares it, so passing the
+//                    rate a route needs costs nothing on the common input.
+//
+// It is a parameter, and not a constant inside this function, because the
+// routes that read audio do not agree on an answer. Speech routes want 16 kHz
+// mono; source separation does not, and folding a 44.1 kHz stereo input to
+// 16 kHz mono for demucs or roformer would destroy the very thing they separate
+// (both refuse a rate other than their own outright). Making the caller name
+// the rate keeps that decision where the route is known.
+//
+// Downmixing along with the resample is not an extra liberty: every family a
+// positive rate is used for (silero_vad, sortformer_diar and every ASR family)
+// begins by calling the same mixdown_interleaved_to_mono_average on whatever it
+// is given. Doing it once here produces the identical samples and halves the
+// buffer that is then moved through the request.
+engine::runtime::AudioBuffer read_audio_file(const std::string &path,
+                                             int target_sample_rate);
+
+// Writes 16-bit PCM WAV, creating parent directories.
+//
+// Throws ConfigError, i.e. INVALID_ARGUMENT, ONLY for an empty path, which is a
+// malformed request. Every other failure throws a plain runtime_error, i.e.
+// INTERNAL: the destination is LocalAI's own generated-content directory and
+// not anything the caller named, so a full disk or a permission fault there is
+// a server fault and is worth retrying, which is the opposite of what
+// INVALID_ARGUMENT tells a client.
+void write_audio_file(const std::string &path,
+                      const engine::runtime::AudioBuffer &audio);
+
+engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
+                                              int sample_rate);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_io_ctest.cpp

@@ -0,0 +1,217 @@
+// Tests for audio_io's reading contract, and in particular for the resampling
+// that keeps a 44.1 or 48 kHz upload from reaching a family that only accepts
+// 16 kHz.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE: see the note at the top of
+// result_map_ctest.cpp. This file links the audio.cpp engine, so it is built
+// and run by ctest, not by backend/cpp/run-unit-tests.sh.
+//
+//   make -C backend/cpp/audio-cpp test-engine
+
+#include "audio_io.h"
+
+#include "loaded_model.h"
+
+#include <algorithm>
+#include <cmath>
+#include <cstdio>
+#include <filesystem>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// A one-second tone, interleaved across `channels`. Real audio rather than
+// silence so a resample that dropped its input would be visible as a flat
+// buffer, not just as a different length.
+static engine::runtime::AudioBuffer tone(int sample_rate, int channels,
+                                         float seconds) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = channels;
+    const auto frames =
+        static_cast<size_t>(static_cast<double>(sample_rate) * seconds);
+    buffer.samples.reserve(frames * static_cast<size_t>(channels));
+    for (size_t frame = 0; frame < frames; ++frame) {
+        const float value = 0.5f * std::sin(2.0f * 3.14159265f * 220.0f *
+                                            static_cast<float>(frame) /
+                                            static_cast<float>(sample_rate));
+        for (int channel = 0; channel < channels; ++channel) {
+            buffer.samples.push_back(value);
+        }
+    }
+    return buffer;
+}
+
+static float peak(const std::vector<float> &samples) {
+    float highest = 0.0f;
+    for (const float sample : samples) {
+        highest = std::max(highest, std::abs(sample));
+    }
+    return highest;
+}
+
+static std::filesystem::path scratch_dir() {
+    const auto dir = std::filesystem::temp_directory_path() / "audiocpp-io-ctest";
+    std::filesystem::create_directories(dir);
+    return dir;
+}
+
+// The I2 fixture. Before the resample this returned a 44.1 kHz buffer, which
+// silero_vad and sortformer_diar both reject with a plain runtime_error, which
+// the server maps to INTERNAL. A 44.1 kHz WAV is an ordinary upload.
+static void test_441k_stereo_is_read_as_16k_mono() {
+    const auto path = scratch_dir() / "input-44100-stereo.wav";
+    write_audio_file(path.string(), tone(44100, 2, 1.0f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "44.1 kHz input is resampled to 16 kHz");
+    check(audio.channels == 1, "stereo input is downmixed to mono");
+    // Linear resampling lands within a sample or two of the exact ratio.
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 15990 && frames < 16010,
+          "one second in stays one second out");
+    check(peak(audio.samples) > 0.2f,
+          "the resampled buffer still carries the signal");
+}
+
+static void test_48k_is_read_as_16k() {
+    const auto path = scratch_dir() / "input-48000-mono.wav";
+    write_audio_file(path.string(), tone(48000, 1, 0.5f));
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "48 kHz input is resampled to 16 kHz");
+    const auto frames = static_cast<long long>(audio.samples.size());
+    check(frames > 7990 && frames < 8010, "half a second in, half a second out");
+}
+
+// The common case: the upload is already 16 kHz mono, and nothing is resampled.
+static void test_16k_mono_passes_through_unchanged() {
+    const auto path = scratch_dir() / "input-16000-mono.wav";
+    const auto source = tone(16000, 1, 1.0f);
+    write_audio_file(path.string(), source);
+
+    const auto audio = read_audio_file(path.string(), 16000);
+    check(audio.sample_rate == 16000, "16 kHz stays 16 kHz");
+    check(audio.channels == 1, "mono stays mono");
+    check(audio.samples.size() == source.samples.size(),
+          "a matching rate resamples nothing");
+}
+
+// Rate 0 means "give me the file as it is", which is what a source separation
+// route needs: demucs and roformer refuse anything but their own 44.1 kHz and
+// work on stereo, so the reader must not force them to 16 kHz mono.
+static void test_zero_target_keeps_the_native_format() {
+    const auto path = scratch_dir() / "input-native.wav";
+    write_audio_file(path.string(), tone(44100, 2, 0.25f));
+
+    const auto audio = read_audio_file(path.string(), 0);
+    check(audio.sample_rate == 44100, "a zero target keeps the file's rate");
+    check(audio.channels == 2, "a zero target keeps the file's channels");
+}
+
+static void test_missing_file_is_a_config_error() {
+    bool threw_config_error = false;
+    try {
+        read_audio_file((scratch_dir() / "does-not-exist.wav").string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+        // Any other type maps to INTERNAL, which is what this asserts against.
+    }
+    check(threw_config_error, "a missing input file is INVALID_ARGUMENT, not INTERNAL");
+}
+
+static void test_unreadable_file_is_a_config_error() {
+    const auto path = scratch_dir() / "not-a-wav.wav";
+    {
+        FILE *file = fopen(path.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("this is not a RIFF header", file);
+            fclose(file);
+        }
+    }
+    bool threw_config_error = false;
+    try {
+        read_audio_file(path.string(), 16000);
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "a non-WAV input is INVALID_ARGUMENT, not INTERNAL");
+}
+
+// The write side of the same distinction. The destination is LocalAI's own
+// generated-content directory, not a caller-supplied path, so a failure to
+// write it is a server fault: INTERNAL, which a client may retry, and not
+// INVALID_ARGUMENT, which tells it the request itself was wrong.
+static void test_write_failure_is_not_a_config_error() {
+    // A regular file where a directory has to be. ENOTDIR defeats root as well
+    // as an ordinary user, unlike a chmod, which CI running as root would walk
+    // straight through.
+    const auto blocker = scratch_dir() / "blocking-file";
+    {
+        FILE *file = fopen(blocker.string().c_str(), "wb");
+        if (file != nullptr) {
+            fputs("not a directory", file);
+            fclose(file);
+        }
+    }
+    const auto path = blocker / "nested" / "out.wav";
+
+    bool threw_config_error = false;
+    bool threw_something = false;
+    try {
+        write_audio_file(path.string(), tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+        threw_something = true;
+    } catch (const std::exception &) {
+        threw_something = true;
+    }
+    check(threw_something, "an unwritable destination is reported at all");
+    check(!threw_config_error,
+          "a failed write is INTERNAL, not INVALID_ARGUMENT: the caller did not "
+          "choose the destination and cannot fix it");
+    check(!std::filesystem::exists(path), "and nothing was written");
+}
+
+static void test_empty_output_path_is_a_config_error() {
+    // The one write failure that IS the caller's: no path at all.
+    bool threw_config_error = false;
+    try {
+        write_audio_file("", tone(16000, 1, 0.05f));
+    } catch (const ConfigError &) {
+        threw_config_error = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config_error, "an empty output path stays INVALID_ARGUMENT");
+}
+
+int main() {
+    test_441k_stereo_is_read_as_16k_mono();
+    test_48k_is_read_as_16k();
+    test_16k_mono_passes_through_unchanged();
+    test_zero_target_keeps_the_native_format();
+    test_missing_file_is_a_config_error();
+    test_unreadable_file_is_a_config_error();
+    test_write_failure_is_not_a_config_error();
+    test_empty_output_path_is_a_config_error();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_io checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/audio_units.cpp

@@ -0,0 +1,139 @@
+#include "audio_units.h"
+
+#include <algorithm>
+#include <cmath>
+#include <limits>
+
+namespace audiocpp_backend {
+
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels) {
+    const std::size_t lanes = channels > 0 ? static_cast<std::size_t>(channels)
+                                           : static_cast<std::size_t>(1);
+    // Truncating division is deliberate: a trailing partial frame is not a
+    // position every channel reached, so counting it would overstate the length.
+    return static_cast<std::int64_t>(sample_count / lanes);
+}
+
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0;
+    }
+    // Split into whole seconds plus a remainder so the intermediate product
+    // cannot overflow on long recordings, and so rates like 44100 stay exact.
+    // The remainder division truncates deliberately: that matches Go's
+    // time.Duration conventions and keeps successive sample indices monotonic.
+    const std::int64_t rate = static_cast<std::int64_t>(sample_rate);
+    const std::int64_t whole_seconds = samples / rate;
+    const std::int64_t remainder = samples % rate;
+    return whole_seconds * 1000000000LL + (remainder * 1000000000LL) / rate;
+}
+
+float samples_to_seconds(std::int64_t samples, int sample_rate) {
+    if (sample_rate <= 0) {
+        return 0.0f;
+    }
+    return static_cast<float>(static_cast<double>(samples) /
+                              static_cast<double>(sample_rate));
+}
+
+std::int64_t seconds_to_samples(double seconds, int sample_rate) {
+    // !(seconds > 0.0) rather than seconds <= 0.0: every comparison against NaN
+    // is false, so the <= form lets NaN reach the cast below, which is undefined
+    // behaviour and lands on INT64_MIN in practice. This is the one entry point
+    // fed by untrusted-shaped input (a float-seconds timestamp off the wire, or
+    // a boundary from a model that diverged), and a hugely negative sample index
+    // used later as an offset or a length is a wild pointer rather than merely a
+    // wrong timestamp.
+    if (sample_rate <= 0 || !(seconds > 0.0)) {
+        return 0;
+    }
+    const double scaled = seconds * static_cast<double>(sample_rate);
+    // Bound before the cast for the same reason: converting a double at or above
+    // 2^63 (infinity included) is undefined behaviour, so saturate instead.
+    const double limit =
+        static_cast<double>(std::numeric_limits<std::int64_t>::max());
+    if (scaled >= limit) {
+        return std::numeric_limits<std::int64_t>::max();
+    }
+    // Round rather than truncate: these functions exist to cross the float
+    // seconds boundary the VAD and diarize messages use, so a value that came
+    // from samples_to_seconds converts back to the sample it started as.
+    // Truncation lost one sample about half the time, starting at n=1.
+    //
+    // That round trip is exact only below roughly 2^23 samples. Past that the
+    // float samples_to_seconds returns can no longer resolve adjacent indices
+    // and the trip fails whatever the rounding. Both the first failing INDEX
+    // and the duration it stands for depend on the rate, so they are listed per
+    // rate rather than folded into one range; measured:
+    //
+    //   16 kHz    16384001 samples   17.1 min
+    //   44.1 kHz  11289602 samples    4.3 min
+    //   48 kHz    12288002 samples    4.3 min
+    //   96 kHz    12288002 samples    2.1 min
+    //
+    // The shortest recording this bites is therefore a couple of minutes of
+    // 96 kHz audio. It is a property of the float seconds API itself, not of
+    // the rounding here, and it is why nothing should use these to carry a
+    // sample-accurate position in a long recording.
+    return static_cast<std::int64_t>(std::llround(scaled));
+}
+
+std::vector<float> s16le_to_f32(const std::string &bytes) {
+    std::vector<float> samples;
+    const size_t count = bytes.size() / 2;
+    samples.reserve(count);
+    for (size_t i = 0; i < count; ++i) {
+        const auto low = static_cast<unsigned char>(bytes[i * 2]);
+        const auto high = static_cast<unsigned char>(bytes[i * 2 + 1]);
+        const auto raw = static_cast<std::int16_t>(
+            static_cast<std::uint16_t>(low) |
+            (static_cast<std::uint16_t>(high) << 8));
+        // 32768 on decode against 32767 on encode is deliberate, not a typo.
+        // 32768 is what keeps INT16_MIN at exactly -1.0 and every other code
+        // inside the [-1, 1] range this header promises; dividing by 32767
+        // would decode INT16_MIN to -1.00003. See f32_to_s16le for the other
+        // half of the pair. The cost is that a round trip shrinks a sample by
+        // 32767/32768, well under one LSB.
+        samples.push_back(static_cast<float>(raw) / 32768.0f);
+    }
+    return samples;
+}
+
+std::string f32_to_s16le(const std::vector<float> &samples) {
+    std::string bytes;
+    bytes.reserve(samples.size() * 2);
+    for (const float sample : samples) {
+        // NaN maps to silence. A NaN sample rendered as a full-scale click is
+        // worse audio than a dropped one, and this unit converts audio that may
+        // have originated off the wire.
+        //
+        // This guard also removes what used to be a spelling hazard in the
+        // clamp below. std::min and std::max return their first argument when
+        // the comparison is false, and every comparison against NaN is false,
+        // so before this branch existed the choice of spelling silently decided
+        // whether a NaN reached std::lround, whose result is unspecified for
+        // NaN. These three leaked it, the last being the idiomatic C++17 way to
+        // write a clamp and so the likeliest future edit:
+        //   std::min(std::max(sample, -1.0f), 1.0f)
+        //   std::max(std::min(sample, 1.0f), -1.0f)
+        //   std::clamp(sample, -1.0f, 1.0f)
+        // The order is no longer load-bearing now that the guard runs first,
+        // but the history is why the guard is here, so do not drop it.
+        if (std::isnan(sample)) {
+            bytes.push_back(0);
+            bytes.push_back(0);
+            continue;
+        }
+        const float clamped = std::max(-1.0f, std::min(1.0f, sample));
+        // 32767 rather than 32768 so +1.0 saturates at INT16_MAX instead of
+        // overflowing to INT16_MIN. See s16le_to_f32 for why decode differs.
+        const auto value =
+            static_cast<std::int16_t>(std::lround(clamped * 32767.0f));
+        const auto raw = static_cast<std::uint16_t>(value);
+        bytes.push_back(static_cast<char>(raw & 0xFF));
+        bytes.push_back(static_cast<char>((raw >> 8) & 0xFF));
+    }
+    return bytes;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units.h

@@ -0,0 +1,50 @@
+#pragma once
+
+// Time and sample-format conversion between audio.cpp's runtime types (sample
+// indices, float PCM) and LocalAI's proto types. Standard library only.
+//
+// LocalAI uses three different time units:
+//   TranscriptSegment / TranscriptWord start,end : int64 nanoseconds
+//   VADSegment start,end                         : float seconds
+//   DiarizeSegment start,end                     : float seconds
+
+#include <cstddef>
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Frames in an interleaved buffer of `sample_count` floats laid out across
+// `channels` channels. A frame is one per-channel position, which is the unit
+// every duration and every span boundary in this backend is expressed in, so a
+// stereo buffer must not report twice its real length: feeding sample_count
+// straight to samples_to_seconds makes a 3 second stereo clip come back as 6.
+//
+// A non-positive channel count is treated as mono, matching
+// engine::runtime::AudioBuffer's own default of 1 and keeping a reader that
+// reports 0 channels from dividing by zero.
+std::int64_t interleaved_frame_count(std::size_t sample_count, int channels);
+
+// Returns 0 when sample_rate is not positive rather than dividing by zero.
+// Uses integer arithmetic so 44.1 kHz does not lose precision.
+std::int64_t samples_to_nanoseconds(std::int64_t samples, int sample_rate);
+
+float samples_to_seconds(std::int64_t samples, int sample_rate);
+
+// Rounds to nearest. Negative seconds and NaN both yield 0, and a value too
+// large to convert saturates at INT64_MAX rather than overflowing. Round trips
+// with samples_to_seconds only below roughly 2^23 samples, past which the float
+// seconds can no longer resolve adjacent sample indices.
+std::int64_t seconds_to_samples(double seconds, int sample_rate);
+
+// Decodes little-endian signed 16-bit PCM. A trailing odd byte is dropped.
+std::vector<float> s16le_to_f32(const std::string &bytes);
+
+// Encodes to little-endian signed 16-bit PCM, clamping to [-1, 1] first so an
+// overshooting sample saturates instead of wrapping to the opposite sign.
+// A NaN sample encodes to 0, on the grounds that silence beats a full-scale
+// click.
+std::string f32_to_s16le(const std::vector<float> &samples);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/audio_units_test.cpp

@@ -0,0 +1,226 @@
+// Unit tests for audio_units. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+
+#include "audio_units.cpp"
+
+#include <cfenv>
+#include <cmath>
+#include <cstdio>
+#include <limits>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static bool close_to(float a, float b, float tol) { return std::fabs(a - b) <= tol; }
+
+using namespace audiocpp_backend;
+
+static void test_nanoseconds() {
+    // LocalAI TranscriptSegment/TranscriptWord times are nanoseconds
+    // (Go reads them as time.Duration).
+    check(samples_to_nanoseconds(16000, 16000) == 1000000000LL, "1s at 16k is 1e9 ns");
+    check(samples_to_nanoseconds(8000, 16000) == 500000000LL, "0.5s at 16k");
+    check(samples_to_nanoseconds(0, 16000) == 0, "zero samples is zero ns");
+    check(samples_to_nanoseconds(1000, 0) == 0, "zero sample rate yields zero, not UB");
+    // 44.1 kHz must not lose precision to float arithmetic.
+    check(samples_to_nanoseconds(44100, 44100) == 1000000000LL, "1s at 44.1k");
+    check(samples_to_nanoseconds(22050, 44100) == 500000000LL, "0.5s at 44.1k");
+    // The cases above all land on values a float happens to hold exactly, so
+    // they do not actually rule float arithmetic out. These do:
+    // a fraction that does not divide evenly, and a duration whose magnitude
+    // exceeds a float's 24-bit mantissa at nanosecond resolution.
+    check(samples_to_nanoseconds(44099, 44100) == 999977324LL,
+          "44.1k fraction is exact, not rounded through a float");
+    check(samples_to_nanoseconds(44100LL * 3600, 44100) == 3600000000000LL,
+          "one hour at 44.1k is exact to the nanosecond");
+    // A naive samples * 1e9 would overflow int64 here; the split into whole
+    // seconds plus a remainder is what keeps this correct.
+    check(samples_to_nanoseconds(44100LL * 360000, 44100) == 360000000000000LL,
+          "100 hours at 44.1k does not overflow");
+    // Double arithmetic is close enough to pass everything above, but still
+    // truncates this one a nanosecond short. Integer division does not.
+    check(samples_to_nanoseconds(4004, 8000) == 500500000LL,
+          "0.5005s at 8k is exact to the nanosecond");
+    // Truncation, not rounding: this matches Go's time.Duration conventions and
+    // keeps successive sample indices monotonic. The exact value here is
+    // 22675.7...; rounding to nearest would give 22676.
+    check(samples_to_nanoseconds(1, 44100) == 22675LL,
+          "a sub-nanosecond fraction truncates rather than rounding up");
+}
+
+static void test_seconds() {
+    check(close_to(samples_to_seconds(24000, 24000), 1.0f, 1e-6f), "1s at 24k");
+    check(close_to(samples_to_seconds(12000, 24000), 0.5f, 1e-6f), "0.5s at 24k");
+    check(close_to(samples_to_seconds(100, 0), 0.0f, 1e-6f), "zero sample rate is 0s");
+    check(seconds_to_samples(1.0, 16000) == 16000, "1s to samples at 16k");
+    check(seconds_to_samples(0.5, 16000) == 8000, "0.5s to samples at 16k");
+    check(seconds_to_samples(1.0, 0) == 0, "zero sample rate yields zero samples");
+    check(seconds_to_samples(-1.0, 16000) == 0, "negative seconds clamps to zero");
+
+    // seconds_to_samples is the one entry point fed by untrusted-shaped input:
+    // a float-seconds timestamp off the wire, or a VAD boundary from a model
+    // that diverged. A hugely negative sample index used later as an offset or
+    // a length is a wild pointer, not merely a wrong timestamp.
+    const double nan_seconds = std::numeric_limits<double>::quiet_NaN();
+    const double inf_seconds = std::numeric_limits<double>::infinity();
+    const std::int64_t max_samples = std::numeric_limits<std::int64_t>::max();
+    check(seconds_to_samples(nan_seconds, 16000) == 0, "NaN seconds yields zero");
+    check(seconds_to_samples(inf_seconds, 16000) == max_samples,
+          "infinite seconds saturates instead of overflowing");
+    check(seconds_to_samples(1e30, 16000) == max_samples,
+          "out of range seconds saturates instead of overflowing");
+    check(seconds_to_samples(-inf_seconds, 16000) == 0,
+          "negative infinity clamps to zero");
+
+    // Crossing the float-seconds boundary and back is the expected round trip
+    // for the VAD and diarize messages, so it must not lose a sample.
+    // Truncation loses one about half the time, starting at n=1.
+    check(seconds_to_samples(samples_to_seconds(1, 44100), 44100) == 1,
+          "one sample survives the seconds round trip at 44.1k");
+    check(seconds_to_samples(samples_to_seconds(1, 16000), 16000) == 1,
+          "one sample survives the seconds round trip at 16k");
+    check(seconds_to_samples(samples_to_seconds(4001, 8000), 8000) == 4001,
+          "4001 samples survive the seconds round trip at 8k");
+}
+
+static void test_s16le_round_trip() {
+    const std::vector<float> original = {0.0f, 0.5f, -0.5f, 1.0f, -1.0f};
+    const std::string encoded = f32_to_s16le(original);
+    check(encoded.size() == original.size() * 2, "two bytes per sample");
+
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == original.size(), "round trip keeps the sample count");
+    for (size_t i = 0; i < original.size(); ++i) {
+        // 16-bit quantisation: one LSB is ~3.05e-5. Guard the index so a short
+        // result reports a named failure instead of aborting the whole suite.
+        check(i < decoded.size() && close_to(decoded[i], original[i], 1e-4f),
+              "round trip preserves sample " + std::to_string(i));
+    }
+}
+
+static void test_s16le_endianness() {
+    // 0.5 encodes to 16384 = 0x4000, little endian is 0x00 0x40.
+    const std::string encoded = f32_to_s16le({0.5f});
+    check(encoded.size() == 2, "one sample is two bytes");
+    check(static_cast<unsigned char>(encoded[0]) == 0x00, "low byte first");
+    check(static_cast<unsigned char>(encoded[1]) == 0x40, "high byte second");
+}
+
+static void test_s16le_clamping() {
+    // Values outside [-1, 1] must clamp, not wrap around to the opposite sign.
+    const std::string encoded = f32_to_s16le({2.0f, -2.0f});
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+    check(decoded.size() == 2, "two samples survive clamping");
+    check(decoded.size() > 0 && decoded[0] > 0.99f,
+          "positive overshoot clamps to full scale");
+    check(decoded.size() > 1 && decoded[1] < -0.99f,
+          "negative overshoot clamps to full scale");
+}
+
+static void test_s16le_decode_range() {
+    // INT16_MIN is the one value that pins the decode scale. Dividing by 32767
+    // instead of 32768 would decode it to -1.00003, outside the [-1, 1] range
+    // the header promises, and every other test would still pass.
+    const std::vector<float> decoded = s16le_to_f32(std::string("\x00\x80", 2));
+    check(decoded.size() == 1, "INT16_MIN decodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == -1.0f,
+          "INT16_MIN decodes to exactly -1.0, not past full scale");
+}
+
+static void test_s16le_nan_input() {
+    // A NaN sample must not reach std::lround, whose result is unspecified for
+    // NaN. Asserting a range is not enough to pin this: the three outcomes the
+    // plausible clamp spellings produce (full scale, negative full scale, zero)
+    // are all finite and all inside [-1, 1], so a range check passes for every
+    // one of them. Only an exact value distinguishes them.
+    // NaN maps to silence, not to full scale: a NaN sample rendered as a
+    // full-scale click is worse audio than a dropped one, and this unit
+    // converts audio that may have originated off the wire.
+    //
+    // volatile so the NaN cannot be constant-folded, which would let the
+    // compiler evaluate the conversion at compile time and raise no
+    // floating-point exception at run time for the check below to observe.
+    volatile float nan_source = std::numeric_limits<float>::quiet_NaN();
+    const std::vector<float> input = {nan_source};
+
+    std::feclearexcept(FE_ALL_EXCEPT);
+    const std::string encoded = f32_to_s16le(input);
+    const bool raised_invalid = std::fetestexcept(FE_INVALID) != 0;
+    const std::vector<float> decoded = s16le_to_f32(encoded);
+
+    check(decoded.size() == 1, "a NaN sample still encodes to one sample");
+    check(decoded.size() == 1 && decoded[0] == 0.0f,
+          "a NaN sample encodes to exactly zero, not to a full-scale click");
+    // Independent of the value: a quiet NaN raises invalid-operation as soon as
+    // it reaches any ordered comparison, which is what std::min and std::max
+    // use, so this fails unless the NaN is diverted before the clamp runs at
+    // all. That is what stops the explicit guard from being dropped in favour
+    // of a clamp spelling that happens to yield zero.
+    check(!raised_invalid,
+          "encoding a NaN sample raises no invalid-operation exception");
+}
+
+static void test_s16le_odd_length() {
+    // A truncated frame must drop the dangling byte rather than read past it.
+    const std::string odd(5, '\0');
+    check(s16le_to_f32(odd).size() == 2, "odd byte count drops the trailing byte");
+    check(s16le_to_f32(std::string()).empty(), "empty input yields no samples");
+}
+
+static void test_interleaved_frame_count() {
+    // Mono is a pass-through, which is the only case the VAD path exercises.
+    check(interleaved_frame_count(16000, 1) == 16000, "mono frames equal samples");
+    // The case that matters: a stereo buffer holds two floats per position, so a
+    // one second 16 kHz stereo clip is 32000 floats and still one second. Handing
+    // the raw float count to samples_to_seconds reports two seconds instead.
+    check(interleaved_frame_count(32000, 2) == 16000,
+          "stereo frames are half the samples");
+    check(samples_to_seconds(interleaved_frame_count(32000, 2), 16000) == 1.0f,
+          "a one second stereo clip measures one second, not two");
+    check(interleaved_frame_count(48000, 3) == 16000,
+          "three channels divide by three");
+    // engine::runtime::AudioBuffer defaults channels to 1, but a reader is free
+    // to report 0, and dividing by that is undefined rather than merely wrong.
+    check(interleaved_frame_count(1000, 0) == 1000,
+          "zero channels is treated as mono");
+    check(interleaved_frame_count(1000, -2) == 1000,
+          "a negative channel count is treated as mono");
+    // A dangling partial frame is not a position every channel reached.
+    check(interleaved_frame_count(3, 2) == 1,
+          "a trailing partial frame is not counted");
+    check(interleaved_frame_count(0, 2) == 0, "an empty buffer has no frames");
+    // Past 2^32 floats, so a size_t narrowed to 32 bits on the way in, or a
+    // signed 32-bit intermediate, shows up here rather than in a multi-hour
+    // recording nobody tests with.
+    check(interleaved_frame_count(static_cast<std::size_t>(9000000000ULL), 2) ==
+              4500000000LL,
+          "a buffer beyond 2^32 floats counts frames without truncating");
+}
+
+int main() {
+    test_interleaved_frame_count();
+    test_nanoseconds();
+    test_seconds();
+    test_s16le_round_trip();
+    test_s16le_endianness();
+    test_s16le_clamping();
+    test_s16le_decode_range();
+    test_s16le_nan_input();
+    test_s16le_odd_length();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all audio_units checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/capability_routing.cpp

@@ -0,0 +1,411 @@
+#include "capability_routing.h"
+
+#include <algorithm>
+
+namespace audiocpp_backend {
+namespace {
+
+struct NamedTask {
+    Task task;
+    const char *name;
+};
+
+// Short names are exactly the strings audio.cpp prints and parses in
+// framework/runtime/session.cpp, so a name pinned here survives conversion at
+// the engine boundary and a name copied out of audio.cpp is accepted here. All
+// thirteen have an upstream name; only "spk" is absent from the --task table in
+// docs/usage.md.
+const NamedTask kTaskNames[] = {
+    {Task::Vad, "vad"},
+    {Task::Asr, "asr"},
+    {Task::Diarization, "diar"},
+    {Task::SourceSeparation, "sep"},
+    {Task::AudioGeneration, "gen"},
+    {Task::Tts, "tts"},
+    {Task::VoiceCloning, "clon"},
+    {Task::VoiceConversion, "vc"},
+    {Task::SpeechToSpeech, "s2s"},
+    {Task::Alignment, "align"},
+    {Task::VoiceDesign, "vdes"},
+    {Task::SpeakerRecognition, "spk"},
+    {Task::Svc, "svc"},
+};
+
+// Accepted on input but never emitted. "spkrec" was this backend's own earlier
+// name for the kind; upstream only ever knew "spk".
+const NamedTask kTaskAliases[] = {
+    {Task::SpeakerRecognition, "spkrec"},
+};
+
+// First match wins, which is safe because a Capabilities value holds at most one
+// entry per task: it mirrors upstream runtime::TaskCapability (model.h), which
+// pairs one kind with a modes vector, and no loader's supported_tasks list
+// repeats a kind.
+bool family_supports(const Capabilities &caps, Task task, Mode mode) {
+    for (const auto &capability : caps.tasks) {
+        if (capability.task != task) {
+            continue;
+        }
+        return std::find(capability.modes.begin(), capability.modes.end(), mode) !=
+               capability.modes.end();
+    }
+    return false;
+}
+
+// Mode preference per RPC. Only AudioTranscriptionStream has a fallback: a
+// server-streaming transcription can be satisfied by an offline run that emits
+// one delta then the final result. Live transcription cannot, because it is
+// bidirectional and must consume audio incrementally.
+std::vector<Mode> mode_candidates(Rpc rpc) {
+    switch (rpc) {
+    case Rpc::TtsStream:
+    case Rpc::AudioTranscriptionLive:
+        return {Mode::Streaming};
+    case Rpc::AudioTranscriptionStream:
+        return {Mode::Streaming, Mode::Offline};
+    default:
+        return {Mode::Offline};
+    }
+}
+
+std::vector<Task> task_candidates(Rpc rpc, const RequestShape &shape) {
+    switch (rpc) {
+    case Rpc::Tts:
+    case Rpc::TtsStream:
+        // A supplied speaker clip is the strongest signal: the caller named the
+        // voice they want. Free-form instructions come next. Both fall back to
+        // plain Tts so a family without the specialised task still answers.
+        if (shape.has_voice_reference) {
+            return {Task::VoiceCloning, Task::Tts, Task::VoiceDesign};
+        }
+        if (shape.has_instructions) {
+            return {Task::VoiceDesign, Task::Tts, Task::VoiceCloning};
+        }
+        return {Task::Tts, Task::VoiceCloning, Task::VoiceDesign};
+    case Rpc::AudioTranscription:
+    case Rpc::AudioTranscriptionStream:
+    case Rpc::AudioTranscriptionLive:
+        // Asr first: `prompt` is also whisper-style decoding context, so its
+        // presence must not hijack a real ASR family into forced alignment.
+        if (shape.has_prompt_text) {
+            return {Task::Asr, Task::Alignment};
+        }
+        return {Task::Asr};
+    case Rpc::Vad:
+        return {Task::Vad};
+    case Rpc::Diarize:
+        return {Task::Diarization};
+    case Rpc::SoundGeneration:
+        return {Task::AudioGeneration};
+    case Rpc::AudioTransform:
+        // Svc is listed for completeness but is unreachable by auto-routing, by
+        // design: the only families advertising it (seed_vc, vevo2) also
+        // advertise VoiceConversion, which always wins, and no request signal
+        // means "this input is singing". Singing voice conversion therefore
+        // requires an explicit task:svc pin.
+        return {Task::SourceSeparation, Task::VoiceConversion, Task::Svc,
+                Task::SpeechToSpeech};
+    }
+    return {};
+}
+
+// The reasons behind unsupported_surfaces(), spelled once because AudioEncode
+// and AudioDecode share theirs. Each is phrased in terms of what upstream does
+// and does not have, so a reader can check it against the pinned checkout
+// rather than take it on trust. Every one of them was checked against
+// audio.cpp e800d435d130dc776baf6f3e6129bb62b1495c89, and one of the four
+// claims this backend was planned against did not survive that check: see
+// kTransformStreamReason.
+//
+// A latent upstream inconsistency worth knowing about but deliberately NOT put
+// on the wire, because it would mislead: model_spec/schema.cpp's task-string
+// whitelist does accept "codec" (and "dialogue"), while
+// model_spec/metadata.cpp's parse_task_kind has no branch for either and
+// throws "unknown model spec task". So a spec declaring "codec" validates and
+// then fails to load. That is a hole in upstream's own validation, not a codec
+// task this backend could reach.
+const char *const kCodecReason =
+    "audio.cpp's VoiceTaskKind has no codec entry, so no family can be asked to "
+    "turn PCM into codec frames or back; miocodec carries a Codec tag in "
+    "upstream's README but its loader advertises only vc and s2s";
+// NOT "streaming exists for tts and asr only", which is what this backend was
+// planned to say and is false: silero_vad advertises vad with RunMode::Streaming
+// (src/models/silero_vad/session.cpp). The claim that actually holds is the
+// narrower one below, about the four tasks AudioTransform routes to.
+// The trailing clause is not padding. The premise is an absence, and an absence
+// does not on its own make the RPC impossible: an offline sep family could be
+// buffered and emitted as a stream, which is what several LocalAI backends do.
+// Stopping at "nothing advertises streaming" would imply an impossibility the
+// evidence does not support. What is true, and what the caller needs, is that
+// this backend declines to dress an offline call up as a streaming one.
+const char *const kTransformStreamReason =
+    "no audio.cpp family advertises streaming for any task AudioTransform routes "
+    "to (sep, vc, svc, s2s); upstream advertises RunMode::Streaming for tts, asr "
+    "and vad only, and no conversion or separation family even implements its "
+    "IStreamingVoiceTaskSession interface, so a streaming transform here would be "
+    "a buffered offline call in disguise, which this backend does not pretend to "
+    "offer";
+// "clip-to-clip processing against a target voice", NOT "voice conversion". The
+// latter is true of miocodec and FALSE of vevo2, whose s2s route is `editing`
+// and only `editing`: src/models/vevo2/session.cpp's default_route_for_task maps
+// SpeechToSpeech to Editing and route_matches_task accepts nothing else, and
+// docs/models/vevo2.md defines that route as "Edit source speech into new target
+// text while using the target voice", requiring --target-text. It rewrites what
+// was said. vevo2's actual voice conversion is its separate vc task, which is
+// why upstream's README tags the family "TTS, Music, VC, Edit". The conclusion
+// is unaffected: neither family converses.
+const char *const kAudioToAudioReason =
+    "LocalAI's contract here is OpenAI-Realtime shaped, an audio conversation "
+    "emitting audio, transcript and tool-call deltas from a system prompt and a "
+    "tool list; audio.cpp's s2s is offline clip-to-clip processing against a "
+    "target voice, declared only by miocodec (voice conversion) and vevo2 "
+    "(speech editing), with no conversation, system prompt or tool loop";
+const char *const kVoiceEmbedReason =
+    "no audio.cpp family advertises the spk (SpeakerRecognition) task, so "
+    "nothing in the engine can produce a speaker embedding; the task kind "
+    "itself exists upstream, and TitaNet and ECAPA-TDNN exist as internal "
+    "conditioning encoders, but neither is registered as a loadable family";
+
+// The tasks an RPC is ever willing to route to, independent of request shape.
+//
+// DERIVED from task_candidates rather than restated, so a task added to an
+// RPC's candidate list cannot become inadmissible as a pin by omission. Setting
+// every shape flag yields each RPC's widest list: the per-flag branches only
+// reorder the same three tasks for Tts, and only ADD Alignment for
+// transcription, so the union is what comes back.
+std::vector<Task> admissible_tasks(Rpc rpc) {
+    RequestShape widest;
+    widest.has_voice_reference = true;
+    widest.has_instructions = true;
+    widest.has_prompt_text = true;
+    return task_candidates(rpc, widest);
+}
+
+std::string join_task_names(const std::vector<Task> &tasks) {
+    std::string out;
+    for (const Task task : tasks) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += task_name(task);
+    }
+    if (out.empty()) {
+        out = "nothing";
+    }
+    return out;
+}
+
+std::string join_attempts(const std::vector<Task> &tasks,
+                          const std::vector<Mode> &modes) {
+    std::string out;
+    for (const Task task : tasks) {
+        for (const Mode mode : modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += task_name(task);
+            out += "/";
+            out += mode_name(mode);
+        }
+    }
+    return out;
+}
+
+} // namespace
+
+const char *task_name(Task task) {
+    for (const auto &entry : kTaskNames) {
+        if (entry.task == task) {
+            return entry.name;
+        }
+    }
+    return "unknown";
+}
+
+const char *mode_name(Mode mode) {
+    return mode == Mode::Streaming ? "streaming" : "offline";
+}
+
+const char *rpc_name(Rpc rpc) {
+    switch (rpc) {
+    case Rpc::Tts:
+        return "TTS";
+    case Rpc::TtsStream:
+        return "TTSStream";
+    case Rpc::AudioTranscription:
+        return "AudioTranscription";
+    case Rpc::AudioTranscriptionStream:
+        return "AudioTranscriptionStream";
+    case Rpc::AudioTranscriptionLive:
+        return "AudioTranscriptionLive";
+    case Rpc::Vad:
+        return "VAD";
+    case Rpc::Diarize:
+        return "Diarize";
+    case Rpc::SoundGeneration:
+        return "SoundGeneration";
+    case Rpc::AudioTransform:
+        return "AudioTransform";
+    }
+    return "unknown";
+}
+
+bool parse_task_name(const std::string &value, Task &out) {
+    for (const auto &entry : kTaskNames) {
+        if (value == entry.name) {
+            out = entry.task;
+            return true;
+        }
+    }
+    for (const auto &entry : kTaskAliases) {
+        if (value == entry.name) {
+            out = entry.task;
+            return true;
+        }
+    }
+    return false;
+}
+
+std::string describe_capabilities(const Capabilities &caps) {
+    std::string out;
+    for (const auto &capability : caps.tasks) {
+        for (const Mode mode : capability.modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += task_name(capability.task);
+            out += "/";
+            out += mode_name(mode);
+        }
+    }
+    if (out.empty()) {
+        out = "nothing";
+    }
+    return out;
+}
+
+const std::vector<UnsupportedSurface> &unsupported_surfaces() {
+    // Ordered as UnsupportedRpc declares them. unsupported_surface() names each
+    // index in a switch rather than casting the enum, so the order is checked at
+    // compile time rather than trusted.
+    static const std::vector<UnsupportedSurface> kSurfaces = {
+        {"AudioEncode", kCodecReason},
+        {"AudioDecode", kCodecReason},
+        {"AudioTransformStream", kTransformStreamReason},
+        {"AudioToAudioStream", kAudioToAudioReason},
+        {"VoiceEmbed", kVoiceEmbedReason},
+    };
+    return kSurfaces;
+}
+
+// A switch with NO default label, deliberately. -Wswitch is on under -Wall, so a
+// sixth UnsupportedRpc added without a case here is a BUILD diagnostic, which is
+// the only place this class of mistake can be caught for free: a positional
+// static_cast<size_t>(rpc) would compile fine and read past the end of the table
+// at run time, on the one code path whose entire job is to be diagnosable. The
+// table stays a table because the tests iterate it.
+//
+// The trailing return is unreachable through the enum and exists only for a
+// caller that hands over a value outside it, which is already undefined
+// behaviour by the time it arrives.
+const UnsupportedSurface &unsupported_surface(UnsupportedRpc rpc) {
+    const std::vector<UnsupportedSurface> &surfaces = unsupported_surfaces();
+    switch (rpc) {
+    case UnsupportedRpc::AudioEncode:
+        return surfaces[0];
+    case UnsupportedRpc::AudioDecode:
+        return surfaces[1];
+    case UnsupportedRpc::AudioTransformStream:
+        return surfaces[2];
+    case UnsupportedRpc::AudioToAudioStream:
+        return surfaces[3];
+    case UnsupportedRpc::VoiceEmbed:
+        return surfaces[4];
+    }
+    return surfaces[0];
+}
+
+std::string unsupported_surface_message(const Capabilities &caps, const char *rpc,
+                                        const char *reason) {
+    return std::string("audio-cpp: the ") + rpc +
+           " RPC is not available through this backend because " + reason +
+           ". Loaded family '" + caps.family +
+           "' supports: " + describe_capabilities(caps);
+}
+
+std::string unsupported_surface_message(const char *rpc, const char *reason) {
+    return std::string("audio-cpp: the ") + rpc +
+           " RPC is not available through this backend because " + reason +
+           ". No model is loaded, so there is no family to list; loading one "
+           "would not change this answer";
+}
+
+Route resolve_route(Rpc rpc, const RequestShape &shape,
+                    const Capabilities &caps) {
+    Route route;
+
+    std::vector<Task> tasks;
+    if (!shape.pinned_task.empty()) {
+        Task pinned = Task::Tts;
+        if (!parse_task_name(shape.pinned_task, pinned)) {
+            route.error = "audio-cpp: unknown task option '" + shape.pinned_task +
+                          "'. Known tasks: gen, tts, clon, vc, svc, s2s, asr, "
+                          "align, vad, diar, sep, vdes, spk";
+            return route;
+        }
+        // A pin is honoured exactly, but ONLY on an RPC that could have routed
+        // to it anyway. It used to replace the candidate list wholesale for
+        // every RPC, and because the model's `task:` option is copied into the
+        // shape by all nine handlers, one pin bled across all nine surfaces and
+        // produced wrong 200s rather than errors: nemotron with task:asr made
+        // Vad return 200 with zero segments after a full ASR decode, so 14
+        // seconds of speech was reported as silence, and silero_vad with
+        // task:vad made AudioTranscription return 200 with empty text and four
+        // segments whose spans were VAD segments, which the srt/vtt/lrc writers
+        // then rendered as a well formed subtitle file of four timed EMPTY
+        // cues. Refusing is what the docs already promise: "if the family
+        // cannot serve it, the request is refused rather than rerouted".
+        //
+        // Every legitimate pin survives, because a pin only ever names the task
+        // its own RPC already routes to: svc is in AudioTransform's candidates,
+        // tts/clon/vdes in TTS's, asr in transcription's, vad and diar in
+        // theirs.
+        const std::vector<Task> admissible = admissible_tasks(rpc);
+        if (std::find(admissible.begin(), admissible.end(), pinned) ==
+            admissible.end()) {
+            route.error = std::string("audio-cpp: this model pins task '") +
+                          task_name(pinned) + "', which the " + rpc_name(rpc) +
+                          " RPC never routes to (it routes to " +
+                          join_task_names(admissible) +
+                          "). Remove the task option to reach this RPC, or call "
+                          "the RPC the pinned task serves";
+            return route;
+        }
+        tasks = {pinned};
+    } else {
+        tasks = task_candidates(rpc, shape);
+    }
+
+    const std::vector<Mode> modes = mode_candidates(rpc);
+
+    // Task-major: prefer the right task in a fallback mode over the wrong task
+    // in the preferred mode.
+    for (const Task task : tasks) {
+        for (const Mode mode : modes) {
+            if (family_supports(caps, task, mode)) {
+                route.ok = true;
+                route.task = task;
+                route.mode = mode;
+                return route;
+            }
+        }
+    }
+
+    route.error = std::string("audio-cpp: family '") + caps.family +
+                  "' cannot serve the " + rpc_name(rpc) + " RPC (tried " +
+                  join_attempts(tasks, modes) + "); it supports: " +
+                  describe_capabilities(caps);
+    return route;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/capability_routing.h

@@ -0,0 +1,131 @@
+#pragma once
+
+// Decides which audio.cpp (task, mode) pair serves a given LocalAI RPC, or
+// produces the capability error when none can. Standard library only, so this
+// unit is tested without an audio.cpp checkout; loaded_model.cpp converts
+// to and from engine::runtime types at the boundary.
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Mirrors engine::runtime::VoiceTaskKind, same members and same order.
+enum class Task {
+    Vad,
+    Asr,
+    Diarization,
+    SourceSeparation,
+    AudioGeneration,
+    Tts,
+    VoiceCloning,
+    VoiceConversion,
+    SpeechToSpeech,
+    Alignment,
+    VoiceDesign,
+    SpeakerRecognition,
+    Svc,
+};
+
+// Mirrors engine::runtime::RunMode.
+enum class Mode { Offline, Streaming };
+
+struct TaskCapability {
+    Task task = Task::Vad;
+    std::vector<Mode> modes;
+};
+
+struct Capabilities {
+    std::string family;
+    std::vector<TaskCapability> tasks;
+};
+
+// The LocalAI RPCs this backend serves. The ones it cannot serve at all are in
+// UnsupportedRpc below rather than here: they never reach routing, because no
+// family could satisfy them.
+enum class Rpc {
+    Tts,
+    TtsStream,
+    AudioTranscription,
+    AudioTranscriptionStream,
+    AudioTranscriptionLive,
+    Vad,
+    Diarize,
+    SoundGeneration,
+    AudioTransform,
+};
+
+struct RequestShape {
+    // A speaker reference clip was supplied (TTSRequest.voice resolved to audio).
+    bool has_voice_reference = false;
+    // TTSRequest.instructions is set.
+    bool has_instructions = false;
+    // TranscriptRequest.prompt is set.
+    bool has_prompt_text = false;
+    // The model's `task:` option, empty when unset. Overrides routing.
+    std::string pinned_task;
+};
+
+struct Route {
+    bool ok = false;
+    Task task = Task::Tts;
+    Mode mode = Mode::Offline;
+    // Set when ok is false. Suitable verbatim as an UNIMPLEMENTED message.
+    std::string error;
+};
+
+Route resolve_route(Rpc rpc, const RequestShape &shape, const Capabilities &caps);
+
+// Canonical audio.cpp short names: gen, tts, clon, vc, svc, s2s, asr, align,
+// vad, diar, sep, vdes, spk. parse_task_name additionally accepts "spkrec" as
+// a legacy alias; task_name only ever emits "spk".
+const char *task_name(Task task);
+const char *mode_name(Mode mode);
+const char *rpc_name(Rpc rpc);
+bool parse_task_name(const std::string &value, Task &out);
+
+// "asr/offline, asr/streaming", for error messages.
+std::string describe_capabilities(const Capabilities &caps);
+
+// The RPCs in LocalAI's backend contract that audio.cpp has no counterpart for,
+// as opposed to the ones in Rpc above, which a particular family may or may not
+// be able to serve. Nothing routes to these: the refusal is a property of the
+// engine, not of the loaded model, so loading a different family cannot change
+// it.
+//
+// This is deliberately NOT deferred work. Each entry names the upstream
+// limitation that keeps it out of Rpc, and each becomes an ordinary routing
+// entry the day upstream lifts that limitation.
+enum class UnsupportedRpc {
+    AudioEncode,
+    AudioDecode,
+    AudioTransformStream,
+    AudioToAudioStream,
+    VoiceEmbed,
+};
+
+struct UnsupportedSurface {
+    // The RPC's name as backend.proto spells it, for the message.
+    const char *rpc;
+    // Why audio.cpp cannot serve it, in terms of what upstream does and does
+    // not have. Stated so a caller can tell "not built yet" from "not possible".
+    const char *reason;
+};
+
+// The table behind the five refusals. Exposed whole so a test can assert every
+// entry rather than the one somebody remembered to cover, and so the reasons
+// are data in one place instead of string literals hand-copied into handlers.
+const std::vector<UnsupportedSurface> &unsupported_surfaces();
+const UnsupportedSurface &unsupported_surface(UnsupportedRpc rpc);
+
+// Message for an RPC this backend cannot serve at all, as opposed to one this
+// particular family cannot serve. `reason` states the upstream limitation.
+std::string unsupported_surface_message(const Capabilities &caps, const char *rpc,
+                                        const char *reason);
+
+// Same, for the no-model-loaded case. Says so explicitly rather than naming an
+// empty family, and says that loading one would not help, because the caller's
+// obvious next move otherwise is to load a model and try again.
+std::string unsupported_surface_message(const char *rpc, const char *reason);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/capability_routing_test.cpp

@@ -0,0 +1,600 @@
+// Unit tests for capability_routing. Standard library only. The harness
+// compiles this as a single translation unit, so the implementation is
+// included directly rather than linked.
+
+#include "capability_routing.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// Mirrors what supertonic advertises: TTS offline and streaming.
+static Capabilities supertonic() {
+    return Capabilities{"supertonic",
+                        {{Task::Tts, {Mode::Offline, Mode::Streaming}}}};
+}
+
+// Mirrors chatterbox: TTS, cloning and voice conversion, offline only.
+static Capabilities chatterbox() {
+    return Capabilities{"chatterbox",
+                        {{Task::Tts, {Mode::Offline}},
+                         {Task::VoiceCloning, {Mode::Offline}},
+                         {Task::VoiceConversion, {Mode::Offline}}}};
+}
+
+// Mirrors nemotron_asr: ASR offline and streaming.
+static Capabilities nemotron() {
+    return Capabilities{"nemotron_asr",
+                        {{Task::Asr, {Mode::Offline, Mode::Streaming}}}};
+}
+
+// Mirrors qwen3_asr: ASR offline only.
+static Capabilities qwen3_asr() {
+    return Capabilities{"qwen3_asr", {{Task::Asr, {Mode::Offline}}}};
+}
+
+// Mirrors qwen3_forced_aligner: alignment only.
+static Capabilities aligner() {
+    return Capabilities{"qwen3_forced_aligner",
+                        {{Task::Alignment, {Mode::Offline}}}};
+}
+
+// Mirrors htdemucs: separation only.
+static Capabilities htdemucs() {
+    return Capabilities{"htdemucs", {{Task::SourceSeparation, {Mode::Offline}}}};
+}
+
+static void test_plain_tts() {
+    const auto r = resolve_route(Rpc::Tts, RequestShape{}, chatterbox());
+    check(r.ok, "plain TTS routes");
+    check(r.task == Task::Tts, "plain TTS picks Tts, not VoiceCloning");
+    check(r.mode == Mode::Offline, "TTS runs offline");
+}
+
+static void test_tts_with_voice_reference_prefers_cloning() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    const auto r = resolve_route(Rpc::Tts, shape, chatterbox());
+    check(r.ok, "TTS with a voice reference routes");
+    check(r.task == Task::VoiceCloning, "voice reference prefers VoiceCloning");
+}
+
+// supertonic has no VoiceCloning: a voice reference must fall back to Tts
+// rather than failing the request.
+static void test_tts_voice_reference_falls_back_to_tts() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    const auto r = resolve_route(Rpc::Tts, shape, supertonic());
+    check(r.ok, "voice reference on a clone-less family still routes");
+    check(r.task == Task::Tts, "falls back to Tts");
+}
+
+static void test_tts_instructions_prefer_voice_design() {
+    RequestShape shape;
+    shape.has_instructions = true;
+    Capabilities caps{"qwen3_tts",
+                      {{Task::Tts, {Mode::Offline}},
+                       {Task::VoiceDesign, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::Tts, shape, caps);
+    check(r.ok, "TTS with instructions routes");
+    check(r.task == Task::VoiceDesign, "instructions prefer VoiceDesign");
+}
+
+// A voice reference is a stronger signal than free-form instructions: cloning
+// a specific voice is what the user asked for.
+static void test_voice_reference_beats_instructions() {
+    RequestShape shape;
+    shape.has_voice_reference = true;
+    shape.has_instructions = true;
+    Capabilities caps{"omnivoice",
+                      {{Task::Tts, {Mode::Offline}},
+                       {Task::VoiceCloning, {Mode::Offline}},
+                       {Task::VoiceDesign, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::Tts, shape, caps);
+    check(r.ok, "both signals present routes");
+    check(r.task == Task::VoiceCloning, "voice reference outranks instructions");
+}
+
+static void test_tts_stream_requires_streaming() {
+    const auto ok = resolve_route(Rpc::TtsStream, RequestShape{}, supertonic());
+    check(ok.ok, "streaming TTS routes on supertonic");
+    check(ok.mode == Mode::Streaming, "TTSStream runs in streaming mode");
+
+    const auto bad = resolve_route(Rpc::TtsStream, RequestShape{}, chatterbox());
+    check(!bad.ok, "streaming TTS is refused on an offline-only family");
+    check(bad.error.find("chatterbox") != std::string::npos,
+          "error names the family");
+    check(bad.error.find("tts/offline") != std::string::npos,
+          "error lists what the family does support");
+    check(bad.error.find("TTSStream") != std::string::npos,
+          "error names the RPC that was refused");
+    check(bad.error.find("tts/streaming") != std::string::npos,
+          "error lists the (task, mode) pairs that were tried");
+}
+
+static void test_transcription_stream_falls_back_to_offline() {
+    const auto streaming =
+        resolve_route(Rpc::AudioTranscriptionStream, RequestShape{}, nemotron());
+    check(streaming.ok && streaming.mode == Mode::Streaming,
+          "streaming ASR uses streaming mode when offered");
+
+    const auto offline =
+        resolve_route(Rpc::AudioTranscriptionStream, RequestShape{}, qwen3_asr());
+    check(offline.ok, "streaming ASR falls back on an offline-only family");
+    check(offline.mode == Mode::Offline, "fallback mode is offline");
+    check(offline.task == Task::Asr, "fallback task is still Asr");
+}
+
+// Task preference dominates mode preference: it is better to run the right
+// task in a fallback mode than the wrong task in the preferred mode. This is
+// the only RPC where the two orderings can disagree, because it is the only
+// one with more than one acceptable mode.
+static void test_task_preference_beats_mode_preference() {
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    Capabilities mixed{"mixed_asr_aligner",
+                       {{Task::Asr, {Mode::Offline}},
+                        {Task::Alignment, {Mode::Streaming}}}};
+    const auto r = resolve_route(Rpc::AudioTranscriptionStream, shape, mixed);
+    check(r.ok, "mixed family routes");
+    check(r.task == Task::Asr,
+          "the preferred task wins even in its fallback mode");
+    check(r.mode == Mode::Offline,
+          "the fallback mode is accepted to keep the preferred task");
+}
+
+// Live transcription is bidirectional and cannot be faked from an offline run.
+static void test_live_transcription_has_no_offline_fallback() {
+    const auto r =
+        resolve_route(Rpc::AudioTranscriptionLive, RequestShape{}, qwen3_asr());
+    check(!r.ok, "live transcription is refused on an offline-only family");
+}
+
+static void test_alignment_needs_prompt_text() {
+    const auto without =
+        resolve_route(Rpc::AudioTranscription, RequestShape{}, aligner());
+    check(!without.ok, "aligner without a transcript is refused");
+
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    const auto with = resolve_route(Rpc::AudioTranscription, shape, aligner());
+    check(with.ok, "aligner with a transcript routes");
+    check(with.task == Task::Alignment, "routes to Alignment");
+}
+
+// A real ASR family must not be hijacked to Alignment just because the caller
+// passed a prompt: `prompt` is also whisper-style decoding context.
+static void test_prompt_does_not_hijack_asr() {
+    RequestShape shape;
+    shape.has_prompt_text = true;
+    const auto r = resolve_route(Rpc::AudioTranscription, shape, nemotron());
+    check(r.ok, "ASR with a prompt routes");
+
+    // nemotron advertises Asr alone, so the assertion has to be made against a
+    // family that advertises both: otherwise "Asr is preferred" only restates
+    // that Asr is the only option, and reversing the preference order passes.
+    Capabilities both{"asr_with_aligner",
+                      {{Task::Asr, {Mode::Offline}},
+                       {Task::Alignment, {Mode::Offline}}}};
+    const auto pref = resolve_route(Rpc::AudioTranscription, shape, both);
+    check(pref.ok, "a family offering both routes");
+    check(pref.task == Task::Asr, "Asr is preferred over Alignment");
+}
+
+static void test_audio_transform_prefers_separation() {
+    const auto sep =
+        resolve_route(Rpc::AudioTransform, RequestShape{}, htdemucs());
+    check(sep.ok && sep.task == Task::SourceSeparation, "separation routes");
+
+    // htdemucs advertises separation alone, so the check above cannot fail on
+    // ordering. This family advertises both, which is what pins the preference.
+    Capabilities sep_and_vc{"sep_and_vc",
+                            {{Task::SourceSeparation, {Mode::Offline}},
+                             {Task::VoiceConversion, {Mode::Offline}}}};
+    const auto pref =
+        resolve_route(Rpc::AudioTransform, RequestShape{}, sep_and_vc);
+    check(pref.ok && pref.task == Task::SourceSeparation,
+          "separation is preferred over voice conversion");
+
+    Capabilities miocodec{"miocodec",
+                          {{Task::VoiceConversion, {Mode::Offline}},
+                           {Task::SpeechToSpeech, {Mode::Offline}}}};
+    const auto vc = resolve_route(Rpc::AudioTransform, RequestShape{}, miocodec);
+    check(vc.ok && vc.task == Task::VoiceConversion,
+          "voice conversion is preferred over speech-to-speech");
+}
+
+static void test_pinned_task_overrides_routing() {
+    RequestShape shape;
+    shape.pinned_task = "s2s";
+    Capabilities miocodec{"miocodec",
+                          {{Task::VoiceConversion, {Mode::Offline}},
+                           {Task::SpeechToSpeech, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::AudioTransform, shape, miocodec);
+    check(r.ok && r.task == Task::SpeechToSpeech, "pinned task wins");
+
+    RequestShape bad;
+    bad.pinned_task = "not-a-task";
+    const auto e = resolve_route(Rpc::AudioTransform, bad, miocodec);
+    check(!e.ok, "an unknown pinned task is an error");
+    check(e.error.find("not-a-task") != std::string::npos,
+          "error names the bad task");
+
+    // A pinned task the family does not offer must fail, not silently reroute.
+    RequestShape unsupported;
+    unsupported.pinned_task = "sep";
+    const auto u = resolve_route(Rpc::AudioTransform, unsupported, miocodec);
+    check(!u.ok, "a pinned but unsupported task is refused");
+}
+
+// A pin lives on the MODEL, and every one of the nine handlers copies it into
+// the shape, so a pin set for one RPC arrives at all of them. It used to
+// replace the candidate list wholesale, which turned the other eight into wrong
+// 200s rather than errors: nemotron pinned to asr made Vad answer with zero
+// segments after a full ASR decode, and silero_vad pinned to vad made
+// AudioTranscription answer with empty text and four segments whose spans were
+// VAD segments, which the srt/vtt/lrc writers rendered as timed EMPTY cues.
+static void test_pin_must_be_admissible_for_the_rpc() {
+    Capabilities nemotron_asr{"nemotron_asr",
+                              {{Task::Asr, {Mode::Offline, Mode::Streaming}},
+                               {Task::Vad, {Mode::Offline}}}};
+
+    // The pin is legitimate on the RPC it was meant for.
+    RequestShape asr_pin;
+    asr_pin.pinned_task = "asr";
+    const auto transcription =
+        resolve_route(Rpc::AudioTranscription, asr_pin, nemotron_asr);
+    check(transcription.ok && transcription.task == Task::Asr,
+          "an admissible pin is still honoured exactly");
+
+    // ...and refused on one that never routes to it, EVEN THOUGH the family
+    // advertises the pinned task. That is the whole point: family support is
+    // not the question, RPC admissibility is.
+    const auto vad = resolve_route(Rpc::Vad, asr_pin, nemotron_asr);
+    check(!vad.ok, "an inadmissible pin is refused rather than served");
+    check(vad.error.find("asr") != std::string::npos,
+          "the refusal names the pinned task");
+    check(vad.error.find(rpc_name(Rpc::Vad)) != std::string::npos,
+          "the refusal names the RPC that cannot serve it");
+
+    Capabilities silero{"silero_vad", {{Task::Vad, {Mode::Offline}}}};
+    RequestShape vad_pin;
+    vad_pin.pinned_task = "vad";
+    const auto vad_ok = resolve_route(Rpc::Vad, vad_pin, silero);
+    check(vad_ok.ok && vad_ok.task == Task::Vad, "vad is admissible on Vad");
+    const auto transcribe_vad =
+        resolve_route(Rpc::AudioTranscription, vad_pin, silero);
+    check(!transcribe_vad.ok,
+          "a vad pin cannot make a transcription request return empty cues");
+
+    // Every pin a shipped configuration could sensibly set stays reachable on
+    // the RPC that serves it. This is the list the fix was checked against.
+    struct AdmissibleCase {
+        Rpc rpc;
+        const char *task;
+    };
+    const AdmissibleCase kAdmissible[] = {
+        {Rpc::AudioTransform, "svc"},   {Rpc::AudioTransform, "sep"},
+        {Rpc::AudioTransform, "vc"},    {Rpc::AudioTransform, "s2s"},
+        {Rpc::Tts, "tts"},              {Rpc::Tts, "clon"},
+        {Rpc::Tts, "vdes"},             {Rpc::TtsStream, "tts"},
+        {Rpc::AudioTranscription, "asr"},
+        {Rpc::AudioTranscription, "align"},
+        {Rpc::AudioTranscriptionStream, "asr"},
+        {Rpc::AudioTranscriptionLive, "asr"},
+        {Rpc::Vad, "vad"},              {Rpc::Diarize, "diar"},
+        {Rpc::SoundGeneration, "gen"},
+    };
+    for (const auto &entry : kAdmissible) {
+        Task task = Task::Tts;
+        check(parse_task_name(entry.task, task),
+              std::string("known task name: ") + entry.task);
+        // A family that advertises the pinned task offline and nothing else, so
+        // the ONLY thing that can refuse the route is the admissibility check.
+        Capabilities only{"probe",
+                          {{task, {Mode::Offline, Mode::Streaming}}}};
+        RequestShape pin;
+        pin.pinned_task = entry.task;
+        const auto route = resolve_route(entry.rpc, pin, only);
+        check(route.ok && route.task == task,
+              std::string("pin '") + entry.task + "' stays admissible on " +
+                  rpc_name(entry.rpc));
+    }
+
+    // And the pins that must NOT cross over, one per RPC pair that was
+    // observed producing a wrong 200.
+    const AdmissibleCase kInadmissible[] = {
+        {Rpc::Vad, "asr"},          {Rpc::Diarize, "asr"},
+        {Rpc::AudioTranscription, "vad"}, {Rpc::AudioTranscription, "diar"},
+        {Rpc::Tts, "asr"},          {Rpc::Vad, "tts"},
+        {Rpc::SoundGeneration, "tts"},    {Rpc::AudioTransform, "asr"},
+    };
+    for (const auto &entry : kInadmissible) {
+        Task task = Task::Tts;
+        check(parse_task_name(entry.task, task),
+              std::string("known task name: ") + entry.task);
+        Capabilities only{"probe",
+                          {{task, {Mode::Offline, Mode::Streaming}}}};
+        RequestShape pin;
+        pin.pinned_task = entry.task;
+        const auto route = resolve_route(entry.rpc, pin, only);
+        check(!route.ok,
+              std::string("pin '") + entry.task + "' is refused on " +
+                  rpc_name(entry.rpc));
+    }
+}
+
+static void test_vad_and_diarize() {
+    Capabilities silero{"silero_vad", {{Task::Vad, {Mode::Offline, Mode::Streaming}}}};
+    const auto v = resolve_route(Rpc::Vad, RequestShape{}, silero);
+    check(v.ok && v.task == Task::Vad && v.mode == Mode::Offline, "VAD routes offline");
+
+    const auto d = resolve_route(Rpc::Diarize, RequestShape{}, silero);
+    check(!d.ok, "diarization is refused on a VAD-only family");
+
+    Capabilities sortformer{"sortformer_diar", {{Task::Diarization, {Mode::Offline}}}};
+    const auto ok = resolve_route(Rpc::Diarize, RequestShape{}, sortformer);
+    check(ok.ok && ok.task == Task::Diarization, "diarization routes");
+}
+
+static void test_sound_generation() {
+    Capabilities stable{"stable_audio", {{Task::AudioGeneration, {Mode::Offline}}}};
+    const auto r = resolve_route(Rpc::SoundGeneration, RequestShape{}, stable);
+    check(r.ok && r.task == Task::AudioGeneration, "sound generation routes");
+}
+
+static void test_names_round_trip() {
+    const Task all[] = {Task::Vad, Task::Asr, Task::Diarization,
+                        Task::SourceSeparation, Task::AudioGeneration, Task::Tts,
+                        Task::VoiceCloning, Task::VoiceConversion,
+                        Task::SpeechToSpeech, Task::Alignment, Task::VoiceDesign,
+                        Task::SpeakerRecognition, Task::Svc};
+    for (const Task t : all) {
+        Task parsed = Task::Vad;
+        const bool ok = parse_task_name(task_name(t), parsed);
+        check(ok && parsed == t,
+              std::string("task name round-trips: ") + task_name(t));
+    }
+    check(std::string(mode_name(Mode::Offline)) == "offline", "offline name");
+    check(std::string(mode_name(Mode::Streaming)) == "streaming", "streaming name");
+
+    // The emitted name must be the one audio.cpp itself prints and parses
+    // (framework/runtime/session.cpp), because `task:` is user-facing: a name
+    // copied out of audio.cpp has to be accepted here, and a name pinned here
+    // has to survive conversion at the engine boundary.
+    check(std::string(task_name(Task::SpeakerRecognition)) == "spk",
+          "speaker recognition emits upstream's name 'spk'");
+
+    Task pinned = Task::Vad;
+    check(parse_task_name("spk", pinned) && pinned == Task::SpeakerRecognition,
+          "'spk' parses to SpeakerRecognition");
+
+    // Accepted as a legacy alias so configs written against the earlier name
+    // keep working, but never emitted.
+    Task alias = Task::Vad;
+    check(parse_task_name("spkrec", alias) && alias == Task::SpeakerRecognition,
+          "'spkrec' is still accepted as an alias");
+}
+
+static void test_describe_capabilities() {
+    const std::string described = describe_capabilities(nemotron());
+    check(described.find("asr/offline") != std::string::npos,
+          "description lists asr/offline");
+    check(described.find("asr/streaming") != std::string::npos,
+          "description lists asr/streaming");
+}
+
+static void test_empty_capabilities() {
+    const auto r = resolve_route(Rpc::Tts, RequestShape{}, Capabilities{"mystery", {}});
+    check(!r.ok, "a family advertising nothing is refused");
+    check(r.error.find("mystery") != std::string::npos, "error names the family");
+}
+
+// The table is indexed by UnsupportedRpc's underlying value, so a reordering of
+// either list silently pairs an RPC with another's reason. Nothing else would
+// catch that: both sides still compile and every message still reads plausibly.
+static void test_unsupported_surface_table_matches_the_enum() {
+    check(unsupported_surfaces().size() == 5,
+          "all five unsupported surfaces are tabulated");
+    check(std::string(unsupported_surface(UnsupportedRpc::AudioEncode).rpc) ==
+              "AudioEncode",
+          "UnsupportedRpc::AudioEncode indexes AudioEncode");
+    check(std::string(unsupported_surface(UnsupportedRpc::AudioDecode).rpc) ==
+              "AudioDecode",
+          "UnsupportedRpc::AudioDecode indexes AudioDecode");
+    check(std::string(
+              unsupported_surface(UnsupportedRpc::AudioTransformStream).rpc) ==
+              "AudioTransformStream",
+          "UnsupportedRpc::AudioTransformStream indexes AudioTransformStream");
+    check(std::string(
+              unsupported_surface(UnsupportedRpc::AudioToAudioStream).rpc) ==
+              "AudioToAudioStream",
+          "UnsupportedRpc::AudioToAudioStream indexes AudioToAudioStream");
+    check(std::string(unsupported_surface(UnsupportedRpc::VoiceEmbed).rpc) ==
+              "VoiceEmbed",
+          "UnsupportedRpc::VoiceEmbed indexes VoiceEmbed");
+
+    // Two entries may share a reason (the codec pair does), but two entries
+    // naming the same RPC would mean one of the five is unreachable.
+    for (size_t i = 0; i < unsupported_surfaces().size(); ++i) {
+        for (size_t j = i + 1; j < unsupported_surfaces().size(); ++j) {
+            check(std::string(unsupported_surfaces()[i].rpc) !=
+                      unsupported_surfaces()[j].rpc,
+                  std::string("no duplicate RPC name at ") + std::to_string(i) +
+                      "/" + std::to_string(j));
+        }
+    }
+}
+
+// There is no out-of-range test for unsupported_surface(). It switches over the
+// enumerators with no default label, so a sixth UnsupportedRpc without a case is
+// a -Wswitch diagnostic at build time and cannot reach a run-time check at all.
+
+// Every entry, not just the one somebody remembered to cover. A refusal that
+// drops the family, the RPC or the reason is a refusal the caller cannot act
+// on, which is the entire point of this surface existing.
+static void test_every_unsupported_surface_message_is_diagnosable() {
+    for (const auto &surface : unsupported_surfaces()) {
+        const std::string label = std::string(" [") + surface.rpc + "]";
+        const std::string loaded =
+            unsupported_surface_message(nemotron(), surface.rpc, surface.reason);
+
+        check(loaded.find(surface.rpc) != std::string::npos,
+              "message names the RPC" + label);
+        check(std::string(surface.reason).size() > 20 &&
+                  loaded.find(surface.reason) != std::string::npos,
+              "message gives a substantive upstream reason" + label);
+        check(loaded.find("nemotron_asr") != std::string::npos,
+              "message names the loaded family" + label);
+        check(loaded.find("asr/offline") != std::string::npos &&
+                  loaded.find("asr/streaming") != std::string::npos,
+              "message lists what the family does support" + label);
+
+        // The no-model form keeps the two facts that do not depend on a model
+        // and drops only the one that does, so the caller still learns why.
+        const std::string unloaded =
+            unsupported_surface_message(surface.rpc, surface.reason);
+        check(unloaded.find(surface.rpc) != std::string::npos,
+              "no-model message names the RPC" + label);
+        check(unloaded.find(surface.reason) != std::string::npos,
+              "no-model message gives the upstream reason" + label);
+        check(unloaded.find("nemotron_asr") == std::string::npos,
+              "no-model message names no family" + label);
+        // Without this the caller's obvious next move is to load a model and
+        // retry, which cannot work: the refusal is a property of the engine.
+        check(unloaded.find("would not change this answer") != std::string::npos,
+              "no-model message says loading a model would not help" + label);
+    }
+}
+
+// The reasons are the load-bearing half of this feature and each was checked
+// against the pinned upstream checkout. Pinning the distinguishing phrase here
+// means a later edit that guts one into a generic "not supported" fails rather
+// than passes quietly.
+static void test_unsupported_reasons_name_the_upstream_limitation() {
+    const auto &encode = unsupported_surface(UnsupportedRpc::AudioEncode);
+    const auto &decode = unsupported_surface(UnsupportedRpc::AudioDecode);
+    check(std::string(encode.reason).find("VoiceTaskKind") != std::string::npos &&
+              std::string(encode.reason).find("codec") != std::string::npos,
+          "the AudioEncode reason names the missing VoiceTaskKind entry");
+    // miocodec is the family a reader will reach for first, because upstream's
+    // README tags it Codec. Naming it and its actual advertised tasks is what
+    // stops the next person re-deriving the same dead end.
+    check(std::string(encode.reason).find("miocodec") != std::string::npos,
+          "the AudioEncode reason disposes of miocodec's README Codec tag");
+    check(std::string(encode.reason) == decode.reason,
+          "AudioEncode and AudioDecode refuse for the same reason");
+
+    const auto &transform =
+        unsupported_surface(UnsupportedRpc::AudioTransformStream);
+    // The reason must be scoped to the tasks AudioTransform routes to. The
+    // broader claim, "upstream streams tts and asr only", is FALSE: silero_vad
+    // advertises vad with RunMode::Streaming. A refusal resting on a false
+    // premise is worse than a bare UNIMPLEMENTED, because it will be believed.
+    check(std::string(transform.reason).find("sep, vc, svc, s2s") !=
+              std::string::npos,
+          "the AudioTransformStream reason is scoped to the routed tasks");
+    check(std::string(transform.reason).find("tts, asr and vad") !=
+              std::string::npos,
+          "the AudioTransformStream reason counts vad among the streaming tasks");
+    check(std::string(transform.reason).find("tts and asr only") ==
+              std::string::npos,
+          "the AudioTransformStream reason does not repeat the refuted claim");
+    // An absence is not an impossibility. A sep family could be buffered and
+    // emitted as a stream, so the reason has to say this backend declines to
+    // rather than cannot, or it overreaches on a true premise.
+    check(std::string(transform.reason).find("buffered offline call in disguise") !=
+              std::string::npos,
+          "the AudioTransformStream reason does not overclaim impossibility");
+
+    const auto &s2s = unsupported_surface(UnsupportedRpc::AudioToAudioStream);
+    check(std::string(s2s.reason).find("Realtime") != std::string::npos &&
+              std::string(s2s.reason).find("clip-to-clip") != std::string::npos,
+          "the AudioToAudioStream reason contrasts the two contracts");
+    // Naming both s2s families, and what each of them actually does, makes the
+    // claim checkable. It must NOT say s2s is voice conversion full stop: that
+    // is true of miocodec and false of vevo2, whose s2s route is `editing` and
+    // rewrites the spoken content against a target voice.
+    check(std::string(s2s.reason).find("miocodec (voice conversion)") !=
+                  std::string::npos &&
+              std::string(s2s.reason).find("vevo2 (speech editing)") !=
+                  std::string::npos,
+          "the AudioToAudioStream reason names each s2s family's actual task");
+    check(std::string(s2s.reason).find("s2s is offline voice conversion") ==
+              std::string::npos,
+          "the AudioToAudioStream reason does not miscast vevo2 as conversion");
+
+    const auto &embed = unsupported_surface(UnsupportedRpc::VoiceEmbed);
+    check(std::string(embed.reason).find("spk") != std::string::npos,
+          "the VoiceEmbed reason names the task no family advertises");
+    // spk IS a VoiceTaskKind upstream; what is missing is any family that
+    // advertises it. Saying the kind does not exist would be false, and would
+    // send a reader looking in the wrong place.
+    check(std::string(embed.reason).find("no audio.cpp family") !=
+              std::string::npos,
+          "the VoiceEmbed reason blames the families, not the enum");
+    // The speaker encoders DO exist upstream, as conditioning modules inside
+    // TTS and VC families. Not saying so invites "but audio.cpp ships TitaNet".
+    check(std::string(embed.reason).find("TitaNet") != std::string::npos,
+          "the VoiceEmbed reason disposes of the internal speaker encoders");
+    Task parsed = Task::Vad;
+    check(parse_task_name("spk", parsed) && parsed == Task::SpeakerRecognition,
+          "spk is a real task kind, so the reason must not claim otherwise");
+}
+
+// A family advertising nothing still gets a message that reads, rather than one
+// trailing off after "supports: ".
+static void test_unsupported_surface_message_with_empty_capabilities() {
+    const auto &embed = unsupported_surface(UnsupportedRpc::VoiceEmbed);
+    const std::string message = unsupported_surface_message(
+        Capabilities{"mystery", {}}, embed.rpc, embed.reason);
+    check(message.find("mystery") != std::string::npos,
+          "empty-capability message still names the family");
+    check(message.find("supports: nothing") != std::string::npos,
+          "empty-capability message says the family supports nothing");
+}
+
+int main() {
+    test_plain_tts();
+    test_tts_with_voice_reference_prefers_cloning();
+    test_tts_voice_reference_falls_back_to_tts();
+    test_tts_instructions_prefer_voice_design();
+    test_voice_reference_beats_instructions();
+    test_tts_stream_requires_streaming();
+    test_transcription_stream_falls_back_to_offline();
+    test_task_preference_beats_mode_preference();
+    test_live_transcription_has_no_offline_fallback();
+    test_alignment_needs_prompt_text();
+    test_prompt_does_not_hijack_asr();
+    test_audio_transform_prefers_separation();
+    test_pinned_task_overrides_routing();
+    test_pin_must_be_admissible_for_the_rpc();
+    test_vad_and_diarize();
+    test_sound_generation();
+    test_names_round_trip();
+    test_describe_capabilities();
+    test_empty_capabilities();
+    test_unsupported_surface_table_matches_the_enum();
+    test_every_unsupported_surface_message_is_diagnosable();
+    test_unsupported_reasons_name_the_upstream_limitation();
+    test_unsupported_surface_message_with_empty_capabilities();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all capability_routing checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/family_gate.cpp

@@ -0,0 +1,163 @@
+#include "family_gate.h"
+
+#include <cctype>
+#include <cstddef>
+
+namespace audiocpp_backend {
+namespace {
+
+// PRECONDITION: `suffix` must already be lowercase. Both sides are folded, so
+// this reads as symmetric, but only `value` can carry case in practice and a
+// caller passing ".GGUF" would still work today for that reason alone. Do not
+// rely on it: the fold on the suffix side is the only thing standing between
+// this and a helper that answers false for every input, and it is not covered
+// by any test, because with a lowercase suffix no input can distinguish it.
+bool ends_with_ci(const std::string &value, const std::string &suffix) {
+    if (value.size() <= suffix.size()) {
+        return false; // a bare ".gguf" is an extension, not a model file
+    }
+    const size_t offset = value.size() - suffix.size();
+    for (size_t i = 0; i < suffix.size(); ++i) {
+        const auto lhs = static_cast<unsigned char>(value[offset + i]);
+        const auto rhs = static_cast<unsigned char>(suffix[i]);
+        if (std::tolower(lhs) != std::tolower(rhs)) {
+            return false;
+        }
+    }
+    return true;
+}
+
+} // namespace
+
+bool path_looks_like_gguf(const std::string &path) {
+    return ends_with_ci(path, ".gguf");
+}
+
+FamilyDecision decide_family(bool path_is_gguf, const std::string &embedded_family,
+                             const std::string &configured_family) {
+    FamilyDecision decision;
+
+    if (!configured_family.empty()) {
+        decision.ok = true;
+        decision.family = configured_family;
+        return decision;
+    }
+
+    if (path_is_gguf) {
+        if (!embedded_family.empty()) {
+            decision.ok = true;
+            decision.family = embedded_family;
+            return decision;
+        }
+        decision.error =
+            "audio-cpp: this GGUF carries no 'audiocpp.model_spec.family' "
+            "metadata key, so it is not an audio.cpp model. Convert it with "
+            "audiocpp_gguf, or name the family explicitly with the model option "
+            "'family:<name>'";
+        return decision;
+    }
+
+    decision.error =
+        "audio-cpp: a model path that is not a standalone audio.cpp GGUF needs "
+        "an explicit 'family:<name>' model option, because the audio.cpp family "
+        "cannot be inferred from a safetensors or package directory";
+    return decision;
+}
+
+namespace {
+
+// Families that ABORT THE PROCESS on a weight dtype they cannot handle, and the
+// dtypes they can. See the header for why this is a list of crashes rather than
+// a list of preferences.
+//
+// supertonic: upstream's docs/gguf.md:90 records its 16-bit GGUF column as
+// "---", i.e. NOT TESTED, and its q8_0 as "No (unsupported weight dtype)". Only
+// the `orig` package is marked Pass, and its 698 weight tensors are f32 while
+// its 72 index and shape constants are i64. The f16 abort is a LOCAL
+// OBSERVATION rather than an upstream claim, and it is attributed rather than
+// assumed: it is identical through the unary TTS RPC and through TTSStream, so
+// it is the packaging and not the streaming path. q8_0 was never run here and is
+// refused on upstream's "unsupported weight dtype" alone, which is the weaker of
+// the two claims. See the header for why keeping them apart matters.
+//
+// TO REMOVE AN ENTRY: bump AUDIO_CPP_VERSION past a fix, load a package in the
+// refused dtype, and synthesise. If audio comes out, delete the entry. No test
+// can do that for you, which is exactly why it is written here: the test beside
+// this file pins WHAT the table says, not whether upstream has moved on. Do not
+// widen an entry without running that, because what it prevents is a process
+// death rather than a wrong answer.
+struct DtypeAllowList {
+    // NULL TERMINATED, and the terminator occupies one of these slots: both
+    // loops below stop at the first nullptr and have no other bound, so an entry
+    // that named three dtypes would leave them reading past the end of the
+    // array. That is undefined behaviour rather than a wrong answer, and it is
+    // one keystroke away from any edit that widens an entry, so the terminator
+    // is asserted at compile time below rather than trusted.
+    static constexpr std::size_t kSlots = 3;
+
+    const char *family;
+    const char *allowed[kSlots];
+};
+
+constexpr DtypeAllowList kDtypeAllowLists[] = {
+    {"supertonic", {"f32", "i64", nullptr}},
+};
+
+constexpr bool allow_lists_are_terminated() {
+    for (const auto &entry : kDtypeAllowLists) {
+        if (entry.allowed[DtypeAllowList::kSlots - 1] != nullptr) {
+            return false;
+        }
+    }
+    return true;
+}
+
+static_assert(allow_lists_are_terminated(),
+              "every DtypeAllowList must leave its last slot null: the lookups "
+              "below stop at the first nullptr and would otherwise read past "
+              "the end of the array");
+
+const DtypeAllowList *find_allow_list(const std::string &family) {
+    for (const auto &entry : kDtypeAllowLists) {
+        if (family == entry.family) {
+            return &entry;
+        }
+    }
+    return nullptr;
+}
+
+} // namespace
+
+bool family_has_weight_dtype_allow_list(const std::string &family) {
+    return find_allow_list(family) != nullptr;
+}
+
+bool weight_dtype_is_supported(const std::string &family, const std::string &dtype) {
+    const DtypeAllowList *list = find_allow_list(family);
+    if (list == nullptr) {
+        return true;
+    }
+    for (const char *const *name = list->allowed; *name != nullptr; ++name) {
+        if (dtype == *name) {
+            return true;
+        }
+    }
+    return false;
+}
+
+std::string supported_weight_dtypes(const std::string &family) {
+    const DtypeAllowList *list = find_allow_list(family);
+    if (list == nullptr) {
+        return {};
+    }
+    std::string out;
+    for (const char *const *name = list->allowed; *name != nullptr; ++name) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += *name;
+    }
+    return out;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/family_gate.h

@@ -0,0 +1,86 @@
+#pragma once
+
+// Decides which audio.cpp family a model path belongs to, and refuses paths
+// this backend must not claim. Standard library only.
+//
+// This is the guard against issue #9287. A model config with no explicit
+// backend makes LocalAI probe every installed backend and bind to the first
+// Load that succeeds, so accepting an arbitrary GGUF here would capture
+// unrelated LLMs. audio.cpp GGUFs carry an audiocpp.model_spec.family metadata
+// key; llama.cpp GGUFs do not.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+// True when the path ends in ".gguf", case insensitively, and has a stem.
+bool path_looks_like_gguf(const std::string &path);
+
+struct FamilyDecision {
+    bool ok = false;
+    std::string family;
+    // Set when ok is false. Suitable verbatim as an INVALID_ARGUMENT message.
+    std::string error;
+};
+
+// Precedence:
+//   1. an explicit `family:` option, so a user can override wrong metadata;
+//   2. for a GGUF, the family embedded in audiocpp.model_spec.family;
+//   3. otherwise refuse.
+// A directory path never consults embedded metadata: there is no single GGUF
+// to read it from.
+FamilyDecision decide_family(bool path_is_gguf, const std::string &embedded_family,
+                             const std::string &configured_family);
+
+// True when `family` can run weights stored as `dtype`, where dtype is the
+// string a TensorMetadata carries ("f32", "f16", "q8_0", "i64", ...).
+//
+// This is a LIST OF FAMILIES THAT CRASH THE PROCESS, not a list of families that
+// perform badly. It exists because the failure is not an exception: loading the
+// supertonic f16 GGUF package reaches ggml_concat with one f16 operand and one
+// f32 one, GGML_ASSERT(a->type == b->type) fails (external/ggml/src/ggml.c:2595)
+// and ggml_abort takes the backend down with SIGABRT on the FIRST request.
+// Nothing upstream of the load can catch that, so an operator sees a model that
+// loaded successfully and a backend that dies on every request with no status
+// and no message.
+//
+// EVIDENCE, per dtype, because the two are not equally attested:
+//   - f16 was OBSERVED to abort here, identically through the unary TTS RPC and
+//     through TTSStream, so it is the packaging and not the streaming path.
+//     Upstream's docs/gguf.md:90 has supertonic's 16-bit column as "---", which
+//     its own legend (:53) defines as not tested, so upstream neither confirms
+//     nor contradicts it.
+//   - q8_0 was NOT run here. Upstream records it as "No (unsupported weight
+//     dtype)" in the same row, which is a weaker claim than the f16 abort: it
+//     says the format is unusable, not that it takes the process down.
+// Both are refused, because the allow list is what the family CAN run (f32 for
+// weights, i64 for the shape and index constants) rather than a list of the
+// dtypes that fail, and a format upstream calls unusable has no business being
+// loaded either way.
+//
+// A family with no entry is unrestricted, which is every family but one.
+//
+// Split out of loaded_model.cpp, where the caller lives, so that the policy is
+// stdlib-only and can be held by a test: the caller needs a real GGUF on disk
+// and an engine, and neither is available to a unit test. What the test pins is
+// that the table says what it is meant to say, so widening it is a deliberate
+// act rather than a typo. It CANNOT pin the removal criterion, which is
+// "upstream fixed it": no test can know that without downloading the package and
+// synthesising, so that step stays a documented manual one at the table itself.
+bool weight_dtype_is_supported(const std::string &family, const std::string &dtype);
+
+// True when `family` has an entry in the table at all, which is the question a
+// caller deciding whether to OPEN THE FILE has to ask. Distinct from
+// "supported_weight_dtypes(family) is empty": that string is also empty for an
+// entry with an empty allow list, and such an entry means "this family can run
+// nothing", which weight_dtype_is_supported already answers by refusing every
+// dtype. Deciding from the string would skip the check on precisely the entry
+// that most needs it.
+bool family_has_weight_dtype_allow_list(const std::string &family);
+
+// The dtypes `family` is restricted to, as "f32, i64", or empty when it is not
+// restricted at all. For the refusal message, so the operator is told what to
+// look for rather than only what is wrong.
+std::string supported_weight_dtypes(const std::string &family);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/family_gate_test.cpp

@@ -0,0 +1,179 @@
+// Unit tests for family_gate. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+//
+// This unit is the guard against issue #9287: when a model config has no
+// explicit backend, LocalAI probes every installed backend and binds to the
+// first Load that succeeds. Accepting an arbitrary GGUF here would capture
+// unrelated LLMs.
+
+#include "family_gate.cpp"
+
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+using namespace audiocpp_backend;
+
+static void test_gguf_suffix_detection() {
+    check(path_looks_like_gguf("/models/chatterbox-q8_0.gguf"), "plain .gguf");
+    check(path_looks_like_gguf("/models/CHATTERBOX.GGUF"), "uppercase .GGUF");
+    check(path_looks_like_gguf("/models/x.GgUf"), "mixed case .GgUf");
+    check(path_looks_like_gguf("a.gguf"), "a one character stem is still a stem");
+    check(!path_looks_like_gguf("/models/chatterbox"), "extensionless directory");
+    check(!path_looks_like_gguf("/models/model.safetensors"), "safetensors");
+    check(!path_looks_like_gguf("/models/gguf"), "a name that is merely 'gguf'");
+    check(!path_looks_like_gguf("/models/GGUF"), "an uppercase name that is merely 'GGUF'");
+    check(!path_looks_like_gguf(""), "empty path");
+    check(!path_looks_like_gguf(".gguf"), "a bare extension is not a model file");
+    // The suffix has to be at the end. A prefix or infix match would let
+    // ".gguf.tmp" download artefacts and ".ggufx" siblings through.
+    check(!path_looks_like_gguf("/models/model.gguf.tmp"), ".gguf in the middle");
+    check(!path_looks_like_gguf("/models/model.ggufx"), "a longer extension");
+    // Every character of the suffix has to match, including the last one.
+    check(!path_looks_like_gguf("/models/model.ggug"), "a near miss in the final character");
+    check(!path_looks_like_gguf("/models/model_gguf"), "a near miss in the first character");
+    check(!path_looks_like_gguf("/models/.gguf-notes"), "a leading .gguf");
+}
+
+static void test_explicit_family_always_wins() {
+    // Explicit configuration beats metadata, so a user can force a family when
+    // upstream metadata is wrong or absent.
+    const auto gguf = decide_family(true, "chatterbox", "omnivoice");
+    check(gguf.ok && gguf.family == "omnivoice", "explicit family overrides GGUF metadata");
+    check(gguf.error.empty(), "an accepted decision carries no error text");
+
+    const auto dir = decide_family(false, "", "qwen3_tts");
+    check(dir.ok && dir.family == "qwen3_tts", "explicit family satisfies a directory path");
+
+    // A GGUF with no embedded spec is still loadable when the user names the
+    // family: the option is an override, not a tie-break that needs metadata to
+    // break against.
+    const auto bare = decide_family(true, "", "supertonic");
+    check(bare.ok && bare.family == "supertonic",
+          "explicit family rescues a GGUF that carries no spec");
+}
+
+static void test_gguf_metadata_supplies_the_family() {
+    const auto d = decide_family(true, "nemotron_asr", "");
+    check(d.ok, "an audio.cpp GGUF loads with no family option");
+    check(d.family == "nemotron_asr", "family comes from the embedded spec");
+    check(d.error.empty(), "an accepted GGUF carries no error text");
+}
+
+// THE GATE. A llama.cpp GGUF has no audiocpp.model_spec.family key.
+static void test_foreign_gguf_is_refused() {
+    const auto d = decide_family(true, "", "");
+    check(!d.ok, "a GGUF with no audio.cpp spec is refused");
+    check(d.family.empty(), "no family is guessed");
+    check(!d.error.empty(), "a refusal always says why");
+    check(d.error.find("audiocpp.model_spec.family") != std::string::npos,
+          "error names the missing metadata key so the cause is diagnosable");
+    check(d.error.find("family:") != std::string::npos,
+          "error names the option that would override it");
+}
+
+static void test_directory_without_family_is_refused() {
+    const auto d = decide_family(false, "", "");
+    check(!d.ok, "a non-GGUF path with no family option is refused");
+    check(d.family.empty(), "a refused directory guesses no family");
+    check(d.error.find("family:") != std::string::npos,
+          "error names the required option");
+}
+
+// A directory path never consults embedded metadata, because there is no single
+// GGUF to read it from.
+static void test_directory_ignores_embedded_family() {
+    const auto d = decide_family(false, "chatterbox", "");
+    check(!d.ok, "a directory is refused even when an embedded family is supplied");
+    check(d.family.empty(), "a refused directory does not adopt the embedded family");
+    // If the GGUF branch ever leaked into the directory branch this message
+    // would start blaming a metadata key that a directory has no place to carry.
+    check(d.error.find("audiocpp.model_spec.family") == std::string::npos,
+          "a directory refusal does not blame GGUF metadata it could not have");
+}
+
+// Pins the weight-dtype allow list. Not a style preference: an entry here is a
+// family that ABORTS THE PROCESS on the first request when handed the wrong
+// dtype, so the model loads and then every request kills the backend with no
+// status and no message.
+//
+// What this test can and cannot do, stated so the next reader does not expect
+// more of it: it pins WHAT THE TABLE SAYS, so widening an entry is a deliberate
+// act rather than a typo, and it pins that an unlisted family is unrestricted.
+// It CANNOT pin the removal criterion, which is "upstream fixed it": knowing
+// that needs the package downloaded and a synthesis run, so it stays a manual
+// step documented at the table in family_gate.cpp.
+static void test_weight_dtype_allow_list() {
+    // The entry that exists, and the exact reason it exists.
+    check(!weight_dtype_is_supported("supertonic", "f16"),
+          "supertonic refuses f16, the package that aborts the process");
+    check(!weight_dtype_is_supported("supertonic", "q8_0"),
+          "supertonic refuses q8_0, which upstream records as unsupported");
+    check(!weight_dtype_is_supported("supertonic", "bf16"),
+          "supertonic refuses bf16, which is untested rather than known good");
+    check(weight_dtype_is_supported("supertonic", "f32"),
+          "supertonic accepts f32, which is what the orig package stores");
+    check(weight_dtype_is_supported("supertonic", "i64"),
+          "supertonic accepts i64: the orig package carries 72 such tensors and "
+          "refusing them would refuse the artifact that works");
+
+    // Every other family is unrestricted, and must stay that way: this guard is
+    // for process death, not for quality.
+    check(weight_dtype_is_supported("nemotron_asr", "q8_0"),
+          "an unlisted family is not restricted");
+    check(weight_dtype_is_supported("citrinet_asr", "f16"),
+          "an unlisted family is not restricted by another family's entry");
+    check(weight_dtype_is_supported("", "anything"),
+          "an empty family name is not restricted");
+
+    // The message the operator reads has to name the remedy, so the refusal is
+    // actionable rather than only correct.
+    check_eq(supported_weight_dtypes("supertonic"), "f32, i64",
+             "the refusal can name what to look for");
+    check_eq(supported_weight_dtypes("nemotron_asr"), "",
+             "an unlisted family reports no restriction");
+
+    // What the caller actually decides on, and it is a DIFFERENT question from
+    // "is the description empty": an entry with an empty allow list would
+    // describe itself as "" while refusing every dtype, so a caller that skipped
+    // the file read on the empty string would skip the check on the one entry
+    // that refuses everything.
+    check(family_has_weight_dtype_allow_list("supertonic"),
+          "a listed family has an allow list");
+    check(!family_has_weight_dtype_allow_list("nemotron_asr"),
+          "an unlisted family has none, which is what lets the caller skip "
+          "opening the file at all");
+    check(!family_has_weight_dtype_allow_list(""),
+          "an empty family name has no allow list");
+}
+
+int main() {
+    test_gguf_suffix_detection();
+    test_explicit_family_always_wins();
+    test_gguf_metadata_supplies_the_family();
+    test_foreign_gguf_is_refused();
+    test_directory_without_family_is_refused();
+    test_directory_ignores_embedded_family();
+    test_weight_dtype_allow_list();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all family_gate checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/generation_request.cpp

@@ -0,0 +1,280 @@
+#include "generation_request.h"
+
+#include <filesystem>
+#include <string>
+#include <system_error>
+#include <utility>
+
+namespace audiocpp_backend {
+namespace {
+
+const char *bool_option(bool value) { return value ? "true" : "false"; }
+
+} // namespace
+
+bool voice_is_reference_file(const std::string &voice) {
+    if (voice.empty()) {
+        return false;
+    }
+    std::error_code ec;
+    return std::filesystem::is_regular_file(std::filesystem::path(voice), ec);
+}
+
+RequestShape build_tts_shape(const backend::TTSRequest &request) {
+    RequestShape shape;
+    shape.has_voice_reference = voice_is_reference_file(request.voice());
+    // !empty() as well as has_instructions(), and it must match the guard in
+    // build_tts_request: a request whose instructions are an empty string
+    // carries no style condition, so telling routing to prefer VoiceDesign for
+    // it would route to a task with nothing to design from.
+    shape.has_instructions =
+        request.has_instructions() && !request.instructions().empty();
+    return shape;
+}
+
+engine::runtime::TaskRequest
+build_tts_request(const backend::TTSRequest &request,
+                  std::optional<engine::runtime::AudioBuffer> reference_audio) {
+    engine::runtime::TaskRequest task;
+
+    // The Transcript, not an option, is where every TTS family reads its
+    // language: chatterbox normalises request.text_input->language into its
+    // voice-clone config, qwen3_tts reads it as out.language, ace_step turns it
+    // into vocal_language. Upstream's own HTTP server does the same and sets no
+    // language option at all (app/server/runtime.cpp build_speech_request).
+    engine::runtime::Transcript transcript;
+    transcript.text = request.text();
+    if (request.has_language()) {
+        transcript.language = request.language();
+    }
+    task.text_input = std::move(transcript);
+
+    engine::runtime::VoiceCondition condition;
+    bool condition_used = false;
+
+    if (reference_audio.has_value()) {
+        // A clip: VoiceReference::audio, at the file's own rate and channel
+        // count. See kVoiceReferenceSampleRate in grpc-server.cpp for why it is
+        // not folded first.
+        engine::runtime::VoiceReference reference;
+        reference.audio = std::move(*reference_audio);
+        condition.speaker = std::move(reference);
+        condition_used = true;
+    } else if (!request.voice().empty()) {
+        // A named preset. cached_voice_id is the channel that actually lands:
+        // supertonic (options.voice), pocket_tts (voice_config.preset_name),
+        // voxcpm2, vibevoice, fish_audio (its saved-reference lookup) and
+        // qwen3_tts CustomVoice all read request.voice->speaker->cached_voice_id,
+        // and upstream's own server puts a non-preset `voice` body field in
+        // exactly this slot (app/server/runtime.cpp build_speech_request).
+        engine::runtime::VoiceReference reference;
+        reference.cached_voice_id = request.voice();
+        condition.speaker = std::move(reference);
+        condition_used = true;
+        // Forward-tolerant alias only. A bare "voice" REQUEST OPTION is read by
+        // no family in the pinned upstream: grepping find_option for it returns
+        // nothing. It is sent so a family adopting the name later works with no
+        // change here, not because it does anything today.
+        task.options["voice"] = request.voice();
+    }
+
+    if (request.has_instructions() && !request.instructions().empty()) {
+        // "instruct" is the key upstream itself maps the OpenAI `instructions`
+        // body field onto (app/server/runtime.cpp: request.options["instruct"]
+        // = value->as_string()), and it is read: qwen3_tts VoiceDesign and
+        // CustomVoice both take find_option(options, {"instruct"}) first, and
+        // omnivoice reads it in resolve_instruct.
+        task.options["instruct"] = request.instructions();
+        // "caption" is irodori_tts's name for the same thing, read in its
+        // make_request and documented as the voice-design caption for the 600M
+        // VoiceDesign model (docs/tts.md). Without this, that family's voice
+        // design cannot be driven from this RPC at all.
+        task.options["caption"] = request.instructions();
+        // The proto field's own name, forwarded for the same forward-tolerant
+        // reason as "voice" above and with the same honest accounting: NO family
+        // in the pinned upstream reads a request option called "instructions".
+        task.options["instructions"] = request.instructions();
+
+        engine::runtime::StyleCondition style;
+        // "instruct", not "instructions". This tag IS read, and only under that
+        // spelling: omnivoice and qwen3_tts both fall back to
+        // request.voice->style->tags.find("instruct") when the option is absent.
+        // Spelling it "instructions" here would have made the whole
+        // StyleCondition dead weight.
+        style.tags["instruct"] = request.instructions();
+        // !empty(), matching the option emission below, and load-bearing rather
+        // than tidiness. core/backend/tts.go's newTTSRequest sets
+        // `Language: &language` UNCONDITIONALLY, so has_language() is true on
+        // every request LocalAI sends and carries "" whenever the caller named
+        // no language. An engaged-but-empty style language is WORSE than an
+        // absent one: supertonic reads text_input->language behind its own
+        // !empty() guard and then OVERRIDES it from style->language with no
+        // guard at all (supertonic/session.cpp), so "" would replace its "en"
+        // default and tokenizer_text.cpp would throw
+        // "invalid Supertonic language: " on every request that set
+        // instructions and no language.
+        if (request.has_language() && !request.language().empty()) {
+            style.language = request.language();
+        }
+        condition.style = std::move(style);
+        condition_used = true;
+    }
+
+    if (condition_used) {
+        task.voice = std::move(condition);
+    }
+
+    if (request.has_language() && !request.language().empty()) {
+        // Forward-tolerant alias, exactly as in build_transcription_request. The
+        // families that read a "language" request option are the ASR ones
+        // (nemotron_asr, hviske_asr, vibevoice_asr, higgs_audio_stt), none of
+        // which this RPC can route to; pocket_tts reads one but from its
+        // ModelLoadRequest at load time, not from here. The Transcript above is
+        // what actually carries the language to a TTS family.
+        task.options["language"] = request.language();
+    }
+
+    // LAST, so an explicit params entry wins over anything derived above. That
+    // matters for "caption": a caller who sets params[caption] has named the
+    // exact string they want, and it must not be overwritten by `instructions`.
+    for (const auto &param : request.params()) {
+        task.options[param.first] = param.second;
+    }
+    return task;
+}
+
+engine::runtime::TaskRequest
+build_sound_generation_request(const backend::SoundGenerationRequest &request,
+                               std::optional<engine::runtime::AudioBuffer> source_audio) {
+    engine::runtime::TaskRequest task;
+
+    engine::runtime::Transcript transcript;
+    transcript.text = request.text();
+    if (request.has_language()) {
+        transcript.language = request.language();
+    }
+    task.text_input = std::move(transcript);
+
+    // src is the input clip for the editing routes. ace_step's repaint, cover
+    // and edit routes need it; stable_audio uses it as init_audio or
+    // inpaint_audio; heartmula refuses it outright.
+    if (source_audio.has_value()) {
+        task.audio_input = std::move(*source_audio);
+    }
+
+    // WHAT LANDS AND WHAT DOES NOT. Three families advertise AudioGeneration in
+    // the pinned upstream: ace_step, heartmula and stable_audio. Every key below
+    // was grepped against find_option/parse_*_option in src/ and include/ rather
+    // than assumed, because a key nobody reads is not a feature and shipping one
+    // while implying it works is the mistake this comment exists to prevent.
+    //
+    // Unknown REQUEST options cannot turn a valid request into an error:
+    // families look theirs up by name and ignore the rest, and the unknown-key
+    // refusals upstream does have are on SESSION options, which arrive at load
+    // time. So a forward-tolerant alias is free; it is just not a feature.
+
+    if (request.has_duration()) {
+        // duration_seconds is the key that works, and it works everywhere:
+        // ace_step (request_parser.cpp), heartmula (session.cpp, which also
+        // refuses a non-positive value) and stable_audio (request.cpp) all read
+        // it. This is the SoundGeneration analogue of Task 9's return_timestamps.
+        task.options["duration_seconds"] = std::to_string(request.duration());
+        // The proto field's own name. Read by exactly one family, omnivoice, and
+        // omnivoice advertises Tts rather than AudioGeneration, so this RPC can
+        // never route to it: DEAD here, kept only as a forward-tolerant alias.
+        task.options["duration"] = std::to_string(request.duration());
+    }
+    if (request.has_temperature()) {
+        // Read by heartmula. ace_step's sampling temperature is a different,
+        // narrower knob it calls lm_temperature (it drives the caption/thinking
+        // LM, not the audio diffusion), so this is deliberately NOT mapped onto
+        // it; a caller who wants it sets it through the request options that
+        // reach ace_step by name. stable_audio has no temperature at all.
+        task.options["temperature"] = std::to_string(request.temperature());
+    }
+    if (request.has_sample()) {
+        // do_sample is read widely upstream, but only by TTS and ASR families
+        // (chatterbox, index_tts2, miotts, moss, qwen3_tts, vibevoice,
+        // hviske_asr, voxtral_realtime). NO AudioGeneration family reads it, so
+        // it is dead on this route.
+        task.options["do_sample"] = bool_option(request.sample());
+    }
+    if (request.has_src_divisor()) {
+        // Read by nobody, anywhere in the pinned upstream. Forwarded because the
+        // proto documents it as part of this request and a family adopting it
+        // then works unchanged.
+        task.options["src_divisor"] = std::to_string(request.src_divisor());
+    }
+    if (request.has_think()) {
+        // "thinking" is the key ace_step actually reads (request_parser.cpp),
+        // which is why it is sent alongside the proto's own "think". "think" on
+        // its own is read by nobody.
+        task.options["thinking"] = bool_option(request.think());
+        task.options["think"] = bool_option(request.think());
+    }
+    if (request.has_caption()) {
+        // Read only by irodori_tts, which advertises Tts/VoiceCloning/VoiceDesign
+        // and not AudioGeneration, so it is unreachable from this RPC: DEAD here.
+        task.options["caption"] = request.caption();
+    }
+    if (request.has_lyrics()) {
+        // Read by ace_step and heartmula.
+        task.options["lyrics"] = request.lyrics();
+    }
+    if (request.has_bpm()) {
+        // Read by ace_step.
+        task.options["bpm"] = std::to_string(request.bpm());
+    }
+    if (request.has_keyscale()) {
+        // Read by ace_step.
+        task.options["keyscale"] = request.keyscale();
+    }
+    if (request.has_timesignature()) {
+        // Read by ace_step.
+        task.options["timesignature"] = request.timesignature();
+    }
+    if (request.has_instrumental()) {
+        // Read by nobody: "instrumental" appears in the pinned upstream only as
+        // a roformer STEM NAME, never as a request option. A forward-tolerant
+        // alias and nothing more.
+        task.options["instrumental"] = bool_option(request.instrumental());
+    }
+    if (request.has_language() && !request.language().empty()) {
+        // Alias again: the Transcript above is what ace_step reads as
+        // vocal_language. No AudioGeneration family reads a "language" option.
+        task.options["language"] = request.language();
+    }
+    return task;
+}
+
+bool apply_transform_text_input(engine::runtime::TaskRequest &task) {
+    // Canonical first, alias second, and an empty value falls through to the
+    // next candidate rather than ending the search: a caller who sent
+    // target_text="" and text="the real one" meant the second one.
+    static const char *const kTextKeys[] = {"target_text", "text"};
+
+    std::string text;
+    for (const char *key : kTextKeys) {
+        const auto found = task.options.find(key);
+        if (found != task.options.end() && !found->second.empty()) {
+            text = found->second;
+            break;
+        }
+    }
+    if (text.empty()) {
+        return false;
+    }
+
+    engine::runtime::Transcript transcript;
+    transcript.text = std::move(text);
+    // Inside the has-text branch on purpose. See the header: a language on its
+    // own conditions nothing and must not manufacture a text_input.
+    const auto language = task.options.find("language");
+    if (language != task.options.end()) {
+        transcript.language = language->second;
+    }
+    task.text_input = std::move(transcript);
+    return true;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/generation_request.h

@@ -0,0 +1,108 @@
+#pragma once
+
+// Builds the engine::runtime::TaskRequest for the two audio-PRODUCING offline
+// RPCs, TTS and SoundGeneration, and answers the one filesystem question TTS
+// routing depends on.
+//
+// It is a unit of its own rather than a pair of statics in grpc-server.cpp so
+// that it can be tested: grpc-server.cpp has a main() and cannot be linked into
+// a test binary, and everything here is a pure function of its arguments once
+// the file read has been lifted out (which is why the reference clip arrives as
+// an already-read buffer rather than a path). TTSStream reuses build_tts_request
+// unchanged.
+//
+// Only the plain structs in engine/framework/runtime/session.h are touched, so
+// this compiles against the header without linking engine_runtime, the same way
+// result_map does.
+
+#include "backend.pb.h"
+#include "capability_routing.h"
+
+#include "engine/framework/runtime/session.h"
+
+#include <optional>
+#include <string>
+
+namespace audiocpp_backend {
+
+// True when TTSRequest.voice names an existing regular file, in which case it
+// is a speaker reference clip and routing prefers VoiceCloning; false when it is
+// a named preset (or empty).
+//
+// The overload is LocalAI's, not this backend's: `voice` is the OpenAI speech
+// field and different LocalAI backends have always read it both ways. Deciding
+// it from the filesystem needs no new option and matches how somebody actually
+// configures a cloning family, which is by pointing at a clip.
+//
+// A DIRECTORY is deliberately not a reference: is_regular_file, not exists. A
+// directory named as a voice cannot be read as a WAV, and treating it as a
+// reference would turn a preset typo into "cannot read /x as WAV" instead of
+// letting it travel as the preset name it looks like.
+//
+// The error_code overload is used so an unreadable parent directory answers
+// false rather than throwing. That is the right answer here: the name is then
+// passed on as a preset, and if it really was meant to be a clip the family
+// refuses a request it cannot serve, which is a better message than a
+// filesystem exception thrown while classifying a string.
+bool voice_is_reference_file(const std::string &voice);
+
+// Everything routing needs to know about a TTSRequest, in one place, so that TTS
+// and TTSStream cannot describe the same request differently.
+//
+// `pinned_task` is deliberately NOT filled here: it comes off the LoadedModel,
+// not off the request, and this unit links no engine. The caller must still
+// write `shape.pinned_task = model->pinned_task();` or the model's `task:`
+// option is dead. That is the one field a new handler can forget, so it is the
+// one field left visible at the call site rather than hidden behind this
+// helper.
+RequestShape build_tts_shape(const backend::TTSRequest &request);
+
+// `reference_audio` is the already-read speaker clip, present exactly when
+// voice_is_reference_file(request.voice()) was true. Passing it in rather than a
+// path keeps this function pure and lets the caller do the read where the
+// ordering rules (capability refusal first, then the lane) are enforced.
+//
+// It is taken BY VALUE and moved in: a reference clip is seconds of audio and
+// the caller has no use for it afterwards.
+engine::runtime::TaskRequest
+build_tts_request(const backend::TTSRequest &request,
+                  std::optional<engine::runtime::AudioBuffer> reference_audio);
+
+// `source_audio` is SoundGenerationRequest.src already read, present exactly
+// when the field was set and non-empty. Same reasoning as above.
+engine::runtime::TaskRequest
+build_sound_generation_request(const backend::SoundGenerationRequest &request,
+                               std::optional<engine::runtime::AudioBuffer> source_audio);
+
+// Lifts a text-conditioned transform route's text out of the request params
+// into TaskRequest.text_input, and reports whether it set one.
+//
+// WHY THIS EXISTS. AudioTransform is an audio-in / audio-out RPC and its proto
+// message has no text field, but not every task it routes to is audio-only.
+// vevo2's speech-to-speech and prosody routes read their text from
+// request.text_input (src/models/vevo2/session.cpp fills refs.target_text from
+// exactly there and nowhere else) and refuse the run without one: "Vevo2
+// text/prosody route requires text_input or target_text". The params map is the
+// only channel AudioTransform has that reaches the engine, so the text travels
+// through it and is unpacked here. Without this, s2s is not merely awkward to
+// reach through this RPC, it is unreachable.
+//
+// CALL IT AFTER the params have been copied into task.options, and note that it
+// does NOT erase the keys it reads. vevo2's loader advertises "target_text" in
+// its own documented request-option table, so a family that looks there keeps
+// finding it; the copy in text_input is what the session actually reads today.
+//
+// "target_text" is canonical and "text" is its alias, the same order vevo2's
+// option table declares them in. A request setting both gets target_text, so
+// the canonical spelling wins rather than whichever the map happened to store
+// first. An empty value is not a text: it means the caller sent the key with
+// nothing in it, and a family asked to vocalise "" should say so itself rather
+// than be handed an empty Transcript that looks deliberate.
+//
+// "language" rides along when a text was found, and only then. On its own it
+// conditions nothing, and setting text_input for it alone would turn a plain
+// separation request that happened to carry a language hint into a text-routed
+// one.
+bool apply_transform_text_input(engine::runtime::TaskRequest &task);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/generation_request_ctest.cpp

@@ -0,0 +1,575 @@
+// Tests for the TTS and SoundGeneration request builders, and for the
+// filesystem rule that decides whether TTSRequest.voice is a speaker reference
+// clip or a named preset.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE: see the note at the top of
+// result_map_ctest.cpp. This file needs the generated protobuf messages and the
+// audio.cpp include path, neither of which backend/cpp/run-unit-tests.sh
+// provides, so it is built and run by ctest:
+//
+//   make -C backend/cpp/audio-cpp test-engine
+//
+// The assertions on OPTION KEYS are the point of this file, not decoration.
+// Every one of them names a key that was grepped against the pinned upstream:
+// "instruct" is read, "instructions" is not; "duration_seconds" is read,
+// "duration" is not. A rename that looks harmless is exactly the change that
+// silently stops a family honouring the request, so the spellings are pinned
+// here rather than left to a comment.
+
+#include "generation_request.h"
+
+#include <cstdio>
+#include <filesystem>
+#include <fstream>
+#include <string>
+#include <unordered_map>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+static bool has_key(const std::unordered_map<std::string, std::string> &options,
+                    const std::string &key) {
+    return options.find(key) != options.end();
+}
+
+static std::string option_or(
+    const std::unordered_map<std::string, std::string> &options,
+    const std::string &key, const std::string &fallback) {
+    const auto it = options.find(key);
+    return it == options.end() ? fallback : it->second;
+}
+
+static engine::runtime::AudioBuffer clip(int sample_rate, int channels) {
+    engine::runtime::AudioBuffer buffer;
+    buffer.sample_rate = sample_rate;
+    buffer.channels = channels;
+    // Distinguishable content, so a builder that swapped one buffer for another
+    // (or default-constructed one) is visible rather than merely a size change.
+    buffer.samples = {0.25f, -0.5f, 0.75f, -1.0f};
+    return buffer;
+}
+
+static void test_voice_is_reference_file() {
+    const auto dir = std::filesystem::temp_directory_path() /
+                     "audiocpp-generation-request-ctest";
+    std::filesystem::remove_all(dir);
+    std::filesystem::create_directories(dir);
+
+    const auto file = dir / "reference.wav";
+    {
+        std::ofstream out(file, std::ios::binary);
+        out << "not really a wav, but a regular file";
+    }
+    const auto subdir = dir / "a-directory";
+    std::filesystem::create_directories(subdir);
+
+    check(!voice_is_reference_file(""), "voice_is_reference_file: empty");
+    check(!voice_is_reference_file("alloy"),
+          "voice_is_reference_file: bare preset name");
+    check(!voice_is_reference_file((dir / "absent.wav").string()),
+          "voice_is_reference_file: missing path");
+    check(voice_is_reference_file(file.string()),
+          "voice_is_reference_file: existing regular file");
+    // A directory is NOT a reference. exists() would say yes here and the read
+    // would then fail with "cannot read <dir> as WAV", which sends the operator
+    // after a file problem instead of a preset typo.
+    check(!voice_is_reference_file(subdir.string()),
+          "voice_is_reference_file: directory is not a reference");
+
+    std::filesystem::remove_all(dir);
+}
+
+// build_tts_shape is what TTS and TTSStream both hand to routing, so a wrong
+// answer here silently changes which task a request runs as, with a 200 and no
+// diagnostic. Every field is asserted in both directions.
+static void test_tts_shape() {
+    const auto dir = std::filesystem::temp_directory_path() /
+                     "audiocpp-generation-request-ctest-shape";
+    std::filesystem::remove_all(dir);
+    std::filesystem::create_directories(dir);
+    const auto file = dir / "reference.wav";
+    {
+        std::ofstream out(file, std::ios::binary);
+        out << "a regular file";
+    }
+
+    {
+        backend::TTSRequest request;
+        request.set_text("hello");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference && !shape.has_instructions,
+              "shape: bare request has neither signal");
+        // Never filled here: it comes off the LoadedModel, and leaving it empty
+        // is what makes the caller's assignment visible at the call site.
+        check(shape.pinned_task.empty(), "shape: pinned_task is left to the caller");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(file.string());
+        const auto shape = build_tts_shape(request);
+        check(shape.has_voice_reference,
+              "shape: an existing file is a voice reference");
+        check(!shape.has_instructions, "shape: a clip is not an instruction");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice("alloy");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference,
+              "shape: a preset name is not a voice reference");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(dir.string());
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_voice_reference,
+              "shape: a directory is not a voice reference");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_instructions("a calm older man");
+        const auto shape = build_tts_shape(request);
+        check(shape.has_instructions, "shape: instructions are seen");
+        check(!shape.has_voice_reference,
+              "shape: instructions do not imply a reference");
+    }
+    {
+        // The guard that has to match build_tts_request's. An empty
+        // instructions string builds no style condition, so telling routing to
+        // prefer VoiceDesign for it would route to a task with nothing to
+        // design from.
+        backend::TTSRequest request;
+        request.set_instructions("");
+        const auto shape = build_tts_shape(request);
+        check(!shape.has_instructions,
+              "shape: an empty instructions string is not an instruction");
+    }
+    {
+        backend::TTSRequest request;
+        request.set_voice(file.string());
+        request.set_instructions("a calm older man");
+        const auto shape = build_tts_shape(request);
+        check(shape.has_voice_reference && shape.has_instructions,
+              "shape: both signals are reported when both are set");
+    }
+
+    std::filesystem::remove_all(dir);
+}
+
+static void test_tts_plain() {
+    backend::TTSRequest request;
+    request.set_text("hello there");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.text_input.has_value() && task.text_input->text == "hello there",
+          "tts: text reaches the transcript");
+    // No voice and no instructions means NO voice condition at all. A builder
+    // that always emitted one would make every family think a speaker was
+    // named, and chatterbox in particular refuses a prepare whose voice
+    // condition carries neither audio nor anything else it can use.
+    check(!task.voice.has_value(), "tts: no voice condition when nothing is set");
+    check(task.options.empty(), "tts: no options when nothing is set");
+    check(!task.audio_input.has_value(), "tts: no audio input");
+}
+
+static void test_tts_named_preset() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("alloy");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.voice.has_value() && task.voice->speaker.has_value(),
+          "tts preset: speaker condition present");
+    check(task.voice->speaker->cached_voice_id.has_value() &&
+              *task.voice->speaker->cached_voice_id == "alloy",
+          "tts preset: lands in cached_voice_id");
+    // The clip slot must stay empty, or a cloning family would try to prepare
+    // conditionals from a default-constructed buffer.
+    check(!task.voice->speaker->audio.has_value(),
+          "tts preset: no reference audio");
+    check(!task.voice->style.has_value(), "tts preset: no style condition");
+    check(option_or(task.options, "voice", "") == "alloy",
+          "tts preset: forwarded as the voice option too");
+}
+
+static void test_tts_reference_clip() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("/tmp/reference.wav");
+
+    const auto task = build_tts_request(request, clip(44100, 2));
+
+    check(task.voice.has_value() && task.voice->speaker.has_value(),
+          "tts clip: speaker condition present");
+    check(task.voice->speaker->audio.has_value(),
+          "tts clip: reference audio present");
+    // Rate and channels survive untouched. This is the assertion that fails if
+    // anybody decides to fold the clip to 16 kHz mono on the way in.
+    check(task.voice->speaker->audio->sample_rate == 44100 &&
+              task.voice->speaker->audio->channels == 2 &&
+              task.voice->speaker->audio->samples.size() == 4,
+          "tts clip: rate, channels and samples pass through unchanged");
+    // A clip is NOT also a cached voice id, and the path must not travel as a
+    // preset name: a family reading cached_voice_id would then look up a voice
+    // called "/tmp/reference.wav".
+    check(!task.voice->speaker->cached_voice_id.has_value(),
+          "tts clip: no cached_voice_id");
+    check(!has_key(task.options, "voice"), "tts clip: no voice option");
+}
+
+static void test_tts_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("a calm older man, speaking slowly");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(option_or(task.options, "instruct", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: instruct option is the one qwen3_tts reads");
+    check(option_or(task.options, "caption", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: caption option is the one irodori_tts reads");
+    check(option_or(task.options, "instructions", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: proto field name forwarded as an alias");
+    check(task.voice.has_value() && task.voice->style.has_value(),
+          "tts instructions: style condition present");
+    // "instruct", not "instructions". omnivoice and qwen3_tts both look this tag
+    // up by that exact name and by no other.
+    check(option_or(task.voice->style->tags, "instruct", "") ==
+              "a calm older man, speaking slowly",
+          "tts instructions: style tag is spelled instruct");
+    check(!has_key(task.voice->style->tags, "instructions"),
+          "tts instructions: style tag is NOT spelled instructions");
+    // Instructions alone must not invent a speaker: has_voice_reference is what
+    // routing keys VoiceCloning off, and a speaker here would make a
+    // clone-capable family expect a clip it never received.
+    check(!task.voice->speaker.has_value(),
+          "tts instructions: no speaker without a voice");
+}
+
+static void test_tts_empty_instructions_are_not_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    // has_instructions() is true here, because the field was set. An empty
+    // string is still no instruction, and forwarding it would set an empty
+    // instruct option that qwen3_tts would prefer over its style tag fallback.
+    check(!has_key(task.options, "instruct"),
+          "tts: an empty instructions string sets no instruct option");
+    check(!task.voice.has_value(),
+          "tts: an empty instructions string sets no voice condition");
+}
+
+// THE EXACT SHAPE LocalAI PUTS ON THE WIRE. core/backend/tts.go's newTTSRequest
+// sets Language: &language UNCONDITIONALLY, so has_language() is true on every
+// request that ever reaches this backend, carrying an empty string whenever the
+// caller named no language.
+//
+// An empty StyleCondition::language is not a harmless default. supertonic reads
+// text_input->language behind a !empty() guard and then OVERRIDES it from
+// style->language whenever that optional is engaged, with no guard at all
+// (supertonic/session.cpp generation_options_from_request), so an empty style
+// language replaces its "en" default (session.h) with "" and
+// tokenizer_text.cpp's preprocess throws "invalid Supertonic language: ".
+// Every /v1/audio/speech request carrying instructions and no language would be
+// an INTERNAL. A plain request never sees it, because the style condition only
+// exists when instructions are non-empty.
+static void test_tts_empty_language_is_not_a_language() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_instructions("a calm older man");
+    request.set_language("");
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.voice.has_value() && task.voice->style.has_value(),
+          "tts empty language: the style condition still exists");
+    check(!task.voice->style->language.has_value(),
+          "tts empty language: style language is left unset, not set to empty");
+    // The option emission has always guarded on !empty(); this pins the two to
+    // the same rule so they cannot drift apart again.
+    check(!has_key(task.options, "language"),
+          "tts empty language: no language option");
+    check(task.text_input.has_value() && task.text_input->language.empty(),
+          "tts empty language: transcript language stays empty");
+}
+
+// The one shape routing treats specially: a clip outranks instructions, and the
+// VoiceCondition then has to carry BOTH, because the family that wins is chosen
+// on the clip but may still read the style tag.
+static void test_tts_clip_and_instructions() {
+    backend::TTSRequest request;
+    request.set_text("hello");
+    request.set_voice("/tmp/reference.wav");
+    request.set_instructions("bright and fast");
+    request.set_language("en");
+
+    const auto task = build_tts_request(request, clip(22050, 1));
+
+    check(task.voice.has_value(), "tts clip+instructions: voice condition present");
+    check(task.voice->speaker.has_value() &&
+              task.voice->speaker->audio.has_value() &&
+              task.voice->speaker->audio->sample_rate == 22050,
+          "tts clip+instructions: speaker carries the clip");
+    check(!task.voice->speaker->cached_voice_id.has_value(),
+          "tts clip+instructions: the clip path is not also a preset id");
+    check(task.voice->style.has_value() &&
+              option_or(task.voice->style->tags, "instruct", "") == "bright and fast",
+          "tts clip+instructions: style carries the instruct tag");
+    check(task.voice->style->language.has_value() &&
+              *task.voice->style->language == "en",
+          "tts clip+instructions: a real language does reach the style condition");
+    check(option_or(task.options, "instruct", "") == "bright and fast",
+          "tts clip+instructions: instruct option still emitted");
+    check(!has_key(task.options, "voice"),
+          "tts clip+instructions: still no voice option for a clip");
+}
+
+static void test_tts_language_and_params() {
+    backend::TTSRequest request;
+    request.set_text("ciao");
+    request.set_language("it");
+    request.set_instructions("warm");
+    (*request.mutable_params())["exaggeration"] = "0.7";
+    // An explicit param must win over the value derived from instructions.
+    (*request.mutable_params())["caption"] = "explicitly chosen caption";
+
+    const auto task = build_tts_request(request, std::nullopt);
+
+    check(task.text_input.has_value() && task.text_input->language == "it",
+          "tts language: reaches the transcript");
+    check(option_or(task.options, "language", "") == "it",
+          "tts language: forwarded as an option alias");
+    check(task.voice.has_value() && task.voice->style.has_value() &&
+              task.voice->style->language.has_value() &&
+              *task.voice->style->language == "it",
+          "tts language: reaches the style condition");
+    check(option_or(task.options, "exaggeration", "") == "0.7",
+          "tts params: passed through verbatim");
+    check(option_or(task.options, "caption", "") == "explicitly chosen caption",
+          "tts params: an explicit param overrides the derived caption");
+}
+
+static void test_sound_generation_minimal() {
+    backend::SoundGenerationRequest request;
+    request.set_text("a distant thunderstorm");
+
+    const auto task = build_sound_generation_request(request, std::nullopt);
+
+    check(task.text_input.has_value() &&
+              task.text_input->text == "a distant thunderstorm",
+          "sound: text reaches the transcript");
+    // Unset optionals must emit NOTHING. Emitting a zero for an unset duration
+    // would make heartmula refuse the request ("duration_seconds must be
+    // positive") on a request that never mentioned a duration.
+    check(task.options.empty(), "sound: unset optionals emit no options");
+    check(!task.audio_input.has_value(), "sound: no audio input without src");
+    check(!task.voice.has_value(), "sound: no voice condition");
+}
+
+static void test_sound_generation_full() {
+    backend::SoundGenerationRequest request;
+    request.set_text("a slow blues in E");
+    request.set_duration(30.0f);
+    request.set_temperature(0.8f);
+    request.set_sample(false);
+    request.set_src_divisor(4);
+    request.set_think(true);
+    request.set_caption("smoky bar recording");
+    request.set_lyrics("first line\nsecond line");
+    request.set_bpm(72);
+    request.set_keyscale("E minor");
+    request.set_language("en");
+    request.set_timesignature("4/4");
+    request.set_instrumental(true);
+
+    const auto task = build_sound_generation_request(request, clip(48000, 2));
+
+    // duration_seconds is the key every AudioGeneration family actually reads;
+    // "duration" rides along as an alias. Both spellings are pinned so a
+    // "cleanup" that keeps only the proto's own name is a test failure and not
+    // a silent loss of the duration.
+    check(option_or(task.options, "duration_seconds", "").rfind("30.", 0) == 0,
+          "sound: duration lands as duration_seconds");
+    check(option_or(task.options, "duration", "").rfind("30.", 0) == 0,
+          "sound: duration also forwarded under its own name");
+    check(option_or(task.options, "temperature", "").rfind("0.8", 0) == 0,
+          "sound: temperature forwarded");
+    // Set to FALSE, so this also proves the key is written whenever the field is
+    // present rather than only when the value is truthy.
+    check(option_or(task.options, "do_sample", "") == "false",
+          "sound: sample=false is forwarded as do_sample=false");
+    check(option_or(task.options, "src_divisor", "") == "4",
+          "sound: src_divisor forwarded");
+    check(option_or(task.options, "thinking", "") == "true",
+          "sound: think lands as thinking, the key ace_step reads");
+    check(option_or(task.options, "think", "") == "true",
+          "sound: think also forwarded under its own name");
+    check(option_or(task.options, "caption", "") == "smoky bar recording",
+          "sound: caption forwarded");
+    check(option_or(task.options, "lyrics", "") == "first line\nsecond line",
+          "sound: lyrics forwarded");
+    check(option_or(task.options, "bpm", "") == "72", "sound: bpm forwarded");
+    check(option_or(task.options, "keyscale", "") == "E minor",
+          "sound: keyscale forwarded");
+    check(option_or(task.options, "timesignature", "") == "4/4",
+          "sound: timesignature forwarded");
+    check(option_or(task.options, "instrumental", "") == "true",
+          "sound: instrumental forwarded");
+    check(option_or(task.options, "language", "") == "en",
+          "sound: language forwarded as an option alias");
+    check(task.text_input->language == "en",
+          "sound: language reaches the transcript, which is what ace_step reads");
+    check(task.audio_input.has_value() &&
+              task.audio_input->sample_rate == 48000 &&
+              task.audio_input->channels == 2 &&
+              task.audio_input->samples.size() == 4,
+          "sound: src passes through at its own rate and channel count");
+}
+
+
+// ---------------------------------------------------------------------------
+// apply_transform_text_input
+//
+// AudioTransform has no text field on the wire, so a text-conditioned route
+// (vevo2's speech-to-speech) can only be reached if the text travels as a
+// param and is unpacked into text_input. Every assertion below pins a spelling
+// or a precedence that a family actually depends on, not a shape that merely
+// looks tidy.
+
+static void test_transform_text_absent() {
+    engine::runtime::TaskRequest task;
+    task.options["stem"] = "vocals";
+    check(!apply_transform_text_input(task),
+          "transform text: reports false when no text key is present");
+    check(!task.text_input.has_value(),
+          "transform text: a request with no text keeps text_input unset");
+}
+
+static void test_transform_text_canonical_key() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "sing this line";
+    check(apply_transform_text_input(task), "transform text: target_text reports true");
+    check(task.text_input.has_value() && task.text_input->text == "sing this line",
+          "transform text: target_text becomes text_input.text");
+    check(has_key(task.options, "target_text"),
+          "transform text: target_text survives in options for families that read it there");
+}
+
+static void test_transform_text_alias_key() {
+    engine::runtime::TaskRequest task;
+    task.options["text"] = "say this instead";
+    check(apply_transform_text_input(task), "transform text: text alias reports true");
+    check(task.text_input.has_value() && task.text_input->text == "say this instead",
+          "transform text: the text alias becomes text_input.text");
+}
+
+static void test_transform_text_canonical_wins() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "canonical";
+    task.options["text"] = "alias";
+    check(apply_transform_text_input(task), "transform text: both keys reports true");
+    check(task.text_input.has_value() && task.text_input->text == "canonical",
+          "transform text: target_text wins over text, not whichever hashed first");
+}
+
+static void test_transform_text_empty_is_not_a_text() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "";
+    check(!apply_transform_text_input(task),
+          "transform text: an empty target_text reports false");
+    check(!task.text_input.has_value(),
+          "transform text: an empty target_text leaves text_input unset");
+}
+
+static void test_transform_text_empty_canonical_falls_through_to_alias() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "";
+    task.options["text"] = "the real one";
+    check(apply_transform_text_input(task),
+          "transform text: an empty canonical key does not mask a usable alias");
+    check(task.text_input.has_value() && task.text_input->text == "the real one",
+          "transform text: the alias is used when the canonical key is empty");
+}
+
+static void test_transform_text_language_rides_along() {
+    engine::runtime::TaskRequest task;
+    task.options["target_text"] = "vocalise me";
+    task.options["language"] = "ja";
+    check(apply_transform_text_input(task), "transform text: text plus language reports true");
+    check(task.text_input.has_value() && task.text_input->language == "ja",
+          "transform text: language lands on the Transcript alongside the text");
+    check(has_key(task.options, "language"),
+          "transform text: language survives in options too");
+}
+
+static void test_transform_language_alone_is_not_a_text() {
+    engine::runtime::TaskRequest task;
+    task.options["language"] = "ja";
+    check(!apply_transform_text_input(task),
+          "transform text: a language with no text reports false");
+    check(!task.text_input.has_value(),
+          "transform text: a language alone must not route a separation request through text");
+}
+
+static void test_transform_text_preserves_other_inputs() {
+    engine::runtime::TaskRequest task;
+    engine::runtime::AudioBuffer audio;
+    audio.sample_rate = 44100;
+    audio.channels = 2;
+    audio.samples = {0.1f, 0.2f, 0.3f, 0.4f};
+    task.audio_input = audio;
+    task.options["target_text"] = "keep the audio";
+    check(apply_transform_text_input(task), "transform text: with audio present reports true");
+    check(task.audio_input.has_value() && task.audio_input->samples.size() == 4 &&
+              task.audio_input->sample_rate == 44100,
+          "transform text: the source audio is untouched");
+}
+
+int main() {
+    test_voice_is_reference_file();
+    test_tts_shape();
+    test_tts_plain();
+    test_tts_named_preset();
+    test_tts_reference_clip();
+    test_tts_instructions();
+    test_tts_empty_instructions_are_not_instructions();
+    test_tts_empty_language_is_not_a_language();
+    test_tts_clip_and_instructions();
+    test_tts_language_and_params();
+    test_sound_generation_minimal();
+    test_sound_generation_full();
+    test_transform_text_absent();
+    test_transform_text_canonical_key();
+    test_transform_text_alias_key();
+    test_transform_text_canonical_wins();
+    test_transform_text_empty_is_not_a_text();
+    test_transform_text_empty_canonical_falls_through_to_alias();
+    test_transform_text_language_rides_along();
+    test_transform_language_alone_is_not_a_text();
+    test_transform_text_preserves_other_inputs();
+
+    if (failures != 0) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/inference_lane.cpp

@@ -0,0 +1,114 @@
+#include "inference_lane.h"
+
+#include <algorithm>
+#include <chrono>
+
+namespace audiocpp_backend {
+
+std::int64_t monotonic_millis() {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+               std::chrono::steady_clock::now().time_since_epoch())
+        .count();
+}
+
+int resolve_wait_budget_ms(int policy_ceiling_ms, int request_hint_ms) {
+    // Both sides normalise to 0 for "not specified", which is also the value
+    // that means unbounded on the way out, so the unspecified cases fall out of
+    // the arithmetic instead of needing their own branches.
+    const int ceiling = policy_ceiling_ms > 0 ? policy_ceiling_ms : 0;
+    const int hint = request_hint_ms > 0 ? request_hint_ms : 0;
+
+    if (ceiling == 0) {
+        return hint;
+    }
+    if (hint == 0) {
+        return ceiling;
+    }
+    // The hint can tighten the ceiling but never loosen it.
+    return std::min(ceiling, hint);
+}
+
+bool run_exceeds_budget(bool lane_occupied, std::int64_t run_started_ms,
+                        std::int64_t now_ms, int budget_ms) {
+    if (!lane_occupied || budget_ms <= 0) {
+        return false;
+    }
+    return now_ms - run_started_ms > static_cast<std::int64_t>(budget_ms);
+}
+
+namespace {
+
+std::string busy_prefix(const std::string &model_label) {
+    return "inference lane for model '" + model_label + "' is busy: ";
+}
+
+// States the measurement and nothing else. A short-budget caller meeting a
+// legitimately long run lands here too, so this text must not declare the run
+// broken; the numbers let a reader decide that for themselves.
+std::string overrun_message(const std::string &model_label,
+                            std::int64_t run_age_ms, int budget_ms) {
+    return busy_prefix(model_label) + "the in-flight run has been running for " +
+           std::to_string(run_age_ms) + " ms, longer than this request's " +
+           std::to_string(budget_ms) + " ms wait budget";
+}
+
+std::string wait_exhausted_message(const std::string &model_label,
+                                   int budget_ms) {
+    return busy_prefix(model_label) + "timed out after " +
+           std::to_string(budget_ms) +
+           " ms waiting for the in-flight run to finish";
+}
+
+} // namespace
+
+void InferenceLane::occupy(int budget_ms) {
+    std::unique_lock<std::mutex> lock(state_mutex_);
+
+    // Checked once, on arrival: if the run already in the lane has outlived what
+    // this caller brought, no amount of waiting can help it, and queueing here
+    // is exactly how a wedged run swallows every handler thread.
+    const std::int64_t arrived_ms = monotonic_millis();
+    if (run_exceeds_budget(occupied_, run_started_ms_, arrived_ms, budget_ms)) {
+        throw LaneUnavailable(overrun_message(
+            model_label_, arrived_ms - run_started_ms_, budget_ms));
+    }
+
+    if (budget_ms > 0) {
+        if (!vacated_.wait_for(lock, std::chrono::milliseconds(budget_ms),
+                               [this] { return !occupied_; })) {
+            throw LaneUnavailable(
+                wait_exhausted_message(model_label_, budget_ms));
+        }
+    } else {
+        vacated_.wait(lock, [this] { return !occupied_; });
+    }
+
+    // Only now, holding both the mutex and the lane. Stamping any earlier would
+    // restart the age of the run for every caller behind this one and make a
+    // genuinely wedged holder look freshly started forever.
+    occupied_ = true;
+    run_started_ms_ = monotonic_millis();
+}
+
+void InferenceLane::vacate() {
+    {
+        std::lock_guard<std::mutex> lock(state_mutex_);
+        occupied_ = false;
+    }
+    // notify_all, not notify_one: a waiter that times out concurrently with a
+    // notification can consume it, and losing the only wakeup would park the
+    // remaining waiters for the rest of the run's lifetime. Waking all of them
+    // still admits exactly one, since the rest re-test occupancy under the mutex
+    // and go back to waiting, and ordering among waiters is not a requirement.
+    vacated_.notify_all();
+}
+
+LaneEntry::LaneEntry(InferenceLane &lane, int budget_ms) : lane_(lane) {
+    // If this throws, the object never existed, so ~LaneEntry does not run and
+    // cannot hand back a lane this caller never held.
+    lane_.occupy(budget_ms);
+}
+
+LaneEntry::~LaneEntry() { lane_.vacate(); }
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/inference_lane.h

@@ -0,0 +1,138 @@
+#pragma once
+
+// Serializes inference against the single audio.cpp model a backend process
+// owns. Standard library only.
+//
+// Why a plain mutex is not enough: an audio.cpp session is not reentrant, so
+// concurrent gRPC handlers have to take turns. But once a GPU call stops making
+// progress there is nothing a host thread can do to take it back, and an
+// unbounded queue behind such a run would absorb the handler threads one by one
+// until nothing is left to answer with. A caller therefore needs to be able to
+// walk away, and needs to be able to tell "the lane is busy with normal work and
+// I ran out of patience" apart from "the run in the lane has already outlived
+// the patience I brought".
+
+#include <condition_variable>
+#include <cstdint>
+#include <mutex>
+#include <stdexcept>
+#include <string>
+
+namespace audiocpp_backend {
+
+// Reading off a monotonic clock, in milliseconds. Monotonic on purpose: a wall
+// clock adjustment must never make an in-flight run look younger or older than
+// it is, because that reading decides whether callers give up.
+std::int64_t monotonic_millis();
+
+// Collapses the per-model configured ceiling and the optional per-request hint
+// into the wait budget a caller actually gets. Returns 0 for "wait
+// indefinitely".
+//
+// Either input may be non-positive, which means "not specified":
+//   - an unspecified hint yields the ceiling,
+//   - an unspecified ceiling means no policy limit, so the hint stands,
+//   - unspecified on both sides is unbounded.
+// A specified hint may only tighten the ceiling. A client asking for a longer
+// wait than the model's policy allows does not get it, because that would let a
+// request weaken an operator's choice.
+int resolve_wait_budget_ms(int policy_ceiling_ms, int request_hint_ms);
+
+// True when a caller carrying budget_ms should give up on arrival rather than
+// queue up. Deliberately a pure function of the lane's observable state so the
+// decision can be tested without threads or sleeping.
+//
+// Only occupancy makes a start timestamp meaningful: a lane nobody holds is
+// never overrunning, whatever timestamp the last holder left behind. An
+// unbounded caller (non-positive budget) has no budget to exceed. And the
+// comparison is strict, so a run whose age exactly equals the budget still has
+// its last millisecond.
+bool run_exceeds_budget(bool lane_occupied, std::int64_t run_started_ms,
+                        std::int64_t now_ms, int budget_ms);
+
+// Thrown when a caller cannot take the lane, in either of the two situations
+// resolve_wait_budget_ms allows for. The message distinguishes them; callers
+// that need to report a status code can treat them alike.
+class LaneUnavailable : public std::runtime_error {
+  public:
+    explicit LaneUnavailable(const std::string &reason)
+        : std::runtime_error(reason) {}
+};
+
+class LaneEntry;
+
+// One lane per loaded model. Shared by every handler thread; not copyable.
+class InferenceLane {
+  public:
+    explicit InferenceLane(std::string model_label)
+        : model_label_(std::move(model_label)) {}
+
+    InferenceLane(const InferenceLane &) = delete;
+    InferenceLane &operator=(const InferenceLane &) = delete;
+
+    const std::string &model_label() const { return model_label_; }
+
+  private:
+    // Occupancy is only reachable through LaneEntry, so there is no way to take
+    // the lane without also having something that gives it back.
+    friend class LaneEntry;
+
+    void occupy(int budget_ms);
+    void vacate();
+
+    const std::string model_label_;
+
+    std::mutex state_mutex_;
+    std::condition_variable vacated_;
+    bool occupied_ = false;
+    // Only meaningful while occupied_ is true.
+    std::int64_t run_started_ms_ = 0;
+};
+
+// Scoped occupancy of a lane. Construct it where the inference happens and it
+// is given back on every exit from that scope, including an exception and
+// including a caller that returns from the middle of a long stream. Throws
+// LaneUnavailable if the lane could not be taken, in which case there is no
+// object and nothing to release.
+//
+// Not reentrant, and it does not detect reentrancy: a second entry constructed
+// while the calling thread already holds the same lane waits for a lane only
+// that thread can release. With a positive budget that surfaces as
+// LaneUnavailable, but in unbounded mode the thread parks with no diagnostic at
+// all. Keep entries one per call: a handler that holds one across a stream must
+// not let a helper it calls construct another.
+class LaneEntry {
+  public:
+    // budget_ms <= 0 waits indefinitely. Pass the output of
+    // resolve_wait_budget_ms.
+    LaneEntry(InferenceLane &lane, int budget_ms);
+    ~LaneEntry();
+
+    LaneEntry(const LaneEntry &) = delete;
+    LaneEntry &operator=(const LaneEntry &) = delete;
+
+    // Deliberately immovable rather than carefully movable. A moved-from entry
+    // would have to stop releasing the lane while the lane still records it as
+    // occupied, and that hazard is not worth the convenience: the lane can only
+    // be recovered by whoever took it.
+    //
+    // To hold a lane for longer than one scope, construct the entry in place
+    // instead of moving one in. Two shapes work:
+    //   std::optional<LaneEntry> held;   // member or local
+    //   held.emplace(lane, budget_ms);   // takes the lane, held.reset() gives it back
+    //   auto held = std::make_unique<LaneEntry>(lane, budget_ms);  // also returnable
+    // Both outlive the acquiring scope and still release exactly once, when they
+    // are reset or destroyed. Prefer the optional for a member whose lifetime is
+    // the handler's; use the unique_ptr when the entry has to be returned, since
+    // an optional of an immovable type is itself immovable and cannot be. A
+    // factory may instead write `return LaneEntry(lane, budget_ms);`, which C++17
+    // guarantees to elide, whereas `LaneEntry entry(...); return entry;` does not
+    // compile, because that form is a move.
+    LaneEntry(LaneEntry &&) = delete;
+    LaneEntry &operator=(LaneEntry &&) = delete;
+
+  private:
+    InferenceLane &lane_;
+};
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/inference_lane_test.cpp

@@ -0,0 +1,618 @@
+// Unit tests for inference_lane. Standard library only. The harness compiles
+// this file as a single translation unit, so the implementation is included
+// directly rather than linked.
+//
+// Two kinds of test live here:
+//
+//   * The pure ones (budget negotiation, the overrun predicate) run with no
+//     threads and no sleeping. They carry the arithmetic, so they are the tests
+//     that must be exhaustive.
+//   * The threaded ones exercise the lane itself. Every one of them is bounded:
+//     contenders use a generous wait budget instead of the unbounded mode
+//     wherever the point of the test does not require unbounded, and a watchdog
+//     in main() puts a ceiling on the whole file. A broken implementation must
+//     go red, not hang, because a hung job costs CI far more than a red one.
+//
+// Wall-clock margins are called out individually. The rule applied throughout:
+// a margin is only allowed if a slow or loaded machine pushes the measurement
+// deeper into the passing region.
+
+#include "inference_lane.cpp"
+
+#include <atomic>
+#include <chrono>
+#include <condition_variable>
+#include <cstdio>
+#include <cstdlib>
+#include <mutex>
+#include <string>
+#include <thread>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using audiocpp_backend::InferenceLane;
+using audiocpp_backend::LaneEntry;
+using audiocpp_backend::LaneUnavailable;
+using audiocpp_backend::resolve_wait_budget_ms;
+using audiocpp_backend::run_exceeds_budget;
+
+// A one-shot, level-triggered signal with a bounded wait. Preferred over sleeps
+// for "the other thread got there" so the tests do not encode a guess about
+// scheduling.
+class Signal {
+  public:
+    void raise() {
+        {
+            std::lock_guard<std::mutex> lock(mutex_);
+            raised_ = true;
+        }
+        cv_.notify_all();
+    }
+
+    bool await(int timeout_ms) {
+        std::unique_lock<std::mutex> lock(mutex_);
+        return cv_.wait_for(lock, std::chrono::milliseconds(timeout_ms),
+                            [this] { return raised_; });
+    }
+
+  private:
+    std::mutex mutex_;
+    std::condition_variable cv_;
+    bool raised_ = false;
+};
+
+static std::int64_t elapsed_ms_since(
+    const std::chrono::steady_clock::time_point &start) {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+               std::chrono::steady_clock::now() - start)
+        .count();
+}
+
+static void nap(int ms) {
+    std::this_thread::sleep_for(std::chrono::milliseconds(ms));
+}
+
+static bool mentions(const std::string &haystack, const std::string &needle) {
+    return haystack.find(needle) != std::string::npos;
+}
+
+// Wording that separates the two failure modes. Kept here so a message reword
+// that erases the distinction breaks these tests loudly.
+static const char *const kTimedOutPhrase = "timed out after";
+static const char *const kStillRunningPhrase = "has been running for";
+
+// Attempts an entry and reports what happened, so the threaded tests can assert
+// on the message rather than only on the exception type.
+struct EntryOutcome {
+    bool acquired = false;
+    std::string message;
+    std::int64_t took_ms = 0;
+};
+
+static EntryOutcome try_entry(InferenceLane &lane, int budget_ms) {
+    EntryOutcome out;
+    const auto start = std::chrono::steady_clock::now();
+    try {
+        LaneEntry entry(lane, budget_ms);
+        out.acquired = true;
+    } catch (const LaneUnavailable &refused) {
+        out.message = refused.what();
+    }
+    out.took_ms = elapsed_ms_since(start);
+    return out;
+}
+
+// ---------------------------------------------------------------------------
+// B9: budget negotiation. Pure, no threads.
+// ---------------------------------------------------------------------------
+static void test_budget_negotiation() {
+    check(resolve_wait_budget_ms(0, 0) == 0,
+          "B9 no ceiling and no request hint is unbounded");
+    check(resolve_wait_budget_ms(-1, -1) == 0,
+          "B9 negative ceiling and negative hint is unbounded");
+
+    check(resolve_wait_budget_ms(5000, 0) == 5000,
+          "B9 absent hint yields the ceiling");
+    check(resolve_wait_budget_ms(5000, -250) == 5000,
+          "B9 negative hint yields the ceiling");
+
+    check(resolve_wait_budget_ms(5000, 1200) == 1200,
+          "B9 a shorter request hint is granted");
+    check(resolve_wait_budget_ms(5000, 1) == 1,
+          "B9 a much shorter request hint is granted");
+
+    check(resolve_wait_budget_ms(5000, 9000) == 5000,
+          "B9 a longer request hint cannot weaken the ceiling");
+    check(resolve_wait_budget_ms(5000, 5001) == 5000,
+          "B9 a hint one ms over the ceiling is clamped");
+    check(resolve_wait_budget_ms(5000, 5000) == 5000,
+          "B9 a hint equal to the ceiling is the ceiling");
+
+    check(resolve_wait_budget_ms(0, 1200) == 1200,
+          "B9 without a policy limit the request hint applies");
+    check(resolve_wait_budget_ms(-5, 1200) == 1200,
+          "B9 a negative ceiling is no policy limit");
+
+    check(resolve_wait_budget_ms(1, 0) == 1,
+          "B9 a one ms ceiling survives negotiation");
+}
+
+// ---------------------------------------------------------------------------
+// B5 and B6: the overrun predicate. Pure, no threads.
+// ---------------------------------------------------------------------------
+static void test_overrun_predicate() {
+    // B5: strictly longer.
+    check(run_exceeds_budget(true, 1000, 1100, 100) == false,
+          "B5 elapsed exactly equal to the budget is not an overrun");
+    check(run_exceeds_budget(true, 1000, 1101, 100) == true,
+          "B5 one ms past the budget is an overrun");
+    check(run_exceeds_budget(true, 1000, 1099, 100) == false,
+          "B5 one ms short of the budget is not an overrun");
+    check(run_exceeds_budget(true, 0, 1, 1) == false,
+          "B5 a one ms budget at one ms elapsed is not an overrun");
+    check(run_exceeds_budget(true, 0, 2, 1) == true,
+          "B5 a one ms budget at two ms elapsed is an overrun");
+
+    // B6: an unoccupied lane is never stuck, whatever the leftover timestamp
+    // says. This is the pure half of B6; the wiring half is threaded below.
+    check(run_exceeds_budget(false, 0, 10000000, 1) == false,
+          "B6 an idle lane with an ancient start timestamp is not an overrun");
+    check(run_exceeds_budget(false, 5, 5, 5) == false,
+          "B6 an idle lane is not an overrun at any elapsed value");
+
+    // An unbounded caller has no budget to exceed, so it never fails fast.
+    check(run_exceeds_budget(true, 0, 10000000, 0) == false,
+          "B2 an unbounded caller never sees an overrun");
+    check(run_exceeds_budget(true, 0, 10000000, -1) == false,
+          "B2 a negative budget never sees an overrun");
+}
+
+// ---------------------------------------------------------------------------
+// B1: mutual exclusion under real contention, and a release admits a waiter.
+// ---------------------------------------------------------------------------
+static void test_mutual_exclusion() {
+    InferenceLane lane("exclusion-model");
+
+    constexpr int kContenders = 4; // "at least three simultaneous contenders"
+    constexpr int kHoldMs = 15;
+    // Generous on purpose: the point of this test is exclusion, not timeouts.
+    // A larger budget only makes a healthy run more likely to pass, while still
+    // bounding a broken one at roughly five seconds instead of forever.
+    constexpr int kBudgetMs = 5000;
+
+    std::atomic<int> in_flight{0};
+    std::atomic<int> peak_in_flight{0};
+    std::atomic<int> completed{0};
+    std::atomic<int> refused{0};
+
+    Signal go;
+    std::vector<std::thread> contenders;
+    for (int i = 0; i < kContenders; i++) {
+        contenders.emplace_back([&] {
+            go.await(5000);
+            try {
+                LaneEntry entry(lane, kBudgetMs);
+                const int now_inside = in_flight.fetch_add(1) + 1;
+                int seen = peak_in_flight.load();
+                while (now_inside > seen &&
+                       !peak_in_flight.compare_exchange_weak(seen, now_inside)) {
+                    // retry with the refreshed value
+                }
+                nap(kHoldMs);
+                in_flight.fetch_sub(1);
+                completed.fetch_add(1);
+            } catch (const LaneUnavailable &) {
+                refused.fetch_add(1);
+            }
+        });
+    }
+    go.raise();
+    for (auto &t : contenders) {
+        t.join();
+    }
+
+    check(refused.load() == 0, "B1 no contender was refused within its budget");
+    check(completed.load() == kContenders,
+          "B1 every contender eventually got the lane");
+    check(peak_in_flight.load() == 1,
+          "B1 never more than one holder inside the lane at once");
+}
+
+// ---------------------------------------------------------------------------
+// B2: unbounded mode waits out a run longer than any bound would allow.
+// ---------------------------------------------------------------------------
+static void test_unbounded_waits_out_the_holder() {
+    InferenceLane lane("patient-model");
+    constexpr int kHoldMs = 300;
+
+    Signal held;
+    Signal patient_done;
+    std::int64_t patient_wait_ms = -1;
+    bool patient_acquired = false;
+
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        nap(kHoldMs);
+    });
+    check(held.await(5000), "B2 holder took the lane");
+
+    std::thread patient([&] {
+        const auto start = std::chrono::steady_clock::now();
+        try {
+            LaneEntry entry(lane, 0);
+            patient_acquired = true;
+        } catch (const LaneUnavailable &) {
+            patient_acquired = false;
+        }
+        patient_wait_ms = elapsed_ms_since(start);
+        patient_done.raise();
+    });
+
+    // Bound on the unbounded mode: a lane that never wakes its waiters goes red
+    // here instead of hanging in join(). The named failure is printed before the
+    // join, so even a hard hang leaves a diagnosis behind for the watchdog.
+    check(patient_done.await(10000), "B2 unbounded caller returned at all");
+    patient.join();
+    holder.join();
+
+    check(patient_acquired, "B2 unbounded caller acquired instead of failing");
+    // Margin: the holder holds for 300 ms, so the waiter must block for about
+    // that long. Asserting only half of it means a loaded machine, which makes
+    // the wait longer, drifts further into passing.
+    check(patient_wait_ms >= kHoldMs / 2,
+          "B2 unbounded caller actually waited for the in-flight run");
+}
+
+// ---------------------------------------------------------------------------
+// B3: a bounded caller that cannot get in gives up with the timeout wording.
+// ---------------------------------------------------------------------------
+static void test_bounded_wait_times_out() {
+    InferenceLane lane("impatient-model");
+    constexpr int kBudgetMs = 120;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B3 holder took the lane");
+
+    // The waiter arrives immediately, so the holder's elapsed time is far below
+    // the budget and the fail-fast path must not trigger here.
+    //
+    // Load-sensitive margin, and the tightest one in this file: what makes this
+    // the timeout path rather than the fail-fast path is the holder's age
+    // staying under 120 ms at the arrival check. All that sits between the
+    // holder's stamp and this call is one signal handover, microseconds against
+    // a 120 ms allowance, but unlike the other margins here load pushes this one
+    // toward failing rather than away from it. If it ever does flip, the symptom
+    // is the wording assertions below going red, not a hang, and the fix is a
+    // larger budget rather than a weaker assertion.
+    const EntryOutcome outcome = try_entry(lane, kBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!outcome.acquired, "B3 bounded caller did not acquire a held lane");
+    check(mentions(outcome.message, kTimedOutPhrase),
+          "B3 failure names the exhausted wait, not an overrunning run");
+    check(!mentions(outcome.message, kStillRunningPhrase),
+          "B3 failure is not worded as an overrun");
+    check(mentions(outcome.message, "impatient-model"),
+          "B3 failure names the model");
+    check(mentions(outcome.message, "120"),
+          "B3 failure reports the budget it waited out");
+    // Margin: wait_for cannot return before its deadline, so the true value is
+    // at least 120 ms and load only raises it. Asserting 100 leaves room for
+    // clock granularity while still catching an implementation that returns
+    // early without waiting.
+    check(outcome.took_ms >= 100, "B3 bounded caller waited out its budget");
+}
+
+// ---------------------------------------------------------------------------
+// B4: a caller whose budget is already exceeded fails at once.
+// ---------------------------------------------------------------------------
+static void test_fail_fast_against_a_long_run() {
+    InferenceLane lane("wedged-model");
+    constexpr int kBudgetMs = 200;
+    constexpr int kRunAgeMs = 400;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B4 holder took the lane");
+
+    // Margin: the arriving caller needs the holder's elapsed time to exceed
+    // 200 ms. Sleeping 400 ms means a loaded machine oversleeps and pushes the
+    // elapsed time further past the budget, never below it.
+    nap(kRunAgeMs);
+    const EntryOutcome outcome = try_entry(lane, kBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!outcome.acquired, "B4 caller did not acquire a long-running lane");
+    check(mentions(outcome.message, kStillRunningPhrase),
+          "B4 failure states the measured age of the in-flight run");
+    check(!mentions(outcome.message, kTimedOutPhrase),
+          "B4 failure is not worded as an exhausted wait");
+    check(mentions(outcome.message, "wedged-model"),
+          "B4 failure names the model");
+    // Margin: a fail-fast return takes microseconds. 150 ms of headroom under a
+    // 200 ms budget separates "returned at once" from "waited out the budget"
+    // by a wide enough gap that scheduler noise cannot close it. The message
+    // assertions above are the load-independent proof; this one pins the timing.
+    check(outcome.took_ms < 150, "B4 caller failed without waiting out its budget");
+}
+
+// ---------------------------------------------------------------------------
+// B6 wiring: an idle lane never looks stuck, however old the last run is.
+// ---------------------------------------------------------------------------
+static void test_idle_lane_is_never_stuck() {
+    InferenceLane lane("idle-model");
+
+    {
+        LaneEntry entry(lane, 0);
+    }
+    // Ages the leftover start timestamp well past the tiny budget used below.
+    // A longer sleep only makes a stale-timestamp bug more visible, so load
+    // helps this test rather than hurting it.
+    nap(80);
+
+    const EntryOutcome first = try_entry(lane, 20);
+    check(first.acquired, "B6 tiny budget still acquires an idle lane");
+
+    nap(80);
+    const EntryOutcome second = try_entry(lane, 1);
+    check(second.acquired, "B6 a one ms budget still acquires an idle lane");
+}
+
+// ---------------------------------------------------------------------------
+// B7: every ownership exit clears the busy state, including an exception
+// thrown from inside the guarded region.
+// ---------------------------------------------------------------------------
+static void test_release_on_exception() {
+    InferenceLane lane("throwing-model");
+
+    struct GuardedRegionFailure {};
+    bool propagated = false;
+    try {
+        LaneEntry entry(lane, 0);
+        throw GuardedRegionFailure{};
+    } catch (const GuardedRegionFailure &) {
+        propagated = true;
+    }
+    check(propagated, "B7 the guarded region's own exception propagated");
+
+    // If the throw had leaked the busy state, this tiny budget would fail.
+    const EntryOutcome after_throw = try_entry(lane, 20);
+    check(after_throw.acquired, "B7 lane is free after an exception unwound it");
+
+    // Same check for a holder that unwinds on another thread, which is the shape
+    // a gRPC handler failing mid-inference actually has.
+    Signal thrown;
+    std::thread unlucky([&] {
+        try {
+            LaneEntry entry(lane, 0);
+            throw GuardedRegionFailure{};
+        } catch (const GuardedRegionFailure &) {
+            thrown.raise();
+        }
+    });
+    check(thrown.await(5000), "B7 worker thread unwound its guarded region");
+    unlucky.join();
+
+    const EntryOutcome after_worker = try_entry(lane, 20);
+    check(after_worker.acquired,
+          "B7 lane is free after a worker thread unwound it");
+}
+
+// ---------------------------------------------------------------------------
+// B8: a waiter must not publish itself as the holder. If it did, its arrival
+// would restart the elapsed-time measurement and hide the real holder.
+//
+// Timeline, with the holder taking the lane at t0 and never letting go:
+//
+//   t0        holder acquires, elapsed measurement starts here and only here
+//   t0+100    waiter arrives with a 400 ms budget, blocks, and times out
+//   t0+500    late caller arrives with a 450 ms budget
+//
+// A correct lane measures 500 ms of holding at the late arrival, which is more
+// than 450, so the late caller fails fast. An implementation that let the
+// waiter stamp itself as holder measures only the 400 ms since the waiter
+// arrived, which is under 450, so the late caller would queue behind a stuck
+// run instead. The two paths are told apart by their wording.
+// ---------------------------------------------------------------------------
+static void test_waiter_does_not_become_the_holder() {
+    InferenceLane lane("stamp-model");
+    constexpr int kWaiterArrivesAfterMs = 100;
+    constexpr int kWaiterBudgetMs = 400;
+    constexpr int kLateBudgetMs = 450;
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B8 holder took the lane");
+
+    // Margin: the waiter must not fail fast on arrival, which needs the
+    // holder's elapsed time to stay under 400 ms. Arriving at 100 ms leaves
+    // 300 ms of slack, so oversleeping under load does not flip the path.
+    nap(kWaiterArrivesAfterMs);
+
+    EntryOutcome waiter_outcome;
+    std::thread waiter([&] { waiter_outcome = try_entry(lane, kWaiterBudgetMs); });
+    waiter.join();
+    check(!waiter_outcome.acquired, "B8 mid-queue waiter did not acquire");
+    check(mentions(waiter_outcome.message, kTimedOutPhrase),
+          "B8 mid-queue waiter waited out its budget and timed out");
+
+    // Margin: the holder has now been in the lane for at least 500 ms against a
+    // 450 ms budget. Load lengthens both sleeps, so the measured age only grows
+    // and the fail-fast path only becomes more certain.
+    const EntryOutcome late = try_entry(lane, kLateBudgetMs);
+    release.raise();
+    holder.join();
+
+    check(!late.acquired, "B8 late caller did not acquire");
+    check(mentions(late.message, kStillRunningPhrase),
+          "B8 elapsed time is still measured from the real holder's acquisition");
+    check(late.took_ms < 200,
+          "B8 late caller failed fast rather than queueing behind the holder");
+}
+
+// ---------------------------------------------------------------------------
+// B8, other direction: the age of a run is measured from the moment its holder
+// acquired, not from the moment that holder arrived. A caller that queued for a
+// while and then got in is starting a fresh run, and its time in the queue must
+// not be billed to it: if it were, every handover would hand the new holder a
+// head start towards looking overrun, and short-budget callers would be turned
+// away from a run that has barely begun.
+//
+//   t0        first holder acquires and holds for 300 ms
+//   t0        second caller arrives and queues
+//   t0+300    second caller acquires, so its own run age is ~0 here
+//   t0+300    a third caller arrives with a 300 ms budget
+//
+// Correct: the third caller sees a run that just started, so it queues and then
+// times out. Billing the queue time to the second caller would show a 300 ms old
+// run instead, and the third caller would be turned away as an overrun.
+// ---------------------------------------------------------------------------
+static void test_run_age_starts_at_acquisition() {
+    InferenceLane lane("handover-model");
+    constexpr int kFirstHoldMs = 300;
+    constexpr int kThirdBudgetMs = 200;
+
+    Signal first_held;
+    Signal handed_over;
+    Signal release_second;
+
+    std::thread first([&] {
+        LaneEntry entry(lane, 0);
+        first_held.raise();
+        nap(kFirstHoldMs);
+    });
+    // Ordering matters only for the diagnosis, not for the assertion: waiting
+    // for the first holder guarantees the second caller really does queue, which
+    // is what gives it queue time to be wrongly billed for.
+    check(first_held.await(5000), "B8 first holder took the lane");
+
+    std::thread second([&] {
+        LaneEntry entry(lane, 0);
+        handed_over.raise();
+        release_second.await(10000);
+    });
+    check(handed_over.await(10000), "B8 queued caller was handed the lane");
+
+    // Margin: the new holder's run is a few ms old against a 200 ms budget, so
+    // this caller must queue. Load can only add a few ms of handover latency,
+    // well inside that slack, while it lengthens the queue time that the buggy
+    // version would bill, making the bug more visible rather than less.
+    const EntryOutcome third = try_entry(lane, kThirdBudgetMs);
+    release_second.raise();
+    second.join();
+    first.join();
+
+    check(!third.acquired, "B8 lane was still held by the queued caller");
+    check(mentions(third.message, kTimedOutPhrase),
+          "B8 a fresh holder's run age excludes the time it spent queueing");
+}
+
+// ---------------------------------------------------------------------------
+// B10: both failure modes carry a usable message, and the fail-fast one reports
+// a measurement rather than diagnosing a cause.
+// ---------------------------------------------------------------------------
+static void test_failure_messages_are_diagnosable() {
+    InferenceLane lane("diagnosable-model");
+
+    Signal held;
+    Signal release;
+    std::thread holder([&] {
+        LaneEntry entry(lane, 0);
+        held.raise();
+        release.await(10000);
+    });
+    check(held.await(5000), "B10 holder took the lane");
+
+    // The two budgets have to straddle the run's age, or both callers take the
+    // same path and the comparisons below are between two fail-fast messages
+    // that differ only in the budget they print.
+    //
+    // Margin: 30 ms is far under the ~120 ms age, and load only ages the run
+    // further, so `fast` fails fast. 400 ms is far over it, with the same
+    // slack and the same safe direction as the B3 test, so `slow` queues and
+    // then times out.
+    nap(120);
+    const EntryOutcome fast = try_entry(lane, 30);
+    const EntryOutcome slow = try_entry(lane, 400);
+    release.raise();
+    holder.join();
+
+    check(mentions(fast.message, kStillRunningPhrase) &&
+              mentions(slow.message, kTimedOutPhrase),
+          "B10 one caller took the fail-fast path and the other timed out");
+    check(fast.message != slow.message,
+          "B10 the two failure modes do not share one message");
+    check(!fast.message.empty() && !slow.message.empty(),
+          "B10 both failures carry text");
+    check(mentions(fast.message, "diagnosable-model") &&
+              mentions(slow.message, "diagnosable-model"),
+          "B10 both failures name the model");
+    // The fail-fast wording must not accuse the run of being stuck: a short
+    // budget meeting a legitimately long run reaches this path too.
+    for (const char *verdict : {"stuck", "wedged", "hung", "deadlock"}) {
+        check(!mentions(fast.message, verdict),
+              std::string("B10 fail-fast message avoids diagnosing '") +
+                  verdict + "'");
+    }
+}
+
+int main() {
+    // Last resort only. Every threaded test above is individually bounded, so
+    // this should never fire; it exists so that an implementation which parks a
+    // thread forever still ends the job instead of occupying a CI runner.
+    std::thread watchdog([] {
+        std::this_thread::sleep_for(std::chrono::seconds(30));
+        fprintf(stderr, "FAIL: watchdog fired, an inference_lane test hung\n");
+        fflush(stderr);
+        std::_Exit(1);
+    });
+    watchdog.detach();
+
+    test_budget_negotiation();
+    test_overrun_predicate();
+    test_mutual_exclusion();
+    test_unbounded_waits_out_the_holder();
+    test_bounded_wait_times_out();
+    test_fail_fast_against_a_long_run();
+    test_idle_lane_is_never_stuck();
+    test_release_on_exception();
+    test_waiter_does_not_become_the_holder();
+    test_run_age_starts_at_acquisition();
+    test_failure_messages_are_diagnosable();
+
+    if (failures == 0) {
+        fprintf(stderr, "\nAll inference_lane tests passed.\n");
+        return 0;
+    }
+    fprintf(stderr, "\n%d inference_lane test(s) failed.\n", failures);
+    return 1;
+}

backend/cpp/audio-cpp/live_watchdog.cpp

@@ -0,0 +1,75 @@
+#include "live_watchdog.h"
+
+#include <utility>
+
+namespace audiocpp_backend {
+
+IdleWatchdog::IdleWatchdog(std::chrono::milliseconds window,
+                           std::function<void()> on_idle)
+    : window_(window), on_idle_(std::move(on_idle)),
+      last_(std::chrono::steady_clock::now()) {
+    if (window_.count() <= 0) {
+        // Disabled: no thread at all, rather than a thread with an infinite
+        // deadline. A thread that exists is a thread that has to be joined on
+        // every exit path, and there is nothing for this one to do.
+        return;
+    }
+    thread_ = std::thread([this] { run(); });
+}
+
+IdleWatchdog::~IdleWatchdog() { disarm(); }
+
+void IdleWatchdog::touch() {
+    std::lock_guard<std::mutex> lock(mu_);
+    last_ = std::chrono::steady_clock::now();
+    // Deliberately does NOT notify. The waiter recomputes its deadline from
+    // last_ every time it wakes, so a touch that lands mid-window is picked up
+    // when the old deadline expires, and a touch is the hot path: it runs once
+    // per frame on the wire.
+}
+
+void IdleWatchdog::disarm() {
+    {
+        std::lock_guard<std::mutex> lock(mu_);
+        stop_ = true;
+    }
+    cv_.notify_all();
+    if (thread_.joinable()) {
+        thread_.join();
+    }
+}
+
+bool IdleWatchdog::fired() const {
+    std::lock_guard<std::mutex> lock(mu_);
+    return fired_;
+}
+
+void IdleWatchdog::run() {
+    std::unique_lock<std::mutex> lock(mu_);
+    while (!stop_) {
+        const auto deadline = last_ + window_;
+        if (cv_.wait_until(lock, deadline, [this] { return stop_; })) {
+            return; // disarmed
+        }
+        // The deadline passed, but last_ may have moved while this thread was
+        // waiting, and a condition variable may also wake spuriously. Re-read
+        // it: without this check a touch that landed mid-window would still be
+        // followed by a cancellation, i.e. a live client cut off mid-sentence.
+        if (std::chrono::steady_clock::now() < last_ + window_) {
+            continue;
+        }
+        fired_ = true;
+        auto callback = on_idle_;
+        lock.unlock();
+        if (callback) {
+            callback();
+        }
+        return; // one shot
+    }
+}
+
+bool live_frame_carries_audio(bool has_audio, bool pcm_empty) {
+    return has_audio && !pcm_empty;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/live_watchdog.h

@@ -0,0 +1,99 @@
+#pragma once
+
+// A one-shot idle timer for a bidirectional stream. Standard library only, so
+// it is tested without an audio.cpp checkout or a gRPC server.
+//
+// WHY IT EXISTS. AudioTranscriptionLive holds the model's inference lane for the
+// whole stream, because the streaming session is stateful and a concurrent run
+// would interleave two callers' audio. Every other RPC in this backend holds the
+// lane across COMPUTE, or across a write to a slow reader, and both of those
+// terminate on their own. A live stream instead blocks in a client-driven read,
+// and a peer that goes silent WITHOUT closing the stream never terminates
+// anything: the lane stays taken and every other RPC against that model queues
+// behind a client that has stopped speaking. A websocket death does cancel the
+// RPC and free it, but "the peer's TCP connection eventually dies" is not a
+// bound anyone can state, so this supplies one.
+//
+// HOW IT ENDS THE STREAM, and the part that is not obvious: gRPC's synchronous
+// ServerReaderWriter::Read has no timeout and cannot be given one. The only way
+// to unblock it from another thread is ServerContext::TryCancel, which is what
+// the callback is for. That means the client sees CANCELLED rather than whatever
+// status the handler goes on to return: the returned status is for the server's
+// own record. Releasing the lane is the point.
+//
+// ONE SHOT on purpose. Once the callback has run the stream is being torn down,
+// so there is nothing left to watch, and a repeating timer would call TryCancel
+// on a context the handler may already have returned from.
+
+#include <chrono>
+#include <condition_variable>
+#include <functional>
+#include <mutex>
+#include <thread>
+
+namespace audiocpp_backend {
+
+class IdleWatchdog {
+public:
+    // A window that is not positive DISABLES the watchdog entirely: no thread is
+    // started and fired() never becomes true. That is the operator's escape
+    // hatch for a client that legitimately holds a stream open through long
+    // pauses, and it is why the option carrying it documents 0 as "no limit"
+    // rather than as "expire immediately".
+    //
+    // `on_idle` runs on the watchdog's own thread with no lock held. It must be
+    // safe to call while the watched thread is blocked in a read, which is the
+    // only reason this class exists; ServerContext::TryCancel is documented as
+    // exactly that.
+    IdleWatchdog(std::chrono::milliseconds window, std::function<void()> on_idle);
+
+    // Joins the thread, so the callback can safely capture anything that
+    // outlives this object's scope and nothing else has to be reasoned about.
+    ~IdleWatchdog();
+
+    IdleWatchdog(const IdleWatchdog &) = delete;
+    IdleWatchdog &operator=(const IdleWatchdog &) = delete;
+
+    // Restarts the window. Call it whenever the peer proves it is still there.
+    void touch();
+
+    // Stops watching and joins. Idempotent, and REQUIRED before any long
+    // non-read work the window must not cover: the caller's own decode is not
+    // the peer going quiet, and cancelling in the middle of it would throw away
+    // a transcript the client is waiting for.
+    void disarm();
+
+    // True once the window elapsed and the callback ran. Stays true after
+    // disarm, so the caller can tell "the peer closed" from "we cancelled it".
+    bool fired() const;
+
+private:
+    void run();
+
+    const std::chrono::milliseconds window_;
+    std::function<void()> on_idle_;
+
+    mutable std::mutex mu_;
+    std::condition_variable cv_;
+    std::chrono::steady_clock::time_point last_;
+    bool stop_ = false;
+    bool fired_ = false;
+    std::thread thread_;
+};
+
+// Whether one message read off a live stream is a frame the decoder can
+// actually consume, which is the ONLY thing that counts as the peer proving it
+// is still there.
+//
+// Split out of the read loop so the distinction is testable, and because
+// getting it wrong is silent. The loop used to touch the watchdog on ANY
+// message, before it filtered on has_audio and on an empty pcm field, so a peer
+// writing unset-oneof or zero-length frames faster than the window held the
+// lane forever: no audio was ever fed, no work was ever done, and the timer
+// that exists to break exactly that grip was reset by the frames doing it.
+// There is one lane per model and one model per process, so that is a single
+// client denying the whole backend. The thrown message already said "no audio
+// frame arrived"; this is the code agreeing with it.
+bool live_frame_carries_audio(bool has_audio, bool pcm_empty);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/live_watchdog_test.cpp

@@ -0,0 +1,225 @@
+// Unit tests for the live stream idle watchdog. Standard library only; the
+// harness compiles this as a single translation unit, so the implementation is
+// included directly.
+//
+// These are TIMING tests, which is unavoidable: what is under test is a
+// deadline. Every window here is short and every assertion waits several
+// multiples of it, so a loaded machine slows the test down rather than
+// flipping its answer. The one thing never asserted is how SOON something
+// happens, only that it eventually does or never does.
+
+#include "live_watchdog.cpp"
+
+#include <atomic>
+#include <cstdio>
+#include <string>
+#include <thread>
+
+using namespace std::chrono_literals;
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+// A peer that goes quiet without closing. This is the whole point: the lane it
+// holds has to come back.
+static void test_it_fires_when_nothing_touches_it() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(100ms, [&calls] { ++calls; });
+    std::this_thread::sleep_for(600ms);
+    check(watchdog.fired(), "a window that elapses untouched fires");
+    check(calls.load() == 1, "the callback runs exactly once, not once per window");
+}
+
+// A peer that is still speaking must never be cut off. Touches land at a third
+// of the window, for six windows' worth of wall clock.
+static void test_touching_defers_it_indefinitely() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(300ms, [&calls] { ++calls; });
+    for (int i = 0; i < 20; ++i) {
+        std::this_thread::sleep_for(100ms);
+        watchdog.touch();
+    }
+    check(!watchdog.fired(),
+          "a stream touched inside every window is never cancelled");
+    check(calls.load() == 0, "no callback runs while the peer is still there");
+}
+
+// Disarm is what the handler calls when the read side closes, before a decode
+// that can take longer than the window. Firing after that would throw away the
+// transcript the client is waiting for.
+static void test_disarm_stops_it_before_the_window() {
+    std::atomic<int> calls{0};
+    audiocpp_backend::IdleWatchdog watchdog(200ms, [&calls] { ++calls; });
+    std::this_thread::sleep_for(20ms);
+    watchdog.disarm();
+    std::this_thread::sleep_for(600ms);
+    check(!watchdog.fired(), "a disarmed watchdog does not fire");
+    check(calls.load() == 0, "a disarmed watchdog runs no callback");
+}
+
+static void test_disarm_is_idempotent() {
+    audiocpp_backend::IdleWatchdog watchdog(50ms, [] {});
+    watchdog.disarm();
+    watchdog.disarm();
+    watchdog.disarm();
+    check(true, "disarming three times joins once and does not abort");
+}
+
+// The operator's escape hatch, for a client that legitimately holds a stream
+// open through long pauses. Not "expire immediately", which is what a naive
+// reading of a zero timeout would give.
+static void test_a_non_positive_window_disables_it() {
+    std::atomic<int> calls{0};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(0ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(300ms);
+        check(!watchdog.fired(), "a zero window never fires");
+    }
+    {
+        audiocpp_backend::IdleWatchdog watchdog(-5ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(300ms);
+        check(!watchdog.fired(), "a negative window never fires");
+    }
+    check(calls.load() == 0, "a disabled watchdog runs no callback");
+}
+
+// fired() has to survive the disarm, because the handler reads it AFTER the
+// read loop ends to tell "the peer closed" from "we cancelled the peer", and
+// those two get different statuses.
+static void test_fired_survives_a_later_disarm() {
+    audiocpp_backend::IdleWatchdog watchdog(80ms, [] {});
+    std::this_thread::sleep_for(500ms);
+    watchdog.disarm();
+    check(watchdog.fired(), "a watchdog that fired still says so after disarm");
+}
+
+// The destructor joins, so a callback capturing the handler's frame cannot run
+// after that frame is gone. Without the join this is a use after free that only
+// shows up under load.
+//
+// The window is LONGER than the scope on purpose. An earlier version of this
+// test slept past the window inside the scope, so the callback had already run
+// by the time the object was destroyed and a destructor that DETACHED the thread
+// instead of joining it passed unnoticed. Mutation testing is what found that;
+// the shape below kills it, because a detached thread wakes after the object is
+// gone and calls a callback that must never run.
+static void test_the_destructor_joins() {
+    std::atomic<int> calls{0};
+    std::atomic<bool> alive{true};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(200ms, [&calls, &alive] {
+            check(alive.load(),
+                  "the callback never runs after the watched scope ended");
+            ++calls;
+        });
+        std::this_thread::sleep_for(20ms);
+    }
+    alive.store(false);
+    std::this_thread::sleep_for(600ms);
+    check(calls.load() == 0,
+          "destruction stops the timer rather than leaving it running against a "
+          "dead frame");
+}
+
+// The other half of that pair: a callback that DOES fire inside the scope runs
+// exactly once, so the test above is not passing merely because nothing ever
+// fires.
+static void test_a_firing_watchdog_still_joins_cleanly() {
+    std::atomic<int> calls{0};
+    {
+        audiocpp_backend::IdleWatchdog watchdog(50ms, [&calls] { ++calls; });
+        std::this_thread::sleep_for(400ms);
+    }
+    check(calls.load() == 1, "the callback ran once, inside the scope");
+}
+
+// The predicate the live read loop filters on.
+static void test_only_a_frame_with_audio_counts() {
+    using audiocpp_backend::live_frame_carries_audio;
+    check(live_frame_carries_audio(true, false),
+          "a frame with a non-empty pcm field carries audio");
+    check(!live_frame_carries_audio(true, true),
+          "an empty pcm field does not");
+    check(!live_frame_carries_audio(false, false),
+          "an unset audio oneof does not, whatever the pcm field looks like");
+    check(!live_frame_carries_audio(false, true),
+          "and neither does an unset oneof with an empty pcm field");
+}
+
+// The defect this closes, expressed as behaviour rather than as a call order:
+// a peer writing frames the decoder cannot consume, faster than the window,
+// used to hold the model's only inference lane forever, because the read loop
+// touched the watchdog before it filtered them out. One lane per model and one
+// model per process, so that is a single client denying the whole backend,
+// which is exactly what the watchdog exists to prevent.
+static void test_empty_frames_do_not_hold_the_lane() {
+    std::atomic<int> cancels{0};
+    std::atomic<bool> stop{false};
+    audiocpp_backend::IdleWatchdog watchdog(80ms, [&cancels] { ++cancels; });
+
+    // The read loop with the real filter in it: frames arrive continuously,
+    // none of them carries audio, and only a frame that does may touch.
+    std::thread peer([&] {
+        while (!stop.load()) {
+            if (audiocpp_backend::live_frame_carries_audio(false, true)) {
+                watchdog.touch();
+            }
+            std::this_thread::sleep_for(5ms);
+        }
+    });
+
+    std::this_thread::sleep_for(600ms);
+    stop.store(true);
+    peer.join();
+    watchdog.disarm();
+
+    check(cancels.load() == 1,
+          "a flood of frames with no audio in them still releases the lane");
+
+    // The mirror image, so this cannot pass merely because the watchdog always
+    // fires: a peer that keeps sending audio is left alone, exactly as before.
+    std::atomic<int> live_cancels{0};
+    std::atomic<bool> live_stop{false};
+    audiocpp_backend::IdleWatchdog live(80ms, [&live_cancels] { ++live_cancels; });
+    std::thread speaker([&] {
+        while (!live_stop.load()) {
+            if (audiocpp_backend::live_frame_carries_audio(true, false)) {
+                live.touch();
+            }
+            std::this_thread::sleep_for(5ms);
+        }
+    });
+    std::this_thread::sleep_for(600ms);
+    live_stop.store(true);
+    speaker.join();
+    live.disarm();
+    check(live_cancels.load() == 0,
+          "a peer that keeps sending audio is never cancelled");
+}
+
+int main() {
+    test_it_fires_when_nothing_touches_it();
+    test_touching_defers_it_indefinitely();
+    test_disarm_stops_it_before_the_window();
+    test_disarm_is_idempotent();
+    test_a_non_positive_window_disables_it();
+    test_fired_survives_a_later_disarm();
+    test_the_destructor_joins();
+    test_a_firing_watchdog_still_joins_cleanly();
+    test_only_a_frame_with_audio_counts();
+    test_empty_frames_do_not_hold_the_lane();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all live_watchdog checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/loaded_model.cpp

@@ -0,0 +1,802 @@
+#include "loaded_model.h"
+
+#include "family_gate.h"
+
+#include "engine/framework/assets/tensor_source.h"
+
+#include <algorithm>
+#include <cstddef>
+#include <cstdint>
+#include <filesystem>
+#include <utility>
+
+#if defined(__APPLE__)
+#include <mach-o/dyld.h>
+#include <cstdint>
+#include <vector>
+#elif defined(__linux__)
+#include <unistd.h>
+#endif
+
+namespace audiocpp_backend {
+
+// --------------------------------------------------------------------------
+// Enum coupling
+//
+// audiocpp_backend::Task mirrors engine::runtime::VoiceTaskKind positionally so
+// capability_routing can stay stdlib-only and testable without an audio.cpp
+// checkout. Nothing about that mirroring is enforced by the type system, and a
+// drift is silent in the worst possible way: every unit still compiles, every
+// test still passes, and the backend runs a different task than the one the
+// caller asked for.
+//
+// Two mechanisms pin it, and both are needed because they catch different edits:
+//
+//   1. The assertions below pin every enumerator's value on both sides. An
+//      insertion or a reorder anywhere before the last member shifts the values
+//      after it and fails the build here.
+//   2. An enumerator APPENDED after the last one shifts nothing, so no value
+//      assertion can see it. What sees it is the switch in from_engine_task,
+//      which covers the engine enum with no `default:` label. CMakeLists.txt
+//      compiles this file with -Werror=switch so that omission is an error
+//      rather than a warning nobody reads.
+//
+// Neither mechanism catches a pure RENAME of an upstream enumerator, but that
+// does not need catching: the switch stops naming an enumerator that exists and
+// the build fails on its own.
+// --------------------------------------------------------------------------
+
+namespace {
+
+constexpr int kEngine(engine::runtime::VoiceTaskKind kind) {
+    return static_cast<int>(kind);
+}
+constexpr int kMirror(Task task) { return static_cast<int>(task); }
+
+} // namespace
+
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Vad) == 0, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Asr) == 1, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Diarization) == 2, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SourceSeparation) == 3, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::AudioGeneration) == 4, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Tts) == 5, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceCloning) == 6, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceConversion) == 7, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SpeechToSpeech) == 8, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Alignment) == 9, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::VoiceDesign) == 10, "VoiceTaskKind drifted");
+static_assert(kEngine(engine::runtime::VoiceTaskKind::SpeakerRecognition) == 11, "VoiceTaskKind drifted");
+// The last member. Pinning it pins the member count too, as long as the
+// enumerators stay contiguous and unassigned, which upstream's declaration is.
+static_assert(kEngine(engine::runtime::VoiceTaskKind::Svc) == 12,
+              "engine::runtime::VoiceTaskKind gained, lost or reordered a member. "
+              "audiocpp_backend::Task mirrors it positionally: update capability_routing.h, "
+              "to_engine_task and from_engine_task together, then move this pin.");
+
+static_assert(kMirror(Task::Vad) == 0, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Asr) == 1, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Diarization) == 2, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SourceSeparation) == 3, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::AudioGeneration) == 4, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Tts) == 5, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceCloning) == 6, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceConversion) == 7, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SpeechToSpeech) == 8, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Alignment) == 9, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::VoiceDesign) == 10, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::SpeakerRecognition) == 11, "Task drifted from VoiceTaskKind");
+static_assert(kMirror(Task::Svc) == 12, "Task drifted from VoiceTaskKind");
+
+static_assert(static_cast<int>(engine::runtime::RunMode::Offline) == 0, "RunMode drifted");
+static_assert(static_cast<int>(engine::runtime::RunMode::Streaming) == 1,
+              "engine::runtime::RunMode gained, lost or reordered a member. "
+              "audiocpp_backend::Mode mirrors it positionally.");
+static_assert(static_cast<int>(Mode::Offline) == 0, "Mode drifted from RunMode");
+static_assert(static_cast<int>(Mode::Streaming) == 1, "Mode drifted from RunMode");
+
+namespace {
+
+engine::core::BackendType parse_backend_type(const std::string &value) {
+    if (value == "cuda") {
+        return engine::core::BackendType::Cuda;
+    }
+    if (value == "vulkan") {
+        return engine::core::BackendType::Vulkan;
+    }
+    if (value == "metal") {
+        return engine::core::BackendType::Metal;
+    }
+    if (value == "best") {
+        return engine::core::BackendType::BestAvailable;
+    }
+    if (value == "cpu" || value.empty()) {
+        return engine::core::BackendType::Cpu;
+    }
+    throw ConfigError("audio-cpp: unknown backend option '" + value +
+                      "'. Known backends: cpu, cuda, vulkan, metal, best");
+}
+
+std::filesystem::path executable_directory() {
+#if defined(__APPLE__)
+    std::uint32_t size = 0;
+    _NSGetExecutablePath(nullptr, &size);
+    std::vector<char> buffer(size + 1, '\0');
+    if (_NSGetExecutablePath(buffer.data(), &size) != 0) {
+        return std::filesystem::current_path();
+    }
+    return std::filesystem::path(buffer.data()).parent_path();
+#elif defined(__linux__)
+    std::error_code ec;
+    const auto self = std::filesystem::read_symlink("/proc/self/exe", ec);
+    if (ec) {
+        return std::filesystem::current_path();
+    }
+    return self.parent_path();
+#else
+    return std::filesystem::current_path();
+#endif
+}
+
+// Runs the load gate. Throws ConfigError rather than returning a decision,
+// because the only caller is a delegating constructor whose member initializer
+// list has nowhere to put a failure.
+std::string require_family(const std::string &resolved_path,
+                           const ModelOptions &options) {
+    const std::filesystem::path path(resolved_path);
+    std::error_code ec;
+    if (!std::filesystem::exists(path, ec)) {
+        throw ConfigError("audio-cpp: model path does not exist: " + resolved_path);
+    }
+
+    const bool is_gguf = path_looks_like_gguf(resolved_path);
+    // Only a GGUF is asked for embedded metadata. A directory has no single
+    // file to read it from, and probing one would make the gate's refusal
+    // depend on which file happened to be inside.
+    const std::string embedded =
+        is_gguf ? read_gguf_family(resolved_path) : std::string();
+
+    const FamilyDecision decision =
+        decide_family(is_gguf, embedded, options.family);
+    if (!decision.ok) {
+        throw ConfigError(decision.error);
+    }
+    return decision.family;
+}
+
+// Refuses a GGUF whose weights are stored in a dtype the family cannot survive.
+//
+// The POLICY lives in family_gate's weight_dtype_is_supported, which is
+// stdlib-only and therefore testable; this is only the part that needs a file
+// and an engine to read one. See the table there for why an entry exists and
+// what has to be run before deleting it.
+//
+// Only GGUF paths are inspected. A directory of safetensors carries its dtypes
+// per file and has not been tested against this failure, so it is passed
+// through rather than guessed at.
+void require_supported_weight_dtypes(const std::string &family,
+                                     const std::string &resolved_path) {
+    // Asked as "is there an entry", not as "is the description non-empty": an
+    // entry with an empty allow list describes a family that can run nothing,
+    // and reading the description would skip the check on exactly that entry
+    // while weight_dtype_is_supported refused every dtype. No such entry exists
+    // today; the two questions are different ones and only one of them is this
+    // guard's.
+    if (!family_has_weight_dtype_allow_list(family) ||
+        !path_looks_like_gguf(resolved_path)) {
+        return;
+    }
+
+    std::string offending_dtype;
+    std::string offending_tensor;
+    try {
+        const auto source =
+            engine::assets::open_tensor_source(std::filesystem::path(resolved_path));
+        if (source == nullptr) {
+            return;
+        }
+        for (const auto &tensor : source->tensors()) {
+            if (!weight_dtype_is_supported(family, tensor.dtype)) {
+                offending_dtype = tensor.dtype;
+                offending_tensor = tensor.name;
+                break;
+            }
+        }
+    } catch (const std::exception &) {
+        // Unreadable as a tensor source. Not this guard's problem to report:
+        // the registry load below produces a message naming the real fault, and
+        // refusing here would turn every unusual packaging into this error.
+        return;
+    }
+
+    if (offending_dtype.empty()) {
+        return;
+    }
+    throw ConfigError(
+        "audio-cpp: family '" + family + "' cannot run weights stored as '" +
+        offending_dtype + "' (tensor '" + offending_tensor + "' in " +
+        resolved_path +
+        "); it aborts the backend process on the first request rather than "
+        "failing the request. Use the 'orig' GGUF package, whose weights are " +
+        supported_weight_dtypes(family) + ".");
+}
+
+} // namespace
+
+engine::runtime::VoiceTaskKind to_engine_task(Task task) {
+    using K = engine::runtime::VoiceTaskKind;
+    // An explicit switch, never a cast: a cast would keep compiling through
+    // exactly the drift the assertions above exist to catch.
+    switch (task) {
+    case Task::Vad:                return K::Vad;
+    case Task::Asr:                return K::Asr;
+    case Task::Diarization:        return K::Diarization;
+    case Task::SourceSeparation:   return K::SourceSeparation;
+    case Task::AudioGeneration:    return K::AudioGeneration;
+    case Task::Tts:                return K::Tts;
+    case Task::VoiceCloning:       return K::VoiceCloning;
+    case Task::VoiceConversion:    return K::VoiceConversion;
+    case Task::SpeechToSpeech:     return K::SpeechToSpeech;
+    case Task::Alignment:          return K::Alignment;
+    case Task::VoiceDesign:        return K::VoiceDesign;
+    case Task::SpeakerRecognition: return K::SpeakerRecognition;
+    case Task::Svc:                return K::Svc;
+    }
+    // Unreachable for any valid enumerator. No `default:` label, so -Wswitch
+    // still reports a member this switch stops covering.
+    return K::Vad;
+}
+
+Task from_engine_task(engine::runtime::VoiceTaskKind kind) {
+    using K = engine::runtime::VoiceTaskKind;
+    switch (kind) {
+    case K::Vad:                return Task::Vad;
+    case K::Asr:                return Task::Asr;
+    case K::Diarization:        return Task::Diarization;
+    case K::SourceSeparation:   return Task::SourceSeparation;
+    case K::AudioGeneration:    return Task::AudioGeneration;
+    case K::Tts:                return Task::Tts;
+    case K::VoiceCloning:       return Task::VoiceCloning;
+    case K::VoiceConversion:    return Task::VoiceConversion;
+    case K::SpeechToSpeech:     return Task::SpeechToSpeech;
+    case K::Alignment:          return Task::Alignment;
+    case K::VoiceDesign:        return Task::VoiceDesign;
+    case K::SpeakerRecognition: return Task::SpeakerRecognition;
+    case K::Svc:                return Task::Svc;
+    }
+    return Task::Vad;
+}
+
+engine::runtime::RunMode to_engine_mode(Mode mode) {
+    using M = engine::runtime::RunMode;
+    switch (mode) {
+    case Mode::Offline:   return M::Offline;
+    case Mode::Streaming: return M::Streaming;
+    }
+    return M::Offline;
+}
+
+Mode from_engine_mode(engine::runtime::RunMode mode) {
+    using M = engine::runtime::RunMode;
+    switch (mode) {
+    case M::Offline:   return Mode::Offline;
+    case M::Streaming: return Mode::Streaming;
+    }
+    return Mode::Offline;
+}
+
+Capabilities to_capabilities(const std::string &family,
+                             const engine::runtime::CapabilitySet &set) {
+    Capabilities caps;
+    caps.family = family;
+    caps.tasks.reserve(set.supported_tasks.size());
+    for (const auto &supported : set.supported_tasks) {
+        TaskCapability capability;
+        capability.task = from_engine_task(supported.task);
+        capability.modes.reserve(supported.modes.size());
+        for (const auto mode : supported.modes) {
+            capability.modes.push_back(from_engine_mode(mode));
+        }
+        caps.tasks.push_back(std::move(capability));
+    }
+    return caps;
+}
+
+std::string read_gguf_family(const std::string &path) {
+    try {
+        const auto spec =
+            engine::assets::read_gguf_embedded_model_spec(std::filesystem::path(path));
+        if (spec.has_value()) {
+            return spec->family;
+        }
+    } catch (...) {
+        // A file that is not a readable GGUF simply has no family. The load
+        // gate turns that into a clear refusal; a throw here would surface as
+        // an opaque internal error during backend probing.
+    }
+    return {};
+}
+
+std::string resolve_model_path(const std::string &model_path_dir,
+                               const std::string &model_file,
+                               const std::string &model_name) {
+    const std::string candidate = !model_file.empty() ? model_file : model_name;
+
+    // The bundled: form is looked for in BOTH fields, and in ModelOptions.Model
+    // FIRST, because that is the only field it survives in. LocalAI fills
+    // ModelFile by joining ModelPath onto the configured model string
+    // (pkg/model/loader.go, LoadModelWithFile), so a model YAML saying
+    // `model: bundled:silero_vad` arrives here as ModelFile
+    // "/models/bundled:silero_vad" and Model "bundled:silero_vad". Testing
+    // `candidate` alone therefore made the zero-download VAD path reachable only
+    // from a hand-written LoadModel call that left ModelFile empty, and every
+    // model YAML using it failed with "model path does not exist".
+    const std::string bundled_prefix = "bundled:";
+    for (const std::string *field : {&model_name, &model_file}) {
+        if (field->rfind(bundled_prefix, 0) == 0) {
+            const std::string name = field->substr(bundled_prefix.size());
+            return (executable_directory() / "assets" / name).string();
+        }
+    }
+
+    std::filesystem::path path(candidate);
+    if (path.is_absolute() || model_path_dir.empty()) {
+        return path.string();
+    }
+    return (std::filesystem::path(model_path_dir) / path).string();
+}
+
+LoadedModel::LoadedModel(const std::string &resolved_path,
+                         const ModelOptions &options,
+                         std::string model_identity)
+    : LoadedModel(resolved_path, options, require_family(resolved_path, options),
+                  std::move(model_identity)) {}
+
+LoadedModel::LoadedModel(const std::string &resolved_path,
+                         const ModelOptions &options, std::string family,
+                         std::string model_identity)
+    : lane_(family), registry_(engine::runtime::make_default_registry()),
+      identity_(std::move(model_identity)) {
+    if (!registry_.supports_family(family)) {
+        throw ConfigError("audio-cpp: unknown audio.cpp family '" + family + "'");
+    }
+
+    // Before the load, for the same reason parse_backend_type runs before it:
+    // a refusal a metadata read can produce should not cost a full model load.
+    // More importantly it must precede the FIRST REQUEST, since that is where
+    // an unsupported dtype aborts the process rather than failing.
+    require_supported_weight_dtypes(family, resolved_path);
+
+    // Session options are built BEFORE the load, because parse_backend_type
+    // rejects an unknown backend name. Validating after the load would make
+    // `backend:cudaa` cost a full model load, on a fault a string comparison
+    // could have caught.
+    session_options_.backend.type = parse_backend_type(options.backend);
+    session_options_.backend.device = options.device;
+    if (options.threads > 0) {
+        session_options_.backend.threads = options.threads;
+    }
+    for (const auto &entry : options.session_options) {
+        session_options_.options[entry.first] = entry.second;
+    }
+
+    pinned_task_ = options.task;
+    wait_budget_ceiling_ms_ = options.busy_timeout_ms;
+    live_idle_timeout_ms_ = options.live_idle_timeout_ms;
+
+    engine::runtime::ModelLoadRequest request;
+    request.model_path = std::filesystem::path(resolved_path);
+    request.family_hint = family;
+    if (!options.model_spec_override.empty()) {
+        request.model_spec_override =
+            std::filesystem::path(options.model_spec_override);
+    }
+    for (const auto &entry : options.load_options) {
+        request.options[entry.first] = entry.second;
+    }
+
+    try {
+        model_ = registry_.load(request);
+    } catch (const std::exception &err) {
+        throw ConfigError("audio-cpp: failed to load family '" + family +
+                          "' from " + resolved_path + ": " + err.what());
+    }
+    if (model_ == nullptr) {
+        throw ConfigError("audio-cpp: the registry returned no model for " +
+                          resolved_path);
+    }
+
+    const auto &metadata = model_->metadata();
+    const auto &engine_caps = model_->capabilities();
+    variant_ = metadata.variant;
+    description_ = metadata.description;
+    languages_ = engine_caps.languages;
+    supports_timestamps_ = engine_caps.supports_timestamps;
+    capabilities_ = to_capabilities(family, engine_caps);
+}
+
+Route LoadedModel::check_can_serve(Rpc rpc, const RequestShape &shape) const {
+    const Route route = resolve_route(rpc, shape, capabilities_);
+    if (!route.ok) {
+        throw CapabilityError(route.error);
+    }
+    // Returned so a handler can act on the task before running it. It is the
+    // same route session_for will resolve, since both read the immutable
+    // capabilities_ from the same shape.
+    return route;
+}
+
+LoadedModel::Session LoadedModel::session_for(Rpc rpc, const RequestShape &shape,
+                                              LaneEntry &lane) {
+    // Proof of holding only. Nothing here reads it, and nothing should: its
+    // whole job is to make a caller that has not taken the lane fail to
+    // compile. Non-const so it cannot bind to an inline acquire(), whose
+    // temporary would be released at the end of this call.
+    (void)lane;
+    const Route route = resolve_route(rpc, shape, capabilities_);
+    if (!route.ok) {
+        throw CapabilityError(route.error);
+    }
+
+    const SessionKey key{static_cast<int>(route.task), static_cast<int>(route.mode)};
+    auto found = sessions_.find(key);
+    const bool cache_hit = found != sessions_.end();
+    if (!cache_hit) {
+        engine::runtime::TaskSpec spec;
+        spec.task = to_engine_task(route.task);
+        spec.mode = to_engine_mode(route.mode);
+        std::unique_ptr<engine::runtime::IVoiceTaskSession> created;
+        try {
+            created = model_->create_task_session(spec, session_options_);
+        } catch (const std::exception &err) {
+            // NOT a CapabilityError. The family said it supports this pair, and
+            // a throw from here is overwhelmingly an environment fault: a ggml
+            // backend .so that package.sh did not ship, an out of memory, a CUDA
+            // device that is not there. UNIMPLEMENTED would tell LocalAI and
+            // every client "this model cannot do this, never retry", and send an
+            // operator hunting a capability bug instead of a packaging one. A
+            // plain runtime_error maps to INTERNAL, which is what a fixable
+            // deployment fault should look like.
+            throw std::runtime_error(
+                std::string("audio-cpp: family '") + capabilities_.family +
+                "' advertises " + task_name(route.task) + "/" +
+                mode_name(route.mode) + " but refused to create the session: " +
+                err.what());
+        }
+        if (created == nullptr) {
+            // A null return with no throw is the family declining, which is a
+            // genuine capability answer and stays UNIMPLEMENTED.
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' returned no session for " +
+                                  task_name(route.task) + "/" +
+                                  mode_name(route.mode));
+        }
+        found = sessions_.emplace(key, std::move(created)).first;
+    }
+
+    Session session;
+    session.task = route.task;
+    session.mode = route.mode;
+    engine::runtime::IVoiceTaskSession *raw = found->second.get();
+    if (route.mode == Mode::Streaming) {
+        session.streaming =
+            dynamic_cast<engine::runtime::IStreamingVoiceTaskSession *>(raw);
+        if (session.streaming == nullptr) {
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' advertises " + task_name(route.task) +
+                                  "/streaming but its session is not streaming");
+        }
+        // Deliberately NOT reset here, though a cached streaming session does
+        // carry state across chunks. reset() is not callable at this point:
+        // silero_vad's implementation throws "session prepare() must be called
+        // before Silero VAD reset()", so resetting on a cache hit would turn an
+        // ordinary second fetch into a hard error, which is worse than the leak
+        // it would prevent.
+        //
+        // The state is instead cleared by the sequence every streaming caller
+        // owes anyway. IStreamingVoiceTaskSession::start_stream's base
+        // implementation IS a call to reset(), so a caller that runs
+        // prepare(...) then start_stream(...) at the top of each stream gets a
+        // clean session for free. See the STATE CONTRACT in loaded_model.h.
+    } else {
+        session.offline =
+            dynamic_cast<engine::runtime::IOfflineVoiceTaskSession *>(raw);
+        if (session.offline == nullptr) {
+            throw CapabilityError(std::string("audio-cpp: family '") +
+                                  capabilities_.family +
+                                  "' advertises " + task_name(route.task) +
+                                  "/offline but its session is not offline");
+        }
+    }
+    return session;
+}
+
+LaneEntry LoadedModel::acquire(int requested_timeout_ms) {
+    // Constructed straight into the return value. C++17 requires that, which is
+    // what lets an immovable type be returned at all; a named local here would
+    // not compile.
+    return LaneEntry(lane_,
+                     resolve_wait_budget_ms(wait_budget_ceiling_ms_,
+                                            requested_timeout_ms));
+}
+
+std::unique_ptr<LaneEntry> LoadedModel::acquire_owned(int requested_timeout_ms) {
+    return std::make_unique<LaneEntry>(
+        lane_,
+        resolve_wait_budget_ms(wait_budget_ceiling_ms_, requested_timeout_ms));
+}
+
+engine::runtime::TaskResult run_offline(const LoadedModel::Session &session,
+                                        const engine::runtime::TaskRequest &request,
+                                        LaneEntry &lane) {
+    // Proof of holding only, as in session_for.
+    (void)lane;
+    if (session.offline == nullptr) {
+        throw CapabilityError("audio-cpp: no offline session for this request");
+    }
+    session.offline->prepare(engine::runtime::build_preparation_request(request));
+    return session.offline->run(request);
+}
+
+namespace {
+
+engine::runtime::IStreamingVoiceTaskSession &
+require_streaming(const LoadedModel::Session &session) {
+    if (session.streaming == nullptr) {
+        throw CapabilityError("audio-cpp: no streaming session for this request");
+    }
+    return *session.streaming;
+}
+
+// Clears the stream event sink on every exit from the driver, including the
+// exception path. The session is CACHED and outlives the call that installed
+// the sink, so a std::function left behind holding references into that call's
+// frame is called with dangling captures by whoever streams next.
+class ScopedStreamSink {
+public:
+    ScopedStreamSink(engine::runtime::IStreamingVoiceTaskSession &session,
+                     engine::runtime::StreamEventCallback sink)
+        : session_(session) {
+        session_.set_stream_event_sink(std::move(sink));
+    }
+    ~ScopedStreamSink() { session_.set_stream_event_sink(nullptr); }
+
+    ScopedStreamSink(const ScopedStreamSink &) = delete;
+    ScopedStreamSink &operator=(const ScopedStreamSink &) = delete;
+
+private:
+    engine::runtime::IStreamingVoiceTaskSession &session_;
+};
+
+// Frames per chunk to feed a streaming session, from its own policy.
+//
+// FRAMES, not floats. preferred_audio_chunk_samples is a per-channel count
+// everywhere upstream sets it (nemotron_asr uses its frontend sample rate,
+// i.e. one second), and vibevoice_asr refuses a chunk whose float count is not
+// divisible by its channel count, so slicing on floats would both mis-size the
+// window and hand a family a half frame.
+std::int64_t chunk_frames_for(const engine::runtime::StreamingPolicy &policy,
+                              int sample_rate) {
+    if (policy.preferred_audio_chunk_samples > 0) {
+        return policy.preferred_audio_chunk_samples;
+    }
+    // higgs_audio_stt states its window in seconds (4.0) and leaves the sample
+    // count at zero, so this branch is real rather than defensive.
+    if (policy.preferred_audio_chunk_seconds > 0.0 && sample_rate > 0) {
+        const auto frames = static_cast<std::int64_t>(
+            policy.preferred_audio_chunk_seconds * static_cast<double>(sample_rate));
+        if (frames > 0) {
+            return frames;
+        }
+    }
+    // The interface's own default, from IStreamingVoiceTaskSession::streaming_policy.
+    return 512;
+}
+
+} // namespace
+
+void begin_stream(const LoadedModel::Session &session,
+                  const engine::runtime::TaskRequest &request, LaneEntry &lane) {
+    // Proof of holding only, as in session_for.
+    (void)lane;
+    auto &streaming = require_streaming(session);
+    // Order is load-bearing: start_stream's reset() is illegal before prepare().
+    streaming.prepare(engine::runtime::build_preparation_request(request));
+    streaming.start_stream(request);
+}
+
+engine::runtime::TaskResult run_streaming_pull(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+    begin_stream(session, request, lane);
+    while (const auto event = streaming.next_stream_event()) {
+        if (on_event) {
+            on_event(*event);
+        }
+        // No pinned family sets is_final on a pulled event, so this is not what
+        // ends the loop today; the nullopt above is. Honoured anyway, because a
+        // family that does set it is saying the stream is over and pulling once
+        // more would be asking a finished session for another chunk.
+        if (event->is_final) {
+            break;
+        }
+    }
+    return streaming.finish_stream();
+}
+
+engine::runtime::TaskResult run_streaming_audio(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const engine::runtime::AudioBuffer &audio,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+
+    const int channels = audio.channels > 0 ? audio.channels : 1;
+    // REFUSED, not rounded away, and checked before anything is touched so a
+    // refusal leaves no half-started stream on a cached session.
+    //
+    // An interleaved buffer whose float count is not a whole number of frames
+    // is a truncated input, and the integer division below would silently drop
+    // the tail floats: they would never be fed, never reach the transcript, and
+    // nothing would say so. Upstream refuses the same condition rather than
+    // tolerating it, in two places: vibevoice_asr's audio_frame_count throws
+    // "VibeVoice-ASR audio samples must be divisible by channel count"
+    // (session.cpp:70-76), and its process_audio_chunk throws the same with
+    // "streamed" in the text about the chunks this driver hands it
+    // (session.cpp:742-747).
+    //
+    // ConfigError, i.e. INVALID_ARGUMENT, because the buffer came from the
+    // caller's file. read_audio_file's positive-rate path always answers mono
+    // and so cannot reach this, but its native-rate path passes the reader's
+    // sample count through unchanged, and a driver does not get to assume which
+    // path its caller took.
+    if (audio.samples.size() % static_cast<std::size_t>(channels) != 0) {
+        throw ConfigError(
+            "audio-cpp: streaming input is not a whole number of frames: " +
+            std::to_string(audio.samples.size()) + " samples across " +
+            std::to_string(channels) + " channels");
+    }
+
+    // Installed BEFORE the stream begins, so a family that reports during
+    // start_stream is not silently dropped, and destroyed after finish_stream,
+    // because nemotron_asr emits every one of its partials from inside
+    // finalize().
+    ScopedStreamSink sink(streaming,
+                          [&on_event](const engine::runtime::StreamEvent &event) {
+                              if (on_event) {
+                                  on_event(event);
+                              }
+                          });
+
+    begin_stream(session, request, lane);
+
+    const auto total_frames =
+        static_cast<std::int64_t>(audio.samples.size() / static_cast<size_t>(channels));
+    const std::int64_t chunk_frames =
+        chunk_frames_for(streaming.streaming_policy(), audio.sample_rate);
+
+    for (std::int64_t offset = 0; offset < total_frames; offset += chunk_frames) {
+        const std::int64_t end = std::min(offset + chunk_frames, total_frames);
+        engine::runtime::AudioChunk chunk;
+        chunk.sample_rate = audio.sample_rate;
+        chunk.channels = channels;
+        // A FRAME index, which is what every span in a returned event is
+        // expressed in. vibevoice_asr adds the chunk's own frame count to it to
+        // offset the spans it reports, so a float index here would place every
+        // span of a stereo stream at twice its real time.
+        chunk.start_sample = offset;
+        chunk.samples.assign(
+            audio.samples.begin() + static_cast<std::ptrdiff_t>(offset * channels),
+            audio.samples.begin() + static_cast<std::ptrdiff_t>(end * channels));
+        const auto event = streaming.process_audio_chunk(chunk);
+        if (on_event) {
+            on_event(event);
+        }
+    }
+    return streaming.finish_stream();
+}
+
+engine::runtime::TaskResult run_streaming_live(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<bool(std::vector<float> &)> &next_frames,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane) {
+    auto &streaming = require_streaming(session);
+
+    // The contract is the only thing that says what rate and layout the frames
+    // about to arrive are in, and prepare() needs it: see the header.
+    if (!request.audio_input.has_value()) {
+        throw ConfigError(
+            "audio-cpp: a live streaming request carries no audio contract");
+    }
+    const int sample_rate = request.audio_input->sample_rate;
+    const int channels =
+        request.audio_input->channels > 0 ? request.audio_input->channels : 1;
+
+    // Installed BEFORE the stream begins and cleared on every exit, including
+    // the exception path, for the reasons spelled out in run_streaming_audio.
+    ScopedStreamSink sink(streaming,
+                          [&on_event](const engine::runtime::StreamEvent &event) {
+                              if (on_event) {
+                                  on_event(event);
+                              }
+                          });
+
+    begin_stream(session, request, lane);
+
+    const std::int64_t chunk_frames =
+        chunk_frames_for(streaming.streaming_policy(), sample_rate);
+    // chunk_frames_for never returns a non-positive count, so this is never
+    // zero and the accumulation loop below always terminates.
+    const std::size_t chunk_floats = static_cast<std::size_t>(chunk_frames) *
+                                     static_cast<std::size_t>(channels);
+
+    std::int64_t fed_frames = 0;
+    const auto feed = [&](std::vector<float> samples) {
+        engine::runtime::AudioChunk chunk;
+        chunk.sample_rate = sample_rate;
+        chunk.channels = channels;
+        // A FRAME index, counted across the whole stream: vibevoice_asr offsets
+        // every span it reports by it, so restarting it per chunk would put
+        // every word at the top of the recording.
+        chunk.start_sample = fed_frames;
+        chunk.samples = std::move(samples);
+        fed_frames +=
+            static_cast<std::int64_t>(chunk.samples.size()) / channels;
+        const auto event = streaming.process_audio_chunk(chunk);
+        if (on_event) {
+            on_event(event);
+        }
+    };
+
+    std::vector<float> pending;
+    std::vector<float> incoming;
+    while (true) {
+        incoming.clear();
+        if (!next_frames(incoming)) {
+            break;
+        }
+        pending.insert(pending.end(), incoming.begin(), incoming.end());
+        while (pending.size() >= chunk_floats) {
+            std::vector<float> window(pending.begin(),
+                                      pending.begin() +
+                                          static_cast<std::ptrdiff_t>(chunk_floats));
+            pending.erase(pending.begin(),
+                          pending.begin() +
+                              static_cast<std::ptrdiff_t>(chunk_floats));
+            feed(std::move(window));
+        }
+    }
+
+    if (!pending.empty()) {
+        // The tail is whatever did not fill a window. Refused rather than
+        // truncated when it is not a whole number of frames, exactly as in
+        // run_streaming_audio: the division above would drop the stray floats
+        // from the transcript with no diagnostic. Unreachable for a mono live
+        // stream, which is every live stream today.
+        if (pending.size() % static_cast<std::size_t>(channels) != 0) {
+            throw ConfigError(
+                "audio-cpp: live stream ended mid-frame: " +
+                std::to_string(pending.size()) + " trailing samples across " +
+                std::to_string(channels) + " channels");
+        }
+        feed(std::move(pending));
+    }
+
+    if (fed_frames == 0) {
+        // Nothing was spoken. See the header: finalizing an empty stream is not
+        // legal for every family, and an empty transcript is the truthful
+        // answer rather than an engine-internal INTERNAL.
+        return engine::runtime::TaskResult{};
+    }
+    return streaming.finish_stream();
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/loaded_model.h

@@ -0,0 +1,378 @@
+#pragma once
+
+// Owns one audio.cpp model for the life of the process, plus a lazily created
+// session per (task, mode) so the same model answers both TTS and TTSStream.
+// This is the only unit that converts between the stdlib-only mirror types and
+// engine::runtime types.
+
+#include "capability_routing.h"
+#include "inference_lane.h"
+#include "model_options.h"
+
+#include "engine/framework/runtime/model.h"
+#include "engine/framework/runtime/registry.h"
+#include "engine/framework/runtime/session.h"
+
+#include <functional>
+#include <map>
+#include <memory>
+#include <stdexcept>
+#include <string>
+#include <utility>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// User-fixable configuration problem. grpc-server maps this to INVALID_ARGUMENT.
+class ConfigError : public std::runtime_error {
+public:
+    using std::runtime_error::runtime_error;
+};
+
+// The family cannot serve the requested RPC. Maps to UNIMPLEMENTED.
+class CapabilityError : public std::runtime_error {
+public:
+    using std::runtime_error::runtime_error;
+};
+
+engine::runtime::VoiceTaskKind to_engine_task(Task task);
+engine::runtime::RunMode to_engine_mode(Mode mode);
+Task from_engine_task(engine::runtime::VoiceTaskKind kind);
+Mode from_engine_mode(engine::runtime::RunMode mode);
+Capabilities to_capabilities(const std::string &family,
+                             const engine::runtime::CapabilitySet &set);
+
+// Reads audiocpp.model_spec.family from a GGUF. Returns an empty string when
+// the file is not a GGUF, carries no audio.cpp spec, or cannot be read. Never
+// throws: an unreadable file is the load gate's problem, not a crash.
+std::string read_gguf_family(const std::string &path);
+
+// Builds the absolute model path from LocalAI's (ModelPath, ModelFile, Model)
+// triple. Either the Model or the ModelFile field may carry the form
+// "bundled:<name>", which resolves to <executable dir>/assets/<name>, where
+// package.sh puts upstream's bundled silero_vad and marblenet_vad assets. BOTH
+// are checked because LocalAI fills ModelFile by joining ModelPath onto the
+// configured model string, so a model YAML using the form has it intact only in
+// Model.
+std::string resolve_model_path(const std::string &model_path_dir,
+                               const std::string &model_file,
+                               const std::string &model_name);
+
+class LoadedModel {
+public:
+    struct Session {
+        Task task = Task::Tts;
+        Mode mode = Mode::Offline;
+        // Exactly one of these is non-null, matching the resolved mode.
+        engine::runtime::IOfflineVoiceTaskSession *offline = nullptr;
+        engine::runtime::IStreamingVoiceTaskSession *streaming = nullptr;
+    };
+
+    // Throws ConfigError when the path does not exist, the family cannot be
+    // determined, or the registry rejects the family.
+    //
+    // `model_identity` is ModelOptions.Model verbatim: the UNTRANSLATED
+    // controller-side name. It is a constructor argument rather than a setter
+    // so identity and model are inseparable. llama-cpp keeps its equivalent in
+    // a separate global from the model, which leaves a window where a handler
+    // can read one without the other; here a handler that holds the model
+    // through snapshot() necessarily holds the identity it was loaded with.
+    LoadedModel(const std::string &resolved_path, const ModelOptions &options,
+                std::string model_identity);
+
+    LoadedModel(const LoadedModel &) = delete;
+    LoadedModel &operator=(const LoadedModel &) = delete;
+
+    const std::string &family() const noexcept { return capabilities_.family; }
+    // Empty when the controller predates ModelOptions.ModelIdentity, which the
+    // identity check reads as "skip". See check_model_identity in grpc-server.
+    const std::string &identity() const noexcept { return identity_; }
+    const std::string &variant() const noexcept { return variant_; }
+    const std::string &description() const noexcept { return description_; }
+    const std::vector<std::string> &languages() const noexcept { return languages_; }
+    const Capabilities &capabilities() const noexcept { return capabilities_; }
+    bool supports_timestamps() const noexcept { return supports_timestamps_; }
+    const engine::runtime::SessionOptions &session_options() const noexcept {
+        return session_options_;
+    }
+
+    // The model's `task:` option, empty when unset. Every handler must copy it
+    // into RequestShape::pinned_task before calling session_for: routing is
+    // otherwise derived from the RPC alone, and this is the option's only route
+    // from the load to the request that honours it.
+    const std::string &pinned_task() const noexcept { return pinned_task_; }
+
+    // The `live_idle_timeout_ms` option: how long AudioTranscriptionLive waits
+    // for the next audio frame before cancelling the stream to give this
+    // model's lane back. 0 means no limit. See the option in model_options.h
+    // for why it exists and how the default was chosen.
+    int live_idle_timeout_ms() const noexcept { return live_idle_timeout_ms_; }
+
+    // Throws the same CapabilityError session_for would throw when this family
+    // cannot serve the RPC, and RETURNS THE RESOLVED ROUTE otherwise.
+    //
+    // The route is returned rather than computed and dropped because a handler
+    // often has to know which task it is about to run BEFORE running it.
+    // AudioTransform refuses params[stem] on any route but source separation,
+    // and reading that off the route costs microseconds where reading it off
+    // the result costs a whole inference first. A caller with no such need
+    // ignores the value, which is what the three transcription-shaped handlers
+    // do.
+    //
+    // It exists so a refusal does not have to buy a place in the queue first.
+    // resolve_route is a pure function of capabilities_, which is fixed at
+    // construction and never written again, so unlike the session cache it
+    // needs no lane and no lock: a model that cannot transcribe can say so
+    // while another request is halfway through a thirty second run. Without
+    // this the refusal waits for that run to finish only to be told no.
+    //
+    // It does NOT replace the routing inside session_for, and must not be made
+    // to: session_for still needs the route to key the session cache. The two
+    // calls agree because both read the same immutable capabilities. What this
+    // one adds is only the ordering, so call it before acquire().
+    //
+    // Const and lane-free on purpose. If a future edit makes routing depend on
+    // mutable state, this must grow the lane parameter its siblings carry.
+    Route check_can_serve(Rpc rpc, const RequestShape &shape) const;
+
+    // Routes the RPC and returns the cached session, creating it on first use.
+    // Throws CapabilityError when this family cannot serve the RPC, and a plain
+    // runtime_error when it can but the session could not be built, which is an
+    // environment fault rather than a capability answer.
+    //
+    // The `lane` parameter is a PROOF OF HOLDING and is otherwise unused: it
+    // exists so the rule below is a compile error rather than prose. The
+    // session cache is an unsynchronised std::map and the sessions themselves
+    // are not reentrant, so this must only be called with the lane held; the
+    // lane admits one caller at a time, which is exactly the constraint the
+    // sessions impose. Pass the LaneEntry from acquire().
+    //
+    // NON-CONST reference on purpose, and do not "tidy" it to const. A const
+    // reference binds to a temporary, which makes this compile:
+    //
+    //     auto session = model->session_for(rpc, shape, model->acquire(0));
+    //     auto result  = run_offline(session, task, model->acquire(0));
+    //
+    // and each temporary dies at the end of its own full-expression, so the
+    // lane is released between the two calls. That is precisely the split this
+    // parameter exists to prevent, and it is the form a future caller is most
+    // likely to reach for because it reads as tidy. Requiring an lvalue forces
+    // a named entry whose scope spans both calls.
+    //
+    // What it proves is bounded, so do not over-trust it: it proves A lane was
+    // taken, not THIS model's lane. A caller determined to defeat it can
+    // construct an entry on an unrelated InferenceLane and pass that. It
+    // therefore catches the two mistakes that actually happen, forgetting the
+    // lane entirely and taking it after routing, and does not catch lane
+    // identity.
+    //
+    // STATE CONTRACT, and it is the CALLER'S to honour. Sessions are cached per
+    // (task, mode), so a streaming session is normally the same warm object the
+    // previous stream used, carrying that stream's state. session_for hands it
+    // back as it is.
+    //
+    // Every streaming caller must therefore begin a stream through
+    // begin_stream() below, which is prepare() then start_stream() in that
+    // order and is the ONLY implementation of that sequence. start_stream's
+    // base implementation is a call to reset(), which is what clears the
+    // previous stream, and reset() is only legal after prepare(): silero_vad
+    // throws "session prepare() must be called before Silero VAD reset()"
+    // otherwise. That ordering constraint is also why session_for cannot do
+    // this for you. Skipping it does not raise an error, it silently continues
+    // the previous stream.
+    //
+    // Offline sessions need no such care: their interface has no reset and
+    // run() takes a whole request.
+    Session session_for(Rpc rpc, const RequestShape &shape, LaneEntry &lane);
+
+    // Takes the inference lane, or throws LaneUnavailable. Serializes runs
+    // against this model. `requested_timeout_ms` is a per-request wait hint
+    // where a value <= 0 means "use the model's configured ceiling"; a hint may
+    // only tighten that ceiling, never loosen it.
+    //
+    // LaneEntry is deliberately immovable, so bind the result to a named local
+    // in the scope the inference happens in:
+    //
+    //     LaneEntry entry = model.acquire(request_hint_ms);
+    //
+    // which C++17 initializes in place. A handler that has to keep the lane
+    // beyond one scope, for instance in a member that outlives the call that
+    // took it, wants acquire_owned instead.
+    LaneEntry acquire(int requested_timeout_ms);
+
+    // Same lane, heap-allocated so it can be stored or handed on. Prefer
+    // acquire: this one adds a null state that the scoped form does not have.
+    std::unique_ptr<LaneEntry> acquire_owned(int requested_timeout_ms);
+
+private:
+    // Keyed by the enum values so the map needs no custom comparator.
+    using SessionKey = std::pair<int, int>;
+
+    // The public constructor runs the load gate, then delegates here. The
+    // detour exists because lane_ has to be built from the family in the member
+    // initializer list, and the family is only known after the gate has run.
+    // Four parameters rather than three so it cannot be confused with the
+    // public constructor, whose third argument is also a std::string.
+    LoadedModel(const std::string &resolved_path, const ModelOptions &options,
+                std::string family, std::string model_identity);
+
+    // MEMBER ORDER IS LOAD-BEARING BELOW THIS LINE. Members are destroyed in
+    // reverse declaration order.
+    //
+    // lane_ is first so it is destroyed last: nothing that runs during teardown
+    // can then find a lane that has already gone.
+    InferenceLane lane_;
+    // registry_ before model_: the registry owns the loader that produced the
+    // model, and the model may hold loader-owned state.
+    engine::runtime::ModelRegistry registry_;
+    // model_ before sessions_, so sessions_ is destroyed FIRST and the model
+    // second. A session is created from the model and must not outlive it. Do
+    // not reorder these two.
+    std::unique_ptr<engine::runtime::ILoadedVoiceModel> model_;
+    std::map<SessionKey, std::unique_ptr<engine::runtime::IVoiceTaskSession>> sessions_;
+
+    engine::runtime::SessionOptions session_options_;
+    Capabilities capabilities_;
+    std::string variant_;
+    std::string description_;
+    std::vector<std::string> languages_;
+    std::string pinned_task_;
+    std::string identity_;
+    bool supports_timestamps_ = false;
+    int wait_budget_ceiling_ms_ = 0;
+    int live_idle_timeout_ms_ = 0;
+};
+
+// Prepares and runs an offline session. prepare() is called for every run
+// rather than once per session, because SessionPreparationRequest is derived
+// from the request itself (audio contract, text, voice condition) and not from
+// the model: a second request with a different sample rate or length would
+// otherwise run against the first request's contract.
+//
+// `lane` is a PROOF OF HOLDING, unused at runtime, for the same reason
+// session_for takes one: the session is not reentrant and prepare() mutates it,
+// so running without the lane is a data race. Making it a parameter turns that
+// into a compile error instead of a comment. Non-const for the same reason as
+// session_for's: a const reference would bind to `model.acquire(0)` written
+// inline, and that temporary dies at the end of this call, releasing the lane
+// before the caller's next one.
+//
+// Throws CapabilityError when the session is not an offline one. That should be
+// unreachable through session_for, which already refuses a non-offline session
+// for an offline route, and is checked anyway because the alternative is a null
+// dereference.
+engine::runtime::TaskResult run_offline(const LoadedModel::Session &session,
+                                        const engine::runtime::TaskRequest &request,
+                                        LaneEntry &lane);
+
+// THE ONE IMPLEMENTATION of the streaming state obligation described in
+// session_for's STATE CONTRACT: prepare(), then start_stream(), in that order.
+//
+// It is a function rather than a comment because the obligation is invisible
+// when it is broken. Streaming sessions are CACHED per (task, mode), so the
+// object a second stream gets is the warm one the first stream left behind,
+// still holding its audio, its tokens and its started flag. What clears it is
+// start_stream, whose base implementation IS a reset() and whose seven family
+// overrides (nemotron_asr, vibevoice_asr, higgs_audio_stt, voxtral_realtime,
+// supertonic, omnivoice, voxcpm2) every one call reset() as their first
+// statement, verified in the pinned checkout. Nothing in the type system pins
+// that. A future override that dropped the reset would break every call site
+// at once with no compile error and no exception, only a second transcript
+// that begins with the first one's audio, so the fewer call sites there are to
+// break, the better: this is the only one.
+//
+// prepare() must come first and cannot be folded into session_for, because
+// reset() is illegal before prepare() (silero_vad throws "session prepare()
+// must be called before Silero VAD reset()"), and because the preparation
+// request is derived from the REQUEST, not the model: build_preparation_request
+// reads the audio contract, the text and the voice condition off it, so a
+// second stream with a different sample rate or length would otherwise run
+// against the first stream's contract.
+//
+// `lane` is a PROOF OF HOLDING, unused at runtime, exactly as in run_offline.
+//
+// Throws CapabilityError when the session is not a streaming one.
+void begin_stream(const LoadedModel::Session &session,
+                  const engine::runtime::TaskRequest &request, LaneEntry &lane);
+
+// Drives a streaming session that takes NO incremental input, which is the TTS
+// shape (StreamingInputKind::None, StreamingOutputKind::PullEvents): begin the
+// stream, pull events until the session says there are no more, then finish.
+//
+// NO STREAM EVENT SINK IS INSTALLED HERE, and that is deliberate rather than an
+// omission. voxcpm2's start_stream runs the whole synthesis and pushes every
+// chunk to the sink, then its next_stream_event replays those same chunks out
+// of the stored result, so a sink on this path would put every chunk of audio
+// on the wire twice. supertonic and omnivoice ignore set_stream_event_sink
+// outright. The pull loop is therefore the single delivery channel.
+//
+// The returned TaskResult is the session's own merged whole for all three
+// families, NOT a tail the pull loop missed. A caller that already emitted the
+// pulled events must not also emit its audio; see the TTSStream handler.
+engine::runtime::TaskResult run_streaming_pull(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+// Drives a streaming session that CONSUMES audio chunks, which is the ASR shape
+// (StreamingInputKind::AudioChunks): begin the stream, feed the buffer in
+// policy-sized chunks, then finalize.
+//
+// A STREAM EVENT SINK IS INSTALLED HERE, and it is not optional: nemotron_asr
+// reports its partial text ONLY through the sink, and only from inside
+// finalize(), because its decode does not start until the audio is complete.
+// Without the sink that family streams a transcript with no partials at all.
+// The sink is cleared again before returning, including on the exception path:
+// the session is cached and outlives this call, so a sink left holding a
+// reference to the caller's frame is a use after free waiting for the next
+// stream.
+//
+// Both delivery channels are consumed, the sink and the value process_audio_chunk
+// returns, because the families do not agree on which they use, and
+// voxtral_realtime uses BOTH for the same event. The duplicate that produces is
+// absorbed by TranscriptDeltaTracker in stream_delta.h rather than here.
+engine::runtime::TaskResult run_streaming_audio(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const engine::runtime::AudioBuffer &audio,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+// Drives the same ASR shape as run_streaming_audio when the audio DOES NOT
+// EXIST YET, which is the live-microphone case: instead of slicing a buffer it
+// pulls frames from the caller until the input side closes.
+//
+// `next_frames` fills `out` with interleaved float PCM and returns true, or
+// returns false when there is no more input. It is expected to BLOCK, since the
+// only real implementation is a gRPC stream Read, and it may throw: a request
+// the handler has to refuse mid-stream unwinds through here, and the sink is
+// cleared on that path like every other.
+//
+// The audio contract comes from `request.audio_input`, which for a live stream
+// is an EMPTY buffer carrying only the sample rate and channel count. It is not
+// optional: nemotron_asr's streaming prepare() throws "Nemotron ASR streaming
+// prepare() requires an audio contract" without one, and there is no buffer to
+// derive it from here.
+//
+// Frames are BUFFERED to the family's own preferred window rather than fed in
+// whatever sizes the wire delivered them in, because that window is a family's
+// statement about what it can decode (nemotron_asr asks for one second, higgs
+// for four), and a 512-sample gRPC frame is a property of the client's audio
+// callback rather than of the model. The tail shorter than a window is fed at
+// the end.
+//
+// A stream that carried NO AUDIO returns an empty TaskResult and never calls
+// finish_stream. Finalizing an empty stream is not universally legal:
+// nemotron_asr throws "Nemotron ASR finalize requires streamed audio", so a
+// client that opens a session and closes it without speaking would receive an
+// INTERNAL naming an engine internal instead of an empty transcript, which is
+// the truthful answer to "transcribe nothing".
+engine::runtime::TaskResult run_streaming_live(
+    const LoadedModel::Session &session,
+    const engine::runtime::TaskRequest &request,
+    const std::function<bool(std::vector<float> &)> &next_frames,
+    const std::function<void(const engine::runtime::StreamEvent &)> &on_event,
+    LaneEntry &lane);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options.cpp

@@ -0,0 +1,153 @@
+#include "model_options.h"
+
+#include <cctype>
+#include <cerrno>
+#include <climits>
+#include <cstdlib>
+
+namespace audiocpp_backend {
+namespace {
+
+std::string trim(const std::string &value) {
+    size_t begin = 0;
+    while (begin < value.size() &&
+           std::isspace(static_cast<unsigned char>(value[begin])) != 0) {
+        ++begin;
+    }
+    size_t end = value.size();
+    while (end > begin &&
+           std::isspace(static_cast<unsigned char>(value[end - 1])) != 0) {
+        --end;
+    }
+    return value.substr(begin, end - begin);
+}
+
+// Parses a non-negative integer. Returns false on anything else, including
+// empty strings, signs, trailing garbage, and values too large for int.
+//
+// strtol rather than atoi: atoi is undefined behaviour once the digits exceed
+// long, and in practice it hands back a wrapped value. That would let
+// "device:2147483648" through as -2147483648 and send a negative index to the
+// ggml backend selector, from a function whose error text promises the caller a
+// non-negative integer.
+bool parse_non_negative_int(const std::string &value, int &out) {
+    if (value.empty()) {
+        return false;
+    }
+    for (const char ch : value) {
+        if (std::isdigit(static_cast<unsigned char>(ch)) == 0) {
+            return false;
+        }
+    }
+
+    errno = 0;
+    char *end = nullptr;
+    const long parsed = std::strtol(value.c_str(), &end, 10);
+    if (errno == ERANGE || end == nullptr || *end != '\0') {
+        return false;
+    }
+    if (parsed < 0 || parsed > INT_MAX) {
+        return false;
+    }
+
+    out = static_cast<int>(parsed);
+    return true;
+}
+
+bool starts_with(const std::string &value, const std::string &prefix) {
+    return value.size() >= prefix.size() &&
+           value.compare(0, prefix.size(), prefix) == 0;
+}
+
+} // namespace
+
+ParsedOptions parse_model_options(const std::vector<std::string> &entries) {
+    ParsedOptions parsed;
+
+    for (const auto &raw : entries) {
+        const std::string entry = trim(raw);
+        if (entry.empty()) {
+            continue;
+        }
+
+        // Split on the FIRST colon: values are often paths that contain more.
+        const size_t sep = entry.find(':');
+        if (sep == std::string::npos) {
+            parsed.error = "audio-cpp: option '" + entry +
+                           "' is not in key:value form";
+            return parsed;
+        }
+
+        const std::string key = trim(entry.substr(0, sep));
+        const std::string value = trim(entry.substr(sep + 1));
+
+        if (starts_with(key, "load.")) {
+            const std::string inner = key.substr(5);
+            if (inner.empty()) {
+                parsed.error = "audio-cpp: option '" + entry +
+                               "' has an empty load option name";
+                return parsed;
+            }
+            parsed.options.load_options[inner] = value;
+            continue;
+        }
+        if (starts_with(key, "session.")) {
+            const std::string inner = key.substr(8);
+            if (inner.empty()) {
+                parsed.error = "audio-cpp: option '" + entry +
+                               "' has an empty session option name";
+                return parsed;
+            }
+            parsed.options.session_options[inner] = value;
+            continue;
+        }
+
+        if (key == "family") {
+            parsed.options.family = value;
+        } else if (key == "task") {
+            parsed.options.task = value;
+        } else if (key == "backend") {
+            parsed.options.backend = value;
+        } else if (key == "model_spec_override") {
+            parsed.options.model_spec_override = value;
+        } else if (key == "device") {
+            if (!parse_non_negative_int(value, parsed.options.device)) {
+                parsed.error = "audio-cpp: option 'device' needs a non-negative "
+                               "integer, got '" + value + "'";
+                return parsed;
+            }
+            parsed.options.device_set = true;
+        } else if (key == "threads") {
+            if (!parse_non_negative_int(value, parsed.options.threads)) {
+                parsed.error = "audio-cpp: option 'threads' needs a non-negative "
+                               "integer, got '" + value + "'";
+                return parsed;
+            }
+        } else if (key == "busy_timeout_ms") {
+            if (!parse_non_negative_int(value, parsed.options.busy_timeout_ms)) {
+                parsed.error = "audio-cpp: option 'busy_timeout_ms' needs a "
+                               "non-negative integer, got '" + value + "'";
+                return parsed;
+            }
+        } else if (key == "live_idle_timeout_ms") {
+            if (!parse_non_negative_int(value,
+                                        parsed.options.live_idle_timeout_ms)) {
+                parsed.error = "audio-cpp: option 'live_idle_timeout_ms' needs a "
+                               "non-negative integer, got '" + value + "'";
+                return parsed;
+            }
+        } else {
+            // Quotes the whole entry, not just the key: an entry like ":value"
+            // has an empty key and would otherwise leave nothing to grep for.
+            parsed.error = "audio-cpp: unknown option key '" + entry +
+                           "'. Known keys: family, task, backend, device, "
+                           "threads, model_spec_override, busy_timeout_ms, "
+                           "live_idle_timeout_ms, load.<key>, session.<key>";
+            return parsed;
+        }
+    }
+
+    return parsed;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options.h

@@ -0,0 +1,66 @@
+#pragma once
+
+// Parses the model YAML's `options:` list (ModelOptions.Options in
+// backend.proto) into a struct. Standard library only: this unit is compiled
+// and tested by backend/cpp/run-unit-tests.sh without an audio.cpp checkout.
+
+#include <map>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+struct ModelOptions {
+    // audio.cpp model family. Empty means "derive from the GGUF's embedded
+    // audiocpp.model_spec.family key"; a non-GGUF path with an empty family is
+    // rejected at load time, not here.
+    std::string family;
+    // Pins the audio.cpp task, overriding RPC-based routing. Empty means route.
+    std::string task;
+    // ggml backend: cpu, cuda, vulkan, metal, best.
+    std::string backend = "cpu";
+    int device = 0;
+    // True once a `device:` entry has been seen. 0 is both the default and a
+    // legitimate device index, so the value alone cannot tell an explicit
+    // `device:0` from an unset option, and a caller merging in its own fallback
+    // would silently override the explicit choice.
+    bool device_set = false;
+    // 0 means "let the runtime decide".
+    int threads = 0;
+    std::string model_spec_override;
+    // 0 disables the run guard's fail-fast, restoring an unbounded wait.
+    int busy_timeout_ms = 0;
+    // How long AudioTranscriptionLive waits for the next audio frame before it
+    // cancels the stream and gives the model's lane back. 0 means NO LIMIT.
+    //
+    // It exists because that RPC holds the lane for the whole stream, so a peer
+    // that stops sending WITHOUT closing blocks every other request against this
+    // model for as long as its socket stays up. No other RPC can do that: they
+    // hold the lane across compute, which ends on its own.
+    //
+    // 30 seconds, and the number is picked from what the only in-tree client
+    // does. core/http/endpoints/openai/realtime.go drives a 300 ms ticker and
+    // feeds every tick that produced new audio while a turn is open, so 30 s of
+    // silence is a hundred ticks that delivered nothing: the peer is gone, or
+    // its socket is wedged. It is also comfortably longer than any pause a
+    // speaker takes mid-utterance, which is the case that must never be cut off,
+    // and backend.proto allows one stream to span many utterances, so a client
+    // that pauses for longer than this between them should raise it rather than
+    // discover it. Lowering it below a few seconds risks cancelling a live
+    // speaker; 0 turns the limit off for a client that legitimately idles.
+    int live_idle_timeout_ms = 30000;
+    // `load.<key>:<value>` entries, prefix stripped.
+    std::map<std::string, std::string> load_options;
+    // `session.<key>:<value>` entries, prefix stripped.
+    std::map<std::string, std::string> session_options;
+};
+
+struct ParsedOptions {
+    ModelOptions options;
+    // Non-empty means the caller must fail the load with INVALID_ARGUMENT.
+    std::string error;
+};
+
+ParsedOptions parse_model_options(const std::vector<std::string> &entries);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/model_options_test.cpp

@@ -0,0 +1,165 @@
+// Unit tests for model_options. Standard library only, so
+// backend/cpp/run-unit-tests.sh picks this up with no engine checkout.
+//
+// The harness compiles this file as a single translation unit with no other
+// sources, so the implementation is included directly rather than linked.
+//
+// Build and run standalone:
+//   g++ -std=c++17 -I. model_options_test.cpp -o t && ./t
+
+#include "model_options.cpp"
+
+#include <cstdio>
+#include <map>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+// Returns the mapped value, or an empty string when the key is absent. map::at
+// would throw on a miss and abort the whole binary, so one prefix off-by-one
+// would hide every check that follows it instead of failing a single one.
+static std::string lookup(const std::map<std::string, std::string> &values,
+                          const std::string &key) {
+    const auto found = values.find(key);
+    return found == values.end() ? std::string() : found->second;
+}
+
+using audiocpp_backend::parse_model_options;
+
+static void test_defaults() {
+    auto r = parse_model_options({});
+    check(r.error.empty(), "empty option list is not an error");
+    check(r.options.family.empty(), "family defaults to empty");
+    check(r.options.task.empty(), "task defaults to empty");
+    check(r.options.backend == "cpu", "backend defaults to cpu");
+    check(r.options.device == 0, "device defaults to 0");
+    check(r.options.threads == 0, "threads defaults to 0");
+    check(r.options.busy_timeout_ms == 0, "busy_timeout_ms defaults to 0");
+    // NOT zero, unlike every other numeric option here. A live stream holds the
+    // model's lane while it waits on the client, so the default has to bound
+    // that wait; 0 is the explicit "no limit" the operator opts into.
+    check(r.options.live_idle_timeout_ms == 30000,
+          "live_idle_timeout_ms defaults to 30000");
+    check(r.options.load_options.empty(), "load_options defaults empty");
+    check(r.options.session_options.empty(), "session_options defaults empty");
+}
+
+static void test_scalar_options() {
+    auto r = parse_model_options({
+        "family:qwen3_tts",
+        "task:tts",
+        "backend:cuda",
+        "device:1",
+        "threads:8",
+        "busy_timeout_ms:30000",
+        "live_idle_timeout_ms:5000",
+    });
+    check(r.error.empty(), "scalar options parse without error");
+    check(r.options.family == "qwen3_tts", "family parsed");
+    check(r.options.task == "tts", "task parsed");
+    check(r.options.backend == "cuda", "backend parsed");
+    check(r.options.device == 1, "device parsed");
+    check(r.options.threads == 8, "threads parsed");
+    check(r.options.busy_timeout_ms == 30000, "busy_timeout_ms parsed");
+    check(r.options.live_idle_timeout_ms == 5000, "live_idle_timeout_ms parsed");
+    check(parse_model_options({"live_idle_timeout_ms:0"}).options.live_idle_timeout_ms == 0,
+          "an explicit 0 turns the live idle limit off rather than reverting to "
+          "the default");
+}
+
+// Values containing colons must survive: split on the FIRST colon only.
+static void test_value_containing_colon() {
+    auto r = parse_model_options({"model_spec_override:/models/a:b/spec.json"});
+    check(r.error.empty(), "colon-bearing value is not an error");
+    check(r.options.model_spec_override == "/models/a:b/spec.json",
+          "value keeps every colon after the first separator");
+}
+
+static void test_namespaced_options() {
+    auto r = parse_model_options({
+        "load.weight_type:q8_0",
+        "session.miocodec.weight_type:f16",
+        "session.graph_capacity:tiered",
+    });
+    check(r.error.empty(), "namespaced options parse without error");
+    check(r.options.load_options.size() == 1, "one load option");
+    check(lookup(r.options.load_options, "weight_type") == "q8_0", "load prefix stripped");
+    check(r.options.session_options.size() == 2, "two session options");
+    check(lookup(r.options.session_options, "miocodec.weight_type") == "f16",
+          "session prefix stripped, inner dots kept");
+    check(lookup(r.options.session_options, "graph_capacity") == "tiered",
+          "second session option parsed");
+}
+
+static void test_errors() {
+    check(!parse_model_options({"family"}).error.empty(),
+          "entry without a colon is rejected");
+    check(!parse_model_options({"nonsense:1"}).error.empty(),
+          "unknown key is rejected");
+    check(!parse_model_options({"device:abc"}).error.empty(),
+          "non-numeric device is rejected");
+    check(!parse_model_options({"threads:-1"}).error.empty(),
+          "negative threads is rejected");
+    check(!parse_model_options({"load.:x"}).error.empty(),
+          "empty load key is rejected");
+    check(!parse_model_options({"session.:x"}).error.empty(),
+          "empty session key is rejected");
+    check(!parse_model_options({"busy_timeout_ms:abc"}).error.empty(),
+          "non-numeric busy_timeout_ms is rejected");
+    check(!parse_model_options({"live_idle_timeout_ms:abc"}).error.empty(),
+          "non-numeric live_idle_timeout_ms is rejected");
+    check(!parse_model_options({"live_idle_timeout_ms:-1"}).error.empty(),
+          "negative live_idle_timeout_ms is rejected");
+    check(!parse_model_options({"device:-1"}).error.empty(),
+          "negative device is rejected");
+    check(!parse_model_options({"threads:x"}).error.empty(),
+          "non-numeric threads is rejected");
+
+    // Values too large for int must be rejected, not silently wrapped into a
+    // negative device index that then reaches the ggml backend selector.
+    check(!parse_model_options({"device:2147483648"}).error.empty(),
+          "device above INT_MAX is rejected");
+    check(!parse_model_options({"threads:99999999999999"}).error.empty(),
+          "threads above INT_MAX is rejected");
+
+    // The error text must name the offending entry so a user can fix their YAML.
+    const auto r = parse_model_options({"nonsense:1"});
+    check(r.error.find("nonsense") != std::string::npos,
+          "error names the offending key");
+
+    // An empty key still has to give the user something to grep for.
+    const auto empty_key = parse_model_options({":value"});
+    check(empty_key.error.find(":value") != std::string::npos,
+          "unknown-key error names the entry even when the key is empty");
+}
+
+static void test_blank_entries_ignored() {
+    auto r = parse_model_options({"", "   ", "family:supertonic"});
+    check(r.error.empty(), "blank entries are skipped, not rejected");
+    check(r.options.family == "supertonic", "real entry still parsed");
+}
+
+int main() {
+    test_defaults();
+    test_scalar_options();
+    test_value_containing_colon();
+    test_namespaced_options();
+    test_errors();
+    test_blank_entries_ignored();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all model_options checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/package.sh

@@ -0,0 +1,228 @@
+#!/bin/bash
+# Assemble backend/cpp/audio-cpp/package, which becomes the whole content of the
+# FROM scratch backend image. Nothing outside this directory exists at run time.
+set -euo pipefail
+
+CURDIR=$(dirname "$(realpath "$0")")
+REPO_ROOT="${CURDIR}/../../.."
+PACKAGE_DIR="$CURDIR/package"
+BUILD_DIR="$CURDIR/build"
+
+rm -rf "$PACKAGE_DIR"
+mkdir -p "$PACKAGE_DIR/lib" "$PACKAGE_DIR/assets"
+
+cp -avf "$CURDIR/grpc-server" "$PACKAGE_DIR/"
+cp -fv  "$CURDIR/run.sh"      "$PACKAGE_DIR/"
+
+# ENGINE_ENABLE_CPU_ALL_VARIANTS builds the ggml backends as shared objects that
+# are dlopened at run time, so ldd cannot see them and the dependency walk below
+# would leave the image with no CPU backend at all. They also cannot go in lib/:
+# ggml DISCOVERS them by listing dirname(/proc/self/exe) and the current
+# directory, so being on a library path is not enough, they have to be in a
+# directory ggml scans. run.sh execs the bundled loader from the package root
+# exactly so that directory is this one. cmake writes them to build/bin, not
+# next to build/grpc-server, which is why this reads from bin/.
+#
+# -a keeps the libggml.so -> libggml.so.0 -> libggml.so.0.12.0 symlink chain,
+# so the SONAME the binary asks for still names a file here.
+for pattern in '*.so*' '*.dylib*'; do
+    if compgen -G "$BUILD_DIR/bin/$pattern" > /dev/null; then
+        # shellcheck disable=SC2086
+        cp -avf "$BUILD_DIR/bin/"$pattern "$PACKAGE_DIR/"
+    fi
+done
+
+# Upstream ships silero_vad and marblenet_vad as small runtime assets.
+# resolve_model_path() expands "bundled:<name>" to
+# dirname(/proc/self/exe)/assets/<name>, so copying them here is what makes VAD
+# work with nothing downloaded.
+for asset in silero_vad marblenet_vad; do
+    src="$CURDIR/audio.cpp/assets/framework/models/$asset"
+    if [ -d "$src" ]; then
+        cp -rfv "$src" "$PACKAGE_DIR/assets/"
+    else
+        echo "package.sh: bundled asset missing: $src" >&2
+        echo "package.sh: run 'make audio.cpp' before packaging" >&2
+        exit 1
+    fi
+done
+
+# Everything below this point is Linux-only: a bundled ELF loader, an ldd walk
+# and an ld.so --list validation. The macOS equivalent is the otool -L closure in
+# scripts/build/audio-cpp-darwin.sh, which picks up from the exit 0 below.
+#
+# WARNING FOR ANYONE REWORKING THAT SCRIPT. The obvious move is to copy
+# scripts/build/privacy-filter-darwin.sh, and that script assembles its own
+# package under build/darwin and never calls package.sh at all. Adapted as-is it
+# will silently omit assets/, and the bundled: model path form then resolves to
+# nothing, which takes the only zero-download verification path in this backend
+# with it. That is why audio-cpp-darwin.sh copies THIS directory instead of
+# rebuilding one. The Darwin package needs the same root-level layout as the
+# Linux one: grpc-server, run.sh, the ggml dylibs and assets/ in ONE directory,
+# with lib/ for the rest. run.sh's Darwin branch execs grpc-server directly, so
+# _NSGetExecutablePath already names the package root; nothing else is needed
+# beyond putting the files there.
+UNAME_S=$(uname -s)
+if [ "$UNAME_S" = "Darwin" ]; then
+    echo "package.sh: Darwin dylib bundling is deferred to scripts/build/audio-cpp-darwin.sh"
+    ls -lah "$PACKAGE_DIR/" "$PACKAGE_DIR/assets/"
+    exit 0
+fi
+
+# The loader goes in the package ROOT, not in lib/. run.sh explains why at
+# length; the short version is that exec'ing it makes dirname(/proc/self/exe)
+# the directory it sits in, and both the ggml backend scan and the bundled:
+# asset lookup need that to be the package root.
+if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
+    cp -arfLv /lib64/ld-linux-x86-64.so.2 "$PACKAGE_DIR/ld.so"
+elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
+    cp -arfLv /lib/ld-linux-aarch64.so.1 "$PACKAGE_DIR/ld.so"
+else
+    echo "package.sh: unknown architecture" >&2
+    exit 1
+fi
+
+# THE LAYOUT ASSERTION. Everything else in this script checks that the package
+# can LINK. This checks that it can RESOLVE, which is a different property and
+# the one with no other guard on it.
+#
+# The loader, assets/ and the dlopened ggml objects only agree while they share
+# one directory, because run.sh execs the loader and all three are reached
+# through dirname(/proc/self/exe). A tidy-up that moves the loader into lib/,
+# following the llama-cpp layout, produces a package that builds, ships, and
+# then fails at run time with "model path does not exist: <pkg>/lib/assets/..."
+# or "Failed to initialize CPU backend". Fail the build instead.
+#
+# This sits immediately after the loader copy rather than at the end of the
+# script on purpose: everything below dereferences $PACKAGE_DIR/ld.so, so a
+# misplaced loader would otherwise surface as "No such file or directory" from
+# the validation gate and never reach an assertion that could explain it.
+if [ ! -f "$PACKAGE_DIR/ld.so" ]; then
+    echo "package.sh: the bundled loader must be at the package root, not in lib/." >&2
+    echo "package.sh: run.sh execs it, so its directory is dirname(/proc/self/exe)," >&2
+    echo "package.sh: which is where resolve_model_path looks for assets/ and where" >&2
+    echo "package.sh: ggml looks for the CPU variants." >&2
+    exit 1
+fi
+if [ ! -d "$PACKAGE_DIR/assets" ]; then
+    echo "package.sh: assets/ must sit beside the loader at the package root." >&2
+    exit 1
+fi
+# Only assert the ggml half when this build produced CPU variants at all: a
+# cublas or vulkan build links ggml statically and ships none.
+if compgen -G "$BUILD_DIR/bin/libggml-cpu-*.so" > /dev/null && \
+   ! compgen -G "$PACKAGE_DIR/libggml-cpu-*.so" > /dev/null; then
+    echo "package.sh: the build produced libggml-cpu-*.so but none reached the" >&2
+    echo "package.sh: package root, so ggml's scan of dirname(/proc/self/exe)" >&2
+    echo "package.sh: will find no CPU backend." >&2
+    exit 1
+fi
+
+# Libraries the host GPU driver stack owns. package_gpu_libs deliberately ships
+# the CUDA/Vulkan runtime but not the driver, because the driver has to match
+# the kernel module on whatever host runs the image. Copying the build host's
+# copy in would pin it to the build host instead.
+#
+# One regex, used by both the copy loop and the validation gate below. They have
+# to agree: exempting a library from the copy but not from the gate makes the
+# gate reject the very absence the copy loop just created.
+DRIVER_LIB_RE='^(libcuda\.so|libnvidia-)'
+# awk applies string-escape processing to a -v assignment before compiling the
+# regex, so a lone backslash is eaten and awk warns about it. Double them here
+# rather than keeping a second hand-written copy of the pattern, which is the
+# drift this single-source-of-truth exists to prevent.
+DRIVER_LIB_RE_AWK=${DRIVER_LIB_RE//\\/\\\\}
+is_driver_lib() {
+    [[ "$(basename "$1")" =~ $DRIVER_LIB_RE ]]
+}
+
+# Bundle the full dependency closure. grpc-server links the distro gRPC,
+# protobuf and absl stack; copying only the C/C++ runtime leaves the scratch
+# image unable to start. The walk runs over the PACKAGED binary, not the one in
+# $CURDIR, because its RUNPATH is $ORIGIN: only from inside the package does
+# libggml.so.0 resolve to the copy shipped above rather than to nothing.
+# The dlopened ggml objects are walked too, since a dependency of theirs that
+# grpc-server does not itself link would otherwise be missed.
+{
+    ldd "$PACKAGE_DIR/grpc-server"
+    for so in "$PACKAGE_DIR"/*.so*; do
+        [ -f "$so" ] || continue
+        ldd "$so"
+    done
+} | awk '$2 == "=>" && $3 ~ /^\// { print $3 }' | sort -u | \
+while read -r so; do
+    # Skip what is already inside the package: the ggml objects resolve through
+    # $ORIGIN and re-copying them into lib/ would ship two copies of each.
+    case "$so" in "$PACKAGE_DIR"/*) continue ;; esac
+    if is_driver_lib "$so"; then
+        echo "package.sh: leaving driver-owned library to the host: $so"
+        continue
+    fi
+    cp -arfLv "$so" "$PACKAGE_DIR/lib/"
+done
+
+GPU_LIB_SCRIPT="${REPO_ROOT}/scripts/build/package-gpu-libs.sh"
+if [ -f "$GPU_LIB_SCRIPT" ]; then
+    echo "Packaging GPU libraries for BUILD_TYPE=${BUILD_TYPE:-cpu}..."
+    # shellcheck source=/dev/null
+    source "$GPU_LIB_SCRIPT" "$PACKAGE_DIR/lib"
+    package_gpu_libs
+fi
+
+# Resolve every dependency through the same loader and library path the
+# from-scratch image uses. Two distinct failures are rejected, because the
+# loader can still fall back to the host's default directories: a dependency it
+# could not resolve at all, and one it resolved to a file OUTSIDE the package,
+# which would validate here and be absent in the image.
+#
+# The driver libraries are exempt from BOTH rejections, and that exemption is
+# load-bearing on GPU builds rather than tidiness. With BUILD_TYPE=cublas ggml
+# is static (no CPU_ALL_VARIANTS), and ggml/CMakeLists.txt defaults
+# GGML_CUDA_NO_VMM=OFF, so ggml-cuda links CUDA::cuda_driver and grpc-server
+# itself carries DT_NEEDED libcuda.so.1. The copy loop above deliberately leaves
+# that to the host, so inside the CUDA builder it resolves either to a host path
+# or to nothing. Without this exemption every cublas build would fail here and
+# CI would produce no image at all.
+#
+# LD_TRACE_LOADED_OBJECTS + LD_LIBRARY_PATH, NOT `ld.so --library-path --list`,
+# and the difference is not cosmetic. Measured on a stub object built to
+# DT_NEEDED an absent libcuda.so.1: `--list` refuses to trace at all, printing
+# "libdrivertest.so: error while loading shared libraries: libcuda.so.1: cannot
+# open shared object file" and exiting 127, so no per-library line is ever
+# produced and no exemption below could apply. The env form prints
+# "libcuda.so.1 => not found" and exits 0, which is what makes both the
+# unresolved rule and its driver exemption reachable. It is also closer to what
+# run.sh actually does, since run.sh exports LD_LIBRARY_PATH rather than passing
+# --library-path.
+validation_failed=0
+validate_object() {
+    local object="$1"
+    LD_TRACE_LOADED_OBJECTS=1 LD_LIBRARY_PATH="$PACKAGE_DIR/lib:$PACKAGE_DIR" \
+        "$PACKAGE_DIR/ld.so" "$object" | awk -v pkg="$PACKAGE_DIR/" -v obj="$object" \
+                                             -v driver_re="$DRIVER_LIB_RE_AWK" '
+        function base(p,   n, parts) { n = split(p, parts, "/"); return parts[n] }
+        $2 == "=>" && $3 == "not" {
+            if ($1 ~ driver_re) next
+            print "package.sh: unresolved dependency of " obj ": " $1 > "/dev/stderr"
+            bad = 1
+        }
+        $2 == "=>" && $3 ~ /^\// && index($3, pkg) != 1 {
+            if (base($3) ~ driver_re) next
+            print "package.sh: dependency of " obj " resolved outside the package: " $0 > "/dev/stderr"
+            bad = 1
+        }
+        END { exit bad }
+    '
+}
+
+validate_object "$PACKAGE_DIR/grpc-server" || validation_failed=1
+for so in "$PACKAGE_DIR"/*.so*; do
+    [ -f "$so" ] || continue
+    validate_object "$so" || validation_failed=1
+done
+if [ "$validation_failed" -ne 0 ]; then
+    exit 1
+fi
+
+echo "audio-cpp package contents:"
+ls -lah "$PACKAGE_DIR/" "$PACKAGE_DIR/lib/" "$PACKAGE_DIR/assets/"

backend/cpp/audio-cpp/result_map.cpp

@@ -0,0 +1,85 @@
+#include "result_map.h"
+
+#include "transcript_assembly.h"
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+void fill_transcript_result(const engine::runtime::TaskResult &result,
+                            int sample_rate, float duration_seconds,
+                            backend::TranscriptResult *out) {
+    // No null guard on `out`, deliberately. gRPC always hands a handler a
+    // response message, so a null here would be a programming error in a
+    // caller, and a guard that returned quietly would answer the client with an
+    // untouched, empty transcript and an OK status. That is the same
+    // indistinguishable-from-silence failure the rest of this unit exists to
+    // prevent; crashing on the developer's machine is the cheaper outcome.
+    std::vector<Span> speech_segments;
+    speech_segments.reserve(result.speech_segments.size());
+    for (const auto &segment : result.speech_segments) {
+        speech_segments.push_back(
+            Span{segment.span.start_sample, segment.span.end_sample});
+    }
+
+    std::vector<SpeakerSpan> speaker_turns;
+    speaker_turns.reserve(result.speaker_turns.size());
+    for (const auto &turn : result.speaker_turns) {
+        speaker_turns.push_back(
+            SpeakerSpan{Span{turn.span.start_sample, turn.span.end_sample},
+                        turn.speaker_id});
+    }
+
+    std::vector<WordSpan> words;
+    words.reserve(result.word_timestamps.size());
+    for (const auto &word : result.word_timestamps) {
+        words.push_back(
+            WordSpan{Span{word.span.start_sample, word.span.end_sample},
+                     word.word});
+    }
+
+    // The ONLY read of transcript text in this function, and the only one there
+    // may ever be. See THE RULE in the header.
+    const std::string text =
+        result.text_output.has_value() ? result.text_output->text : std::string();
+
+    const AssembledTranscript assembled = assemble_transcript(
+        text, speech_segments, speaker_turns, words, sample_rate);
+
+    out->set_text(assembled.text);
+    // language has no source inside transcript_assembly, which is span-shaped
+    // only, so it is read straight off the engine result here. Left untouched
+    // when the family reported no text output at all: an empty string would be
+    // indistinguishable from a family that genuinely detected no language, and
+    // the field is documented as optional.
+    if (result.text_output.has_value()) {
+        out->set_language(result.text_output->language);
+    }
+    out->set_duration(duration_seconds);
+
+    // Cleared rather than appended to. A caller that fills the same message
+    // twice (a stream's final_result being rebuilt, say) would otherwise emit
+    // every segment twice, and the second call's ids would restart at 0 and
+    // collide with the first call's.
+    out->clear_segments();
+    for (const auto &segment : assembled.segments) {
+        auto *out_segment = out->add_segments();
+        out_segment->set_id(segment.id);
+        // NANOSECONDS. TranscriptSegment and TranscriptWord are the only
+        // messages in backend.proto that use them; VADSegment and DiarizeSegment
+        // are float seconds. assemble_transcript has already converted.
+        out_segment->set_start(segment.start_ns);
+        out_segment->set_end(segment.end_ns);
+        out_segment->set_text(segment.text);
+        out_segment->set_speaker(segment.speaker);
+        for (const auto &word : segment.words) {
+            auto *out_word = out_segment->add_words();
+            out_word->set_start(word.start_ns);
+            out_word->set_end(word.end_ns);
+            out_word->set_text(word.text);
+        }
+    }
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/result_map.h

@@ -0,0 +1,37 @@
+#pragma once
+
+// Converts engine::runtime results into LocalAI proto messages. All of the
+// non-trivial shaping lives in transcript_assembly, which is stdlib-only and
+// unit tested; this unit is the thin engine-typed boundary around it.
+
+#include "backend.pb.h"
+
+#include "engine/framework/runtime/session.h"
+
+namespace audiocpp_backend {
+
+// Fills text, language, duration, segments and per-segment words.
+//
+// THE RULE: the top-level text is TaskResult.text_output verbatim. It is never
+// derived from segments or words. audio.cpp carries transcript text in
+// text_output and nowhere else: speech_segments, speaker_turns and
+// word_timestamps carry spans and labels and no text at all. Deriving the
+// transcript from them therefore returns an EMPTY text for every producer that
+// reports segments without word timing, which real VibeVoice diarized ASR does.
+// An earlier attempt at this backend shipped exactly that bug. assemble_transcript
+// enforces the rule and is heavily tested; this unit's job is not to re-derive
+// it but to not undo it at the proto boundary.
+//
+// `sample_rate` is the rate the result's spans are expressed in, which is the
+// rate of the AudioBuffer that was handed to the session, NOT the rate of the
+// file the caller uploaded. Those differ whenever read_audio_file resampled,
+// which is why the handler passes the buffer's rate rather than the file's.
+//
+// Segments are replaced, not appended to, so a message filled twice does not
+// accumulate. `out` must be non-null and is not checked; see the note at the
+// top of the implementation for why that is not an oversight.
+void fill_transcript_result(const engine::runtime::TaskResult &result,
+                            int sample_rate, float duration_seconds,
+                            backend::TranscriptResult *out);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/result_map_ctest.cpp

@@ -0,0 +1,247 @@
+// Tests for result_map, the engine-to-proto boundary.
+//
+// NAMED _ctest AND NOT _test ON PURPOSE. backend/cpp/run-unit-tests.sh globs
+// every *_test.cpp under backend/cpp/ and compiles it as a single standalone
+// translation unit with no include path beyond its own directory. This file
+// needs backend.pb.h and the audio.cpp framework headers, so it is built and
+// run by ctest instead:
+//
+//   make -C backend/cpp/audio-cpp test-engine
+//
+// Renaming it to *_test.cpp would break the standalone suite for every backend.
+//
+// What is worth testing here is exactly one thing, and it is not the field
+// copying: THE RULE. TaskResult carries transcript text in text_output and
+// nowhere else, so the proto's text must be that string verbatim. An earlier
+// attempt at this backend derived it from the segments, which returns an empty
+// transcript for every producer that reports segments without word timing.
+// transcript_assembly already enforces the rule and is tested on its own; these
+// checks are here so that a future edit cannot undo it at the boundary.
+
+#include "result_map.h"
+
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+namespace rt = engine::runtime;
+
+static const int kRate = 16000;
+
+static rt::SpeechSegment speech(std::int64_t start, std::int64_t end) {
+    rt::SpeechSegment segment;
+    segment.span.start_sample = start;
+    segment.span.end_sample = end;
+    return segment;
+}
+
+static rt::SpeakerTurn turn(std::int64_t start, std::int64_t end,
+                            const std::string &speaker) {
+    rt::SpeakerTurn out;
+    out.span.start_sample = start;
+    out.span.end_sample = end;
+    out.speaker_id = speaker;
+    return out;
+}
+
+static rt::WordTimestamp word(std::int64_t start, std::int64_t end,
+                              const std::string &text) {
+    rt::WordTimestamp out;
+    out.span.start_sample = start;
+    out.span.end_sample = end;
+    out.word = text;
+    return out;
+}
+
+// THE REGRESSION. A diarized ASR result: real text, real speaker turns, and no
+// word timing at all. This is the vibevoice_asr shape, and it is the one that
+// came back empty before.
+static void test_text_survives_segments_without_words() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "hello there general kenobi";
+    transcript.language = "en";
+    result.text_output = transcript;
+    result.speaker_turns.push_back(turn(0, 16000, "speaker_0"));
+    result.speaker_turns.push_back(turn(16000, 32000, "speaker_1"));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 2.0f, &out);
+
+    check(out.text() == "hello there general kenobi",
+          "diarized result keeps text_output verbatim");
+    check(out.language() == "en", "language comes from text_output");
+    check(out.segments_size() == 2, "both speaker turns become segments");
+    if (out.segments_size() == 2) {
+        check(out.segments(0).speaker() == "speaker_0",
+              "first segment keeps its own speaker label");
+        check(out.segments(1).speaker() == "speaker_1",
+              "second segment keeps its own speaker label");
+        check(out.segments(1).start() == 1000000000LL,
+              "segment start is nanoseconds, not samples");
+        check(out.segments(1).end() == 2000000000LL,
+              "segment end is nanoseconds, not samples");
+    }
+}
+
+// The same rule seen from the other side: text present, spans present, and the
+// per-segment text empty because there is nothing truthful to split. A boundary
+// that derived the top-level text from these segments would produce "".
+static void test_speech_segments_do_not_supply_the_text() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "one two three";
+    result.text_output = transcript;
+    result.speech_segments.push_back(speech(0, 8000));
+    result.speech_segments.push_back(speech(8000, 16000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text() == "one two three",
+          "speech segments without words do not empty the transcript");
+    check(out.segments_size() == 2, "both speech segments are emitted");
+    if (out.segments_size() == 2) {
+        check(out.segments(0).text().empty() && out.segments(1).text().empty(),
+              "per-segment text stays empty when there is no word timing");
+    }
+}
+
+static void test_words_reach_the_proto_in_nanoseconds() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "hi there";
+    result.text_output = transcript;
+    result.word_timestamps.push_back(word(0, 8000, "hi"));
+    result.word_timestamps.push_back(word(8000, 16000, "there"));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text() == "hi there", "word-timed result keeps text_output");
+    check(out.segments_size() == 1, "words with no spans yield one covering segment");
+    if (out.segments_size() == 1) {
+        const auto &segment = out.segments(0);
+        check(segment.words_size() == 2, "both words are emitted");
+        if (segment.words_size() == 2) {
+            check(segment.words(0).text() == "hi", "first word text");
+            check(segment.words(0).start() == 0, "first word start");
+            check(segment.words(0).end() == 500000000LL,
+                  "first word end is 0.5 s in nanoseconds");
+            check(segment.words(1).start() == 500000000LL, "second word start");
+            check(segment.words(1).end() == 1000000000LL, "second word end");
+        }
+    }
+}
+
+// The buffer's rate, not the file's, is what the spans mean. Passing 8000 for
+// the same spans has to halve every timestamp, which is what makes resampling
+// the input at read time load-bearing rather than cosmetic.
+static void test_sample_rate_scales_the_timestamps() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "x";
+    result.text_output = transcript;
+    result.speech_segments.push_back(speech(0, 8000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, 8000, 1.0f, &out);
+
+    check(out.segments_size() == 1, "one segment at 8 kHz");
+    if (out.segments_size() == 1) {
+        check(out.segments(0).end() == 1000000000LL,
+              "8000 samples at 8 kHz is one second");
+    }
+}
+
+static void test_duration_is_carried_through() {
+    rt::TaskResult result;
+    rt::Transcript transcript;
+    transcript.text = "x";
+    result.text_output = transcript;
+
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 14.07f, &out);
+
+    check(out.duration() > 14.06f && out.duration() < 14.08f,
+          "duration is set from the argument");
+}
+
+// No text output at all. A VAD-shaped result reaching this boundary must not
+// invent a transcript, and must not overwrite a language the caller had already
+// decided on.
+static void test_missing_text_output_leaves_language_alone() {
+    rt::TaskResult result;
+    result.speech_segments.push_back(speech(0, 16000));
+
+    backend::TranscriptResult out;
+    out.set_language("it");
+    fill_transcript_result(result, kRate, 1.0f, &out);
+
+    check(out.text().empty(), "no text_output means no text");
+    check(out.language() == "it",
+          "a result with no text_output does not clear the language");
+    check(out.segments_size() == 1, "spans are still emitted");
+}
+
+// Filling the same message twice must replace, not accumulate: the second
+// call's ids restart at 0 and would collide with the first call's.
+static void test_refilling_replaces_the_segments() {
+    rt::TaskResult first;
+    rt::Transcript transcript;
+    transcript.text = "first";
+    first.text_output = transcript;
+    first.speech_segments.push_back(speech(0, 16000));
+    first.speech_segments.push_back(speech(16000, 32000));
+
+    backend::TranscriptResult out;
+    fill_transcript_result(first, kRate, 2.0f, &out);
+
+    rt::TaskResult second;
+    rt::Transcript replacement;
+    replacement.text = "second";
+    second.text_output = replacement;
+    second.speech_segments.push_back(speech(0, 16000));
+    fill_transcript_result(second, kRate, 1.0f, &out);
+
+    check(out.text() == "second", "the second fill replaces the text");
+    check(out.segments_size() == 1,
+          "the second fill replaces the segments instead of appending");
+}
+
+static void test_empty_result_is_empty() {
+    rt::TaskResult result;
+    backend::TranscriptResult out;
+    fill_transcript_result(result, kRate, 0.0f, &out);
+
+    check(out.text().empty(), "empty result has no text");
+    check(out.segments_size() == 0, "empty result has no segments");
+}
+
+int main() {
+    test_text_survives_segments_without_words();
+    test_speech_segments_do_not_supply_the_text();
+    test_words_reach_the_proto_in_nanoseconds();
+    test_sample_rate_scales_the_timestamps();
+    test_duration_is_carried_through();
+    test_missing_text_output_leaves_language_alone();
+    test_refilling_replaces_the_segments();
+    test_empty_result_is_empty();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all result_map checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/run.sh

@@ -0,0 +1,58 @@
+#!/bin/bash
+# Entry point for the audio-cpp backend image and for BACKEND_BINARY mode.
+#
+# The image's final stage is FROM scratch, so the package root is / and there is
+# no system loader, no system libc and no fallback library path. Everything the
+# process opens has to be inside the package, and it has to be findable by the
+# two mechanisms that actually do the finding: the dynamic linker, and
+# audio.cpp's own directory scans.
+set -e
+
+CURDIR=$(dirname "$(realpath "$0")")
+
+if [ "$(uname -s)" = "Darwin" ]; then
+	export DYLD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$DYLD_LIBRARY_PATH"
+	exec "$CURDIR/grpc-server" "$@"
+fi
+
+# $CURDIR is on the path as well as $CURDIR/lib: the ggml shared objects the
+# CPU-all-variants build produces sit in the package root, next to the binary,
+# not in lib/. See the comment below for why they cannot live in lib/.
+export LD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$LD_LIBRARY_PATH"
+
+# THE BUNDLED LOADER IS AT THE PACKAGE ROOT, NOT AT lib/ld.so. DO NOT MOVE IT.
+#
+# Exec'ing the loader is what pins the bundled glibc to the matching ld.so, and
+# every other C++ backend here does it. The cost is that /proc/self/exe then
+# names the LOADER rather than grpc-server, and this backend has two consumers
+# of /proc/self/exe that both have to land on the package root:
+#
+#   - ggml's backend registry DISCOVERS the per-microarch libggml-cpu-*.so by
+#     scanning dirname(/proc/self/exe) and the current directory. Those files
+#     are dlopened, never linked, so no library path and no RUNPATH reaches
+#     them: they have to be in a directory ggml scans.
+#   - resolve_model_path() turns "bundled:<name>" into
+#     dirname(/proc/self/exe)/assets/<name>, which is how the bundled
+#     silero_vad and marblenet_vad models resolve with nothing downloaded.
+#
+# backend/cpp/llama-cpp/package.sh answers the first of these by keeping
+# lib/ld.so and moving the ggml objects INTO lib/. That does not generalise
+# here, because it would also drag assets/ into lib/ to keep the second
+# consumer working. Putting the loader in the package root instead makes
+# dirname(/proc/self/exe) the package root, so the binary, the ggml objects and
+# assets/ all sit in the one directory that all three mechanisms agree on.
+#
+# The ggml half has a second chance that the bundled: half does not: LocalAI
+# sets the backend process cwd to the directory holding run.sh
+# (pkg/model/process.go), so ggml's fs::current_path() fallback would find the
+# objects in normal operation whatever the loader's placement. That fallback is
+# worth little here. It holds only for the launcher that sets that cwd, it is
+# gone the moment anyone runs the binary by hand or through a wrapper that
+# chdirs, and resolve_model_path has no equivalent, which would leave the only
+# zero-download path in this backend resting on it. Rooting the loader is the
+# one layout where all three mechanisms agree without depending on the cwd.
+if [ -f "$CURDIR/ld.so" ]; then
+	exec "$CURDIR/ld.so" "$CURDIR/grpc-server" "$@"
+fi
+
+exec "$CURDIR/grpc-server" "$@"

backend/cpp/audio-cpp/stem_selection.cpp

@@ -0,0 +1,129 @@
+#include "stem_selection.h"
+
+#include <cstddef>
+#include <filesystem>
+#include <string>
+
+namespace audiocpp_backend {
+namespace {
+
+// The stem every separation family this backend can reach names its lead vocal
+// track, and the one a caller who names no stem almost always wants: it is what
+// the OpenAI-shaped "isolate the voice" request means. htdemucs (drums, bass,
+// other, vocals) and mel_band_roformer (vocals, instrumental) both have it, and
+// in htdemucs's case it is NOT the first output, which is the whole reason this
+// preference is written down rather than left as "take index 0".
+const char *const kPreferredStem = "vocals";
+
+std::string join_names(const std::vector<std::string> &names) {
+    std::string out;
+    for (const auto &name : names) {
+        if (!out.empty()) {
+            out += ", ";
+        }
+        out += name;
+    }
+    return out;
+}
+
+// Whether a model-supplied stem name can be used as one component of a file
+// name. Deliberately a whitelist of refusals rather than a sanitiser: silently
+// rewriting "vo/cals" to "cals" would make the file the caller receives
+// disagree with the name they would have to ask for.
+bool name_is_writable(const std::string &name) {
+    if (name.empty() || name == "." || name == "..") {
+        return false;
+    }
+    // Control bytes, NUL above all. GGUF strings are length prefixed and demucs
+    // reads its source names out of JSON, which can encode one, so a
+    // std::string holding an embedded NUL survives all the way here. Two such
+    // names differing only AFTER the NUL are distinct std::strings, so the
+    // duplicate check below waves them through, and then path::c_str()
+    // truncates both at the NUL and they open the same file: precisely the
+    // silent overwrite the duplicate check exists to prevent, with the ".wav"
+    // stripped off as well. The rest of the range goes with it, since a newline
+    // or an escape sequence in a file name is a terminal and log injection
+    // nuisance with no legitimate use.
+    for (const char byte : name) {
+        const auto value = static_cast<unsigned char>(byte);
+        if (value < 0x20 || value == 0x7f) {
+            return false;
+        }
+    }
+    // Both separators, not just the host's. A GGUF is a downloaded file and its
+    // strings are not this host's to trust, so a name written on Windows must
+    // not become a directory traversal wherever the check happens to run.
+    return name.find('/') == std::string::npos &&
+           name.find('\\') == std::string::npos;
+}
+
+} // namespace
+
+StemChoice select_named_output(const std::vector<std::string> &names,
+                               const std::string &requested) {
+    StemChoice choice;
+    if (names.empty()) {
+        // Not an error here. The caller distinguishes "this family produces one
+        // unnamed output" from "this family produced nothing", and only it can
+        // tell them apart.
+        return choice;
+    }
+
+    // Every name is checked, not merely the selected one, because every stem is
+    // written. A bad name in the fourth output would otherwise be discovered
+    // only after three files had already been created.
+    for (std::size_t i = 0; i < names.size(); ++i) {
+        if (!name_is_writable(names[i])) {
+            choice.error = "audio-cpp: this model names an output stem '" +
+                           names[i] +
+                           "' that cannot be used as a file name; stems: " +
+                           join_names(names);
+            return choice;
+        }
+        for (std::size_t seen = 0; seen < i; ++seen) {
+            if (names[seen] == names[i]) {
+                choice.error =
+                    "audio-cpp: this model produces two output stems both named '" +
+                    names[i] + "'; one would silently overwrite the other";
+                return choice;
+            }
+        }
+    }
+
+    if (!requested.empty()) {
+        for (std::size_t i = 0; i < names.size(); ++i) {
+            if (names[i] == requested) {
+                choice.index = static_cast<int>(i);
+                return choice;
+            }
+        }
+        choice.error = "audio-cpp: no stem named '" + requested +
+                       "' in this model's output; available stems: " +
+                       join_names(names);
+        return choice;
+    }
+
+    for (std::size_t i = 0; i < names.size(); ++i) {
+        if (names[i] == kPreferredStem) {
+            choice.index = static_cast<int>(i);
+            return choice;
+        }
+    }
+    choice.index = 0;
+    return choice;
+}
+
+std::string sibling_stem_path(const std::string &dst, const std::string &name) {
+    const std::filesystem::path path(dst);
+    // The dst extension is reused rather than forced to ".wav" so the siblings
+    // look like the file the caller named. write_audio_file writes WAV bytes
+    // whatever the extension says, for dst as much as for the siblings, so this
+    // keeps the set consistent instead of making the siblings honest about a
+    // format dst is already lying about.
+    const std::string extension =
+        path.has_extension() ? path.extension().string() : std::string(".wav");
+    return (path.parent_path() / (path.stem().string() + "." + name + extension))
+        .string();
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stem_selection.h

@@ -0,0 +1,63 @@
+#pragma once
+
+// Decides which of a separation model's named stems the AudioTransform response
+// carries in `dst`, and names the sibling files every other stem is written to.
+//
+// It exists because AudioTransformResult carries ONE dst while htdemucs and
+// mel_band_roformer produce several named outputs from a single run. Running
+// once per stem would cost four full inferences for a four stem model, so the
+// handler runs once, writes every stem beside dst, and puts the selected one in
+// dst itself.
+//
+// Standard library only, so backend/cpp/run-unit-tests.sh compiles and runs its
+// test without an audio.cpp checkout. grpc-server.cpp flattens the engine's
+// NamedAudioBuffer list into the plain name vector taken here; nothing in this
+// unit knows about engine::runtime.
+
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+struct StemChoice {
+    // Index into the `names` vector. -1 means nothing was chosen, which happens
+    // for an empty list (the family produced a single unnamed output) and for
+    // every refusal.
+    //
+    // THE CONTRACT THE CALLER INDEXES ON: when `error` is empty and `names` was
+    // not, this is always a valid index into `names`. It is never -1 in that
+    // case, so a caller that checks `error` first can index without a further
+    // guard, and a caller that does not check `error` first would index with a
+    // negative value. Check the error.
+    int index = -1;
+    // Non-empty when the request must be refused, and suitable verbatim as an
+    // INVALID_ARGUMENT message. Two things land here: the caller named a stem
+    // this model does not produce, and the model named stems that cannot both
+    // be written (an unusable file name, or two stems sharing one).
+    std::string error;
+};
+
+// Picks the stem that goes to dst. Preference order: an explicit `requested`,
+// then "vocals", then the first output.
+//
+// An explicit but unknown `requested` is an ERROR rather than a fallback. A
+// caller who asks for "drums" and silently receives "vocals" gets a 200 and a
+// wrong file, which is the failure mode nobody can see; the message therefore
+// lists the stem names this model really has.
+//
+// The names are also validated, because they come from the MODEL (htdemucs
+// reads them from the GGUF's config.sources) and each one becomes a component
+// of a file path this backend writes. A name carrying a path separator would
+// write outside the caller's output directory, and two stems sharing a name
+// would silently overwrite each other. Both are refused before anything is
+// written, which is also why selection has to happen before the first write
+// rather than after the loop: a refused request must leave no files behind.
+StemChoice select_named_output(const std::vector<std::string> &names,
+                               const std::string &requested);
+
+// "/generated/transform-1.wav" + "drums" -> "/generated/transform-1.drums.wav".
+// A dst with no extension gets ".wav", since that is what write_audio_file
+// produces whatever the caller called the file.
+std::string sibling_stem_path(const std::string &dst, const std::string &name);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stem_selection_test.cpp

@@ -0,0 +1,250 @@
+// Unit tests for stem_selection. Standard library only. The harness
+// (backend/cpp/run-unit-tests.sh) compiles this as a single translation unit,
+// so the implementation is included directly.
+//
+// What is actually at stake here: AudioTransformResult carries one dst, a
+// separation model produces several stems, and the caller cannot see which one
+// they got. Every check below is about a wrong file arriving with a 200.
+
+#include "stem_selection.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_equal(const std::string &got, const std::string &want,
+                        const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\", want \"" + want + "\")");
+}
+
+using namespace audiocpp_backend;
+
+// htdemucs's real source order, taken from its GGUF config.sources. vocals is
+// LAST, which is why "first output" is not the default.
+static const std::vector<std::string> kDemucs = {"drums", "bass", "other",
+                                                 "vocals"};
+// mel_band_roformer's, where vocals is first.
+static const std::vector<std::string> kRoformer = {"vocals", "instrumental"};
+
+static void test_default_selection() {
+    const auto demucs = select_named_output(kDemucs, "");
+    check(demucs.error.empty(), "an unrequested selection is not an error");
+    check(demucs.index == 3, "no stem asked for picks vocals, not the first output");
+
+    const auto roformer = select_named_output(kRoformer, "");
+    check(roformer.index == 0, "vocals is picked when it is already first");
+
+    // No vocals anywhere: the first output is the documented fallback.
+    const auto novocals = select_named_output({"accompaniment", "drums"}, "");
+    check(novocals.index == 0 && novocals.error.empty(),
+          "a family with no vocals stem falls back to the first output");
+
+    // Substring matches must not count: "vocals_2" is a different stem.
+    const auto near = select_named_output({"vocals_2", "backing"}, "");
+    check(near.index == 0 && near.error.empty(),
+          "'vocals_2' is not 'vocals', so the fallback and not the preference applies");
+    const auto near_second = select_named_output({"backing", "vocals_2"}, "");
+    check(near_second.index == 0,
+          "a near miss on the preferred name does not pull it to the front");
+}
+
+static void test_explicit_selection() {
+    for (int i = 0; i < 4; ++i) {
+        const auto choice = select_named_output(kDemucs, kDemucs[static_cast<size_t>(i)]);
+        check(choice.index == i && choice.error.empty(),
+              "explicit '" + kDemucs[static_cast<size_t>(i)] + "' selects its own index");
+    }
+    // Including the one the default would have chosen anyway: asking for it
+    // must not be treated as "no request".
+    const auto vocals = select_named_output(kDemucs, "vocals");
+    check(vocals.index == 3, "explicitly asking for vocals still selects vocals");
+}
+
+static void test_unknown_stem_is_refused() {
+    const auto choice = select_named_output(kDemucs, "kazoo");
+    check(choice.index == -1, "an unknown stem selects nothing");
+    check(!choice.error.empty(), "an unknown stem is refused rather than substituted");
+    check(choice.error.find("kazoo") != std::string::npos,
+          "the refusal names the stem that was asked for");
+    // The real names, so the caller can fix the request without guessing.
+    for (const auto &name : kDemucs) {
+        check(choice.error.find(name) != std::string::npos,
+              "the refusal lists the real stem '" + name + "'");
+    }
+    check(choice.error.find("drums, bass, other, vocals") != std::string::npos,
+          "the refusal lists the stems in the model's own order");
+
+    // Case matters: the engine's ids are exact, so a wrong case is a wrong name
+    // rather than a near miss to be forgiven.
+    const auto wrong_case = select_named_output(kDemucs, "Vocals");
+    check(wrong_case.index == -1 && !wrong_case.error.empty(),
+          "stem names are matched case sensitively");
+}
+
+static void test_no_named_outputs() {
+    const auto choice = select_named_output({}, "");
+    check(choice.index == -1, "an empty output list selects nothing");
+    check(choice.error.empty(),
+          "an empty output list is not an error here: the caller decides");
+    const auto requested = select_named_output({}, "vocals");
+    check(requested.index == -1 && requested.error.empty(),
+          "an empty output list stays the caller's decision even when a stem was asked for");
+}
+
+static void test_unwritable_names_are_refused() {
+    // Model-supplied names become file path components. A separator would write
+    // outside the caller's output directory.
+    const std::vector<std::string> traversal = {"vocals", "../../etc/passwd"};
+    const auto escaped = select_named_output(traversal, "vocals");
+    check(escaped.index == -1 && !escaped.error.empty(),
+          "a stem name containing a path separator is refused");
+    check(escaped.error.find("../../etc/passwd") != std::string::npos,
+          "the refusal names the offending stem");
+
+    check(!select_named_output({"vo\\cals", "drums"}, "").error.empty(),
+          "a backslash separator is refused too");
+    check(!select_named_output({"drums", ""}, "").error.empty(),
+          "an empty stem name is refused");
+    check(!select_named_output({"drums", "."}, "").error.empty(),
+          "a stem named '.' is refused");
+    check(!select_named_output({"drums", ".."}, "").error.empty(),
+          "a stem named '..' is refused");
+
+    // The check covers EVERY name, not only the selected one: all of them are
+    // written, so a bad fourth name must not be found after three files exist.
+    const auto late = select_named_output({"vocals", "drums", "bass", "a/b"}, "vocals");
+    check(late.index == -1 && !late.error.empty(),
+          "an unwritable name after the selected one still refuses the whole request");
+
+    // Control bytes, and the NUL case is why the whole range is refused. These
+    // two names are DIFFERENT std::strings, so the duplicate check does not
+    // fire, yet both truncate to "vocals" at path::c_str() and would open one
+    // file: the silent overwrite the duplicate check exists to prevent, with
+    // the ".wav" stripped off into the bargain.
+    const std::string nul_a("vocals\0drums", 12);
+    const std::string nul_b("vocals\0bass", 11);
+    check(nul_a != nul_b, "the two NUL names really are distinct std::strings");
+    check(std::string(nul_a.c_str()) == "vocals" &&
+              std::string(nul_b.c_str()) == "vocals",
+          "and both truncate to the same C string, which is the hazard");
+    const auto nul_pair = select_named_output({nul_a, nul_b}, "");
+    check(nul_pair.index == -1 && !nul_pair.error.empty(),
+          "two stem names differing only after an embedded NUL are refused");
+    check(!select_named_output({"drums", std::string("vo\0cals", 7)}, "").error.empty(),
+          "a single embedded NUL is refused on its own");
+    check(!select_named_output({"drums", "voc\nals"}, "").error.empty(),
+          "a newline in a stem name is refused");
+    check(!select_named_output({"drums", "voc\tals"}, "").error.empty(),
+          "a tab in a stem name is refused");
+    check(!select_named_output({"drums", "voc\033[31mals"}, "").error.empty(),
+          "an escape sequence in a stem name is refused");
+    check(!select_named_output({"drums", "voc\177als"}, "").error.empty(),
+          "DEL in a stem name is refused");
+
+    // The boundary below the refused range is the space, which is an ordinary
+    // file name character and must stay usable, or this check would be
+    // refusing real stem names.
+    const auto spaced = select_named_output({"lead vocals", "drums"}, "lead vocals");
+    check(spaced.index == 0 && spaced.error.empty(),
+          "a space is not a control character and stays usable");
+    // And every byte above DEL: a UTF-8 stem name is ordinary, and signed char
+    // would make those bytes compare as negative.
+    const auto utf8 = select_named_output({"vocals", "b\xc3\xa4sse"}, "b\xc3\xa4sse");
+    check(utf8.index == 1 && utf8.error.empty(),
+          "a UTF-8 stem name is not mistaken for a control character");
+
+    // A leading dot is not a traversal and must stay usable.
+    const auto dotted = select_named_output({".vocals", "drums"}, ".vocals");
+    check(dotted.index == 0 && dotted.error.empty(),
+          "a leading dot in a stem name is allowed");
+}
+
+static void test_duplicate_names_are_refused() {
+    const auto choice = select_named_output({"vocals", "drums", "vocals"}, "vocals");
+    check(choice.index == -1 && !choice.error.empty(),
+          "two stems sharing a name are refused: one file would overwrite the other");
+    check(choice.error.find("vocals") != std::string::npos,
+          "the duplicate refusal names the repeated stem");
+}
+
+static void test_sibling_paths() {
+    check_equal(sibling_stem_path("/generated/transform-1.wav", "drums"),
+                "/generated/transform-1.drums.wav", "sibling beside an absolute dst");
+    check_equal(sibling_stem_path("sep.wav", "vocals"), "sep.vocals.wav",
+                "sibling of a bare file name has no directory");
+    check_equal(sibling_stem_path("/out/sep", "vocals"), "/out/sep.vocals.wav",
+                "an extensionless dst gets .wav");
+    check_equal(sibling_stem_path("/out/take.2.wav", "bass"), "/out/take.2.bass.wav",
+                "only the final extension is treated as the extension");
+    check_equal(sibling_stem_path("/out/sep.WAV", "bass"), "/out/sep.bass.WAV",
+                "the caller's extension spelling is preserved");
+    check_equal(sibling_stem_path("/a b/c d.wav", "other"), "/a b/c d.other.wav",
+                "spaces in the destination survive");
+
+    // The property that matters: no stem can ever be written over dst itself,
+    // or the "dst holds the selected stem" contract would depend on write order.
+    const std::string dst = "/out/sep.wav";
+    for (const auto &name : kDemucs) {
+        check(sibling_stem_path(dst, name) != dst,
+              "the sibling for '" + name + "' is not dst itself");
+    }
+    // Distinct stems must land in distinct files.
+    check(sibling_stem_path(dst, "drums") != sibling_stem_path(dst, "bass"),
+          "two stems get two different sibling paths");
+}
+
+// The contract grpc-server.cpp indexes on: an accepted choice over a non-empty
+// name list is always in range, so the handler needs no bounds guard of its own.
+// A -1 reaching the subscript would become a colossal size_t.
+static void test_accepted_index_is_always_in_range() {
+    const std::vector<std::vector<std::string>> lists = {
+        kDemucs, kRoformer, {"solo"}, {"accompaniment", "drums"}, {"a", "b", "c"}};
+    const std::vector<std::string> requests = {"", "vocals", "drums", "solo", "c",
+                                               "kazoo", "..", "a/b"};
+    for (const auto &names : lists) {
+        for (const auto &requested : requests) {
+            const auto choice = select_named_output(names, requested);
+            if (!choice.error.empty()) {
+                check(choice.index == -1,
+                      "a refusal never carries an index (request '" + requested + "')");
+                continue;
+            }
+            check(choice.index >= 0 &&
+                      choice.index < static_cast<int>(names.size()),
+                  "an accepted choice is in range (request '" + requested + "')");
+            // And the selected name is the one that was asked for, when one was.
+            if (!requested.empty()) {
+                check(names[static_cast<size_t>(choice.index)] == requested,
+                      "an accepted explicit request selects that exact name");
+            }
+        }
+    }
+}
+
+int main() {
+    test_accepted_index_is_always_in_range();
+    test_default_selection();
+    test_explicit_selection();
+    test_unknown_stem_is_refused();
+    test_no_named_outputs();
+    test_unwritable_names_are_refused();
+    test_duplicate_names_are_refused();
+    test_sibling_paths();
+    if (failures != 0) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all stem_selection checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/stream_delta.cpp

@@ -0,0 +1,204 @@
+#include "stream_delta.h"
+
+namespace audiocpp_backend {
+namespace {
+
+// True when `text` begins with `prefix`. An empty prefix matches everything,
+// which is what makes the first fragment take the cumulative branch and the
+// incremental branch alike: they agree there.
+bool starts_with(const std::string &text, const std::string &prefix) {
+    return text.size() >= prefix.size() &&
+           text.compare(0, prefix.size(), prefix) == 0;
+}
+
+// Number of leading bytes of `text` that cannot BEGIN a character, i.e. orphan
+// continuation bytes with no lead byte in front of them.
+//
+// They are unrecoverable rather than early: the byte that would have led them
+// has already gone past, and nothing can be prepended to a fragment after the
+// fact. Holding them would stall the stream for good, and emitting them puts
+// invalid UTF-8 on the wire, so the caller DROPS them. Losing a byte keeps the
+// stream alive; emitting one ends it, and takes the final_result still to come
+// with it.
+std::size_t utf8_orphan_prefix_length(const std::string &text) {
+    std::size_t index = 0;
+    while (index < text.size() &&
+           (static_cast<unsigned char>(text[index]) & 0xC0) == 0x80) {
+        ++index;
+    }
+    return index;
+}
+
+// Length of the longest prefix of `text` that does NOT end inside a multi-byte
+// UTF-8 sequence, i.e. the most that can go on the wire without splitting a
+// character in half.
+//
+// Only the TRAILING sequence is examined; the leading side is
+// utf8_orphan_prefix_length's job. Bytes in the MIDDLE are neither one's
+// business: a family that emitted a malformed sequence inside its own text
+// cannot be repaired here without deleting part of that transcript, and the
+// stream is lost anyway, because final_result.text carries the same bytes
+// through the same proto3 string field.
+//
+// Anything that can never complete is reported as complete, so it goes out
+// rather than being held forever: a lead byte the encoding does not define, and
+// a run of five or more continuation bytes, are both passed through. A tracker
+// that stalled on undecodable input would turn one bad byte into a permanently
+// silent stream, which is worse than the bad byte.
+std::size_t utf8_complete_prefix_length(const std::string &text) {
+    std::size_t index = text.size();
+    std::size_t continuations = 0;
+    while (index > 0 && continuations < 4) {
+        const auto byte = static_cast<unsigned char>(text[index - 1]);
+        if ((byte & 0xC0) == 0x80) {
+            --index;
+            ++continuations;
+            continue;
+        }
+        std::size_t needed = 1;
+        if ((byte & 0x80) == 0x00) {
+            needed = 1;
+        } else if ((byte & 0xE0) == 0xC0) {
+            needed = 2;
+        } else if ((byte & 0xF0) == 0xE0) {
+            needed = 3;
+        } else if ((byte & 0xF8) == 0xF0) {
+            needed = 4;
+        } else {
+            // Not a lead byte this encoding defines, so nothing is waiting on
+            // it and it must not be held.
+            needed = 1;
+        }
+        if (continuations + 1 >= needed) {
+            return text.size();
+        }
+        // The trailing sequence is short by at least one byte: cut before its
+        // lead byte and keep the rest for the next fragment.
+        return index - 1;
+    }
+    return text.size();
+}
+
+} // namespace
+
+std::string TranscriptDeltaTracker::release(const std::string &fragment) {
+    // Held-back bytes go in FRONT of whatever arrived next, or the character
+    // they begin is reassembled in the wrong order.
+    std::string candidate = pending_ + fragment;
+    // A fragment must not BEGIN mid-character either. pending_ always starts on
+    // a lead byte, so this only bites when nothing was held and the caller's
+    // rules dropped the lead byte somewhere upstream; it is the backstop that
+    // makes "no delta this class returns is ever invalid UTF-8" true of the
+    // FRONT as well as the back, independently of those rules being right.
+    candidate.erase(0, utf8_orphan_prefix_length(candidate));
+    const std::size_t cut = utf8_complete_prefix_length(candidate);
+    pending_ = candidate.substr(cut);
+    std::string emitted = candidate.substr(0, cut);
+    assembled_ += emitted;
+    return emitted;
+}
+
+std::string TranscriptDeltaTracker::observe(const std::string &partial_text) {
+    if (partial_text.empty()) {
+        return {};
+    }
+    // The comparisons run against everything KNOWN, delivered plus held back,
+    // rather than against the delivered text alone. Comparing against the
+    // delivered text would treat the held-back byte as new on the very next
+    // report and emit it twice.
+    const std::string known = assembled_ + pending_;
+    // An EXACT repeat, and nothing looser. This absorbs the duplicate delivery
+    // voxtral_realtime produces, which is the ONLY thing rule 2 was ever needed
+    // for: process_available_stream_chunks hands each event it produces to the
+    // sink from inside its loop and RETURNS the last of the batch
+    // (session.cpp:385-386), so that last event arrives twice with byte-equal
+    // text both times. A duplicate IS an exact repeat, so equality covers it.
+    //
+    // It used to discard any report the known text merely STARTED WITH, and that
+    // cost far more than it bought. Two separate defects came out of it, and
+    // both were found by randomized traces rather than by reading:
+    //
+    //   1. A short INCREMENTAL fragment that happens to be a byte prefix of the
+    //      transcript so far was read as a repeat and dropped, losing text with
+    //      a 200 and no diagnostic. Both incremental families emit fragments
+    //      that small routinely: nemotron_asr cuts at a byte offset
+    //      (decoder.cpp:550) and vibevoice_asr at a common prefix
+    //      (session.cpp:89-105). 9.50% of randomized pure-ASCII traces and
+    //      29.12% of French ones ended with a corrupted transcript.
+    //   2. When such a fragment was the LEAD BYTE of a multi-byte character, its
+    //      continuation bytes then arrived alone and began the next delta, which
+    //      is invalid UTF-8, which the Go runtime refuses to unmarshal, which
+    //      ends the stream and the final_result with it.
+    //
+    // What the narrowing gives up is the shrinking-hypothesis case: a cumulative
+    // report SHORTER than what is known is now read as an incremental fragment
+    // and duplicates those bytes at the client. No pinned family produces one.
+    // voxtral is the only cumulative reporter, and its hypothesis is
+    // tokenizer_.decode(streaming_token_ids_) over a vector that is only ever
+    // push_back'ed (session.cpp:436) and cleared by reset() (session.cpp:257),
+    // so within a stream it can only grow. Measured: narrowing this changed not
+    // one byte of 30,000 randomized cumulative traces.
+    //
+    // KEPT DELIBERATELY THOUGH NO TEST CAN SEE IT. Once narrowed to equality
+    // this rule became redundant with rule 3 below: an equal partial has an
+    // empty suffix, so rule 3 would call release("") and emit nothing either
+    // way. Deleting it is therefore an equivalent mutation, and the mutation
+    // harness reports it as a survivor, which is the honest result and not a
+    // gap in the tests. It stays for two reasons: it states the duplicate
+    // absorption where the citation for it lives, and it is independent of rule
+    // 3's condition. A future tightening of rule 3 to, say, require a STRICTLY
+    // longer partial would otherwise send every duplicate down the incremental
+    // branch and put the whole transcript on the wire a second time.
+    if (partial_text == known) {
+        return {};
+    }
+    if (starts_with(partial_text, known)) {
+        // Cumulative: the report is the whole transcript so far.
+        return release(partial_text.substr(known.size()));
+    }
+    // Incremental: the fragment is new text to append.
+    //
+    // A family that REWRITES its hypothesis lands here too, and the client's
+    // view is then wrong in a way nothing downstream can fix. nemotron's
+    // decoder has such a branch (decoder.cpp:552-554): when the new text is not
+    // an extension of what it already emitted, it emits the whole new text. So
+    // "the cat sat" followed by "the cat sap" leaves the client holding
+    // "the cat satthe cat sap", and reconcile then correctly refuses to append
+    // to a contradicted assembly, which leaves concat(deltas) != final with no
+    // signal on the wire. This is NOT repaired here, and the reason is that a
+    // delta stream has no retraction: emitting only the differing suffix would
+    // read as "sap" appended to "the cat sat", which is a different wrong
+    // answer, and emitting a correction would need a wire field that does not
+    // exist. final_result carries the authoritative text either way. It did not
+    // fire in a 331 delta run, because an RNN-T decode is monotonic in practice.
+    return release(partial_text);
+}
+
+std::string TranscriptDeltaTracker::reconcile(const std::string &final_text) {
+    if (final_text.empty() || final_text == assembled_) {
+        // Nothing further is owed. Held-back bytes are dropped rather than
+        // flushed: they are not in the authoritative text, so sending them
+        // would contradict it.
+        pending_.clear();
+        return {};
+    }
+    if (!starts_with(final_text, assembled_)) {
+        // Contradicted. Nothing sent can be taken back, so nothing more is
+        // sent; final_result carries the authoritative text.
+        pending_.clear();
+        return {};
+    }
+    // Compared against the DELIVERED text, so the fragment below already
+    // contains whatever was held back. pending_ is therefore cleared rather
+    // than prepended, or those bytes would go out twice.
+    //
+    // The fragment ends on a character boundary whenever final_text is
+    // well-formed, which is the normal case and the reason a held-back sequence
+    // is always flushed here. It is still cut, so a family handing back a final
+    // text that is itself truncated mid-character cannot put a partial sequence
+    // on the wire through this path either.
+    pending_.clear();
+    return release(final_text.substr(assembled_.size()));
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stream_delta.h

@@ -0,0 +1,133 @@
+#pragma once
+
+// Turns whatever a streaming session calls a "partial transcript" into the
+// incremental deltas AudioTranscriptionStream is contracted to send. Standard
+// library only, so it is tested without an audio.cpp checkout.
+//
+// THIS UNIT EXISTS BECAUSE THE FAMILIES DISAGREE, and the disagreement is
+// invisible at the interface: StreamEvent::partial_text is a Transcript either
+// way. Read out of the pinned upstream, one family at a time:
+//
+//   nemotron_asr      INCREMENTAL. decoder.cpp emits
+//                     current_text.substr(emitted_text.size()) per non-blank
+//                     token, and only through the stream event SINK, during
+//                     finalize(). process_audio_chunk returns empty events.
+//   vibevoice_asr     INCREMENTAL. process_audio_chunk returns
+//                     text.substr(common_prefix_size(...)); the sink is
+//                     deliberately swapped out around its internal run_single,
+//                     so the fragment arrives once, on the return value.
+//   higgs_audio_stt   INCREMENTAL. Same shape as vibevoice_asr.
+//   voxtral_realtime  CUMULATIVE. partial_text is
+//                     tokenizer_.decode(streaming_token_ids_), the whole
+//                     hypothesis so far. process_available_stream_chunks hands
+//                     every event it produces to the sink from INSIDE its loop
+//                     (session.cpp:385-386) and RETURNS only the last of the
+//                     batch, so the last event of each batch arrives twice and
+//                     the others arrive once.
+//
+// Applying either convention to the other family corrupts the transcript: read
+// a cumulative report as a delta and the client sees the transcript repeated on
+// every event; read an incremental fragment as cumulative and the suffix
+// arithmetic eats the front of it. So the tracker decides per fragment, from
+// what it has already delivered, and the one rule it enforces is that TEXT THE
+// CLIENT HAS ALREADY BEEN SENT IS NEVER SENT AGAIN.
+//
+// The cumulative reading is provably safe for voxtral, which is the family it
+// matters for: its decode is a pure concatenation of per-token byte strings
+// (tokenizer_text.cpp:171-183), so decode(ids[0..n]) is an unconditional BYTE
+// PREFIX of decode(ids[0..n+1]) and one of its reports can never be mistaken
+// for an incremental fragment.
+//
+// UTF-8 IS THE OTHER HALF OF THAT SAME FACT. Because that decode concatenates
+// raw token BYTES, a multi-byte character is split across token boundaries, and
+// the difference between two consecutive cumulative reports is then a lone
+// continuation byte. TranscriptStreamResponse.delta is a proto3 `string`, whose
+// wire format REQUIRES valid UTF-8: the C++ runtime serializes an invalid one
+// with at most a warning, but the Go runtime refuses to unmarshal it, and the
+// client loses every remaining delta AND the final_result. So no fragment this
+// class returns ever BEGINS OR ENDS inside a character: an incomplete trailing
+// sequence is held back and merged into the next fragment, and a leading orphan
+// continuation byte, which nothing can ever complete, is dropped.
+//
+// It is NOT a voxtral-only concern, which is what the first attempt at this
+// assumed. The incremental families split characters by the same arithmetic:
+// nemotron_asr's decoder cuts at a BYTE offset (decoder.cpp:550) and
+// vibevoice_asr's common_prefix_size compares BYTES (session.cpp:80-86). Nor is
+// European text the worst case: a Japanese transcript, whose every character is
+// three bytes, carried at least one invalid delta in 33.74% of traces until
+// rule 2 below learned to leave an incomplete fragment alone.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+class TranscriptDeltaTracker {
+public:
+    // Takes one StreamEvent::partial_text and returns the fragment to put on
+    // the wire, empty when there is nothing new.
+    //
+    // The rules, in order, all of them against everything KNOWN (delivered
+    // plus held back), never against the delivered text alone:
+    //   1. An empty partial says nothing.
+    //   2. A partial IDENTICAL to the known text is a repeat: nothing is
+    //      emitted. Identical, not merely a prefix of it. That absorbs voxtral's
+    //      repeat of the last event in each batch, which is the only duplicate
+    //      any pinned family produces and which carries byte-equal text both
+    //      times. Discarding a mere PREFIX used to swallow an incremental
+    //      fragment that coincided with the start of the transcript, corrupting
+    //      the text silently and, when that fragment was a character's lead
+    //      byte, killing the stream outright; see the note at the rule in the
+    //      implementation.
+    //   3. A partial that EXTENDS the known text is a cumulative report: only
+    //      its new suffix is emitted.
+    //   4. Anything else is an incremental fragment: it is emitted whole and
+    //      appended.
+    //
+    // Rule 3 is the one judgement call, since a fragment that happens to begin
+    // with the entire transcript so far is indistinguishable from a cumulative
+    // report. It is read as cumulative because every cumulative family produces
+    // that shape on EVERY event, while an incremental family produces it only
+    // when one fragment repeats everything before it, which no tokenizer output
+    // does in practice.
+    //
+    // What comes back is the fragment MINUS any incomplete trailing UTF-8
+    // sequence, which is carried into the next call, and minus any leading
+    // orphan continuation byte, which is dropped. So an empty return can also
+    // mean "the only new bytes were half a character", and the caller needs no
+    // knowledge of that: writing nothing is exactly right.
+    std::string observe(const std::string &partial_text);
+
+    // Reconciles against TaskResult::text_output, which is authoritative, and
+    // returns the fragment that makes appending every delta equal it.
+    //
+    // This is what makes the OFFLINE FALLBACK a single line rather than its own
+    // branch: with no partials observed, the assembly is empty and the whole
+    // final text comes back as one delta.
+    //
+    // It is also what FLUSHES a held-back UTF-8 sequence, and it can always do
+    // so: the final text is complete, so the fragment from the last delivered
+    // byte to its end ends on a character boundary.
+    //
+    // A final text that CONTRADICTS what was already sent returns empty. A
+    // fragment on the wire cannot be retracted, so the alternative would be to
+    // send the transcript a second time and let the client hold it twice.
+    // final_result carries the authoritative text either way.
+    std::string reconcile(const std::string &final_text);
+
+    // Everything the client has been sent, concatenated. Held-back bytes are
+    // deliberately NOT included: this is what the client holds, not what the
+    // tracker knows.
+    const std::string &assembled() const noexcept { return assembled_; }
+
+private:
+    // Appends the emittable prefix of `fragment` to assembled_ and returns it,
+    // keeping any incomplete trailing UTF-8 sequence in pending_.
+    std::string release(const std::string &fragment);
+
+    std::string assembled_;
+    // An incomplete trailing UTF-8 sequence, computed but not sent. Always a
+    // proper prefix of one character, so at most three bytes.
+    std::string pending_;
+};
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/stream_delta_test.cpp

@@ -0,0 +1,486 @@
+// Unit tests for stream_delta. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+//
+// The traces below are transcribed from the pinned upstream sessions rather
+// than invented, because the whole reason this unit exists is that the four
+// streaming ASR families do NOT agree on what partial_text means.
+
+#include "stream_delta.cpp"
+
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+using audiocpp_backend::TranscriptDeltaTracker;
+
+// Feeds a whole trace and returns what a client appending every emitted
+// fragment would end up holding, which is the only property that matters.
+static std::string client_view(TranscriptDeltaTracker &tracker,
+                               const std::vector<std::string> &partials,
+                               std::vector<std::string> *emitted = nullptr) {
+    std::string view;
+    for (const auto &partial : partials) {
+        const std::string fragment = tracker.observe(partial);
+        if (emitted != nullptr && !fragment.empty()) {
+            emitted->push_back(fragment);
+        }
+        view += fragment;
+    }
+    return view;
+}
+
+// nemotron_asr: decoder.cpp emits current_text.substr(emitted_text.size()) on
+// every non-blank token, i.e. INCREMENTAL fragments, through the stream event
+// sink during finalize().
+static void test_incremental_family() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view =
+        client_view(tracker, {"Local", " AI", " now", " speaks."}, &emitted);
+    check_eq(view, "Local AI now speaks.", "incremental deltas concatenate");
+    check(emitted.size() == 4, "incremental family emits one fragment per partial");
+    check_eq(tracker.assembled(), "Local AI now speaks.",
+             "incremental family assembles the whole transcript");
+    check_eq(tracker.reconcile("Local AI now speaks."), "",
+             "a final result the deltas already cover adds nothing");
+}
+
+// voxtral_realtime: process_one_stream_chunk sets partial_text to
+// tokenizer_.decode(streaming_token_ids_), the WHOLE accumulated hypothesis,
+// and process_available_stream_chunks then hands the SAME event to both the
+// sink and the caller, so every partial arrives twice.
+static void test_cumulative_family_with_duplicate_delivery() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view = client_view(
+        tracker, {"Local", "Local", "Local AI", "Local AI", "Local AI now",
+                  "Local AI now"},
+        &emitted);
+    check_eq(view, "Local AI now", "cumulative partials are not repeated to the client");
+    check(emitted.size() == 3,
+          "the duplicate delivery of each cumulative event emits nothing twice");
+    check_eq(emitted.empty() ? "" : emitted[0], "Local", "first cumulative fragment");
+    check_eq(emitted.size() < 2 ? "" : emitted[1], " AI", "second cumulative fragment");
+    check_eq(emitted.size() < 3 ? "" : emitted[2], " now", "third cumulative fragment");
+}
+
+// The rule that separates the two: a report IDENTICAL to everything known is a
+// repeat and is never sent again.
+//
+// Only an identical one. Rule 2 used to discard any report the known text merely
+// STARTED WITH, and that cost more than it bought: see
+// test_a_short_fragment_is_not_mistaken_for_a_repeat.
+static void test_an_exact_repeat_is_never_resent() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("hello world"), "hello world", "first fragment");
+    check_eq(tracker.observe("hello world"), "", "an identical repeat emits nothing");
+    check_eq(tracker.observe("hello world"), "", "and a third delivery emits nothing");
+    check_eq(tracker.assembled(), "hello world", "the assembly is unchanged by repeats");
+}
+
+// THE TRADE, pinned so it is a decision rather than a surprise. A CUMULATIVE
+// report that SHRINKS is no longer absorbed: it is read as an incremental
+// fragment and duplicates a few bytes at the client.
+//
+// No pinned family produces one. voxtral_realtime is the only cumulative
+// reporter, and its hypothesis is tokenizer_.decode(streaming_token_ids_) over a
+// vector that is only ever push_back'ed (session.cpp:436) and cleared by reset()
+// (session.cpp:257), so within a stream it grows and never shrinks. The
+// duplicate delivery rule 2 really exists for is an EXACT repeat, which the test
+// above still covers.
+static void test_a_shrinking_hypothesis_is_read_as_incremental() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("hello world"), "hello world", "first fragment");
+    check_eq(tracker.observe("hello"), "hello",
+             "a shortened hypothesis is now read as an incremental fragment");
+    check_eq(tracker.assembled(), "hello worldhello",
+             "which duplicates those bytes at the client: the accepted cost");
+}
+
+static void test_empty_partials_are_ignored() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe(""), "", "an empty partial emits nothing");
+    check_eq(tracker.observe("a"), "a", "a real partial after an empty one still emits");
+    check_eq(tracker.observe(""), "", "a later empty partial emits nothing");
+    check_eq(tracker.assembled(), "a", "empty partials do not disturb the assembly");
+}
+
+// The offline fallback: a family with no streaming ASR runs once, so nothing is
+// ever observed and the reconciliation IS the single delta the RPC promises.
+static void test_offline_fallback_is_one_delta() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.reconcile("the whole transcript"), "the whole transcript",
+             "with no partials the final text is emitted whole");
+    check_eq(tracker.assembled(), "the whole transcript",
+             "the reconciliation is recorded as delivered");
+    check_eq(tracker.reconcile("the whole transcript"), "",
+             "reconciling twice does not duplicate");
+}
+
+// A streaming family whose partials stopped short of the final text: the tail
+// is emitted so that appending every delta still equals final_result.text.
+static void test_reconcile_emits_the_tail() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("Local AI"), "Local AI", "partial arrives");
+    check_eq(tracker.reconcile("Local AI now speaks."), " now speaks.",
+             "the untold tail of the final text is emitted");
+    check_eq(tracker.assembled(), "Local AI now speaks.", "tail is recorded");
+}
+
+// Divergence. nemotron's decoder has a rewrite branch: when the new hypothesis
+// is NOT an extension of what it already emitted, it emits the whole new text.
+// Nothing can retract a fragment already written to the wire, so the tracker
+// must not try: it emits nothing further and leaves final_result authoritative.
+static void test_divergent_final_text_is_not_appended() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("the cat"), "the cat", "first hypothesis");
+    check_eq(tracker.reconcile("the dog"), "",
+             "a final text that contradicts the deltas is not appended to them");
+    check_eq(tracker.assembled(), "the cat",
+             "a contradicted assembly is left as it was actually sent");
+}
+
+static void test_empty_final_text() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("something"), "something", "partial arrives");
+    check_eq(tracker.reconcile(""), "", "an empty final text emits nothing");
+    check_eq(tracker.assembled(), "something", "an empty final text changes nothing");
+}
+
+// A whitespace-only fragment is real text: the space between two words is
+// exactly what an incremental family delivers on its own.
+static void test_whitespace_fragments_survive() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("one"), "one", "word");
+    check_eq(tracker.observe(" "), " ", "a bare separator is emitted");
+    check_eq(tracker.observe("two"), "two", "next word");
+    check_eq(tracker.assembled(), "one two", "separator is kept in the assembly");
+}
+
+// The ambiguity this unit cannot resolve, pinned so that a future reader sees
+// the choice rather than rediscovering it: a fragment that EXTENDS everything
+// delivered so far is read as a cumulative report, because that is what every
+// cumulative family produces on every event, while an incremental family
+// producing one is the rare coincidence of a fragment repeating the whole
+// transcript so far.
+static void test_prefix_extension_is_read_as_cumulative() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("I"), "I", "first fragment");
+    check_eq(tracker.observe("I'm"), "'m",
+             "a fragment extending the assembly is treated as a cumulative report");
+    check_eq(tracker.assembled(), "I'm", "cumulative reading assembles once");
+}
+
+// --------------------------------------------------------------------------
+// UTF-8 boundaries
+//
+// TranscriptStreamResponse.delta is a proto3 `string`, and the wire format
+// REQUIRES a string field to be valid UTF-8. The C++ runtime serializes an
+// invalid one with at most a warning; the Go runtime refuses to unmarshal it,
+// so the client loses every delta AND the final_result still to come.
+//
+// Not hypothetical. voxtral_realtime reports the whole hypothesis as
+// tokenizer_.decode(streaming_token_ids_), and that decode is a pure
+// concatenation of raw token BYTES (tokenizer_text.cpp:171-183), so a
+// multi-byte character is split across token boundaries and the cumulative
+// difference between two consecutive reports is a lone continuation byte.
+// --------------------------------------------------------------------------
+
+// True when `text` is well-formed UTF-8. Written out here rather than reused
+// from the implementation on purpose: a test that shares the implementation's
+// idea of a boundary cannot catch the implementation's idea being wrong.
+static bool is_valid_utf8(const std::string &text) {
+    size_t i = 0;
+    while (i < text.size()) {
+        const auto lead = static_cast<unsigned char>(text[i]);
+        size_t length = 0;
+        if ((lead & 0x80) == 0x00) {
+            length = 1;
+        } else if ((lead & 0xE0) == 0xC0) {
+            length = 2;
+        } else if ((lead & 0xF0) == 0xE0) {
+            length = 3;
+        } else if ((lead & 0xF8) == 0xF0) {
+            length = 4;
+        } else {
+            return false;
+        }
+        if (i + length > text.size()) {
+            return false;
+        }
+        for (size_t k = 1; k < length; ++k) {
+            if ((static_cast<unsigned char>(text[i + k]) & 0xC0) != 0x80) {
+                return false;
+            }
+        }
+        i += length;
+    }
+    return true;
+}
+
+// Written as byte escapes so the test does not depend on the encoding of this
+// source file.
+static const std::string kEAcute = "\xC3\xA9";        // 2 bytes
+static const std::string kEuro = "\xE2\x82\xAC";      // 3 bytes
+static const std::string kEmoji = "\xF0\x9F\x8E\xA7"; // 4 bytes
+
+// A cumulative family advancing its hypothesis one BYTE at a time, which is
+// what voxtral_realtime does across a multi-byte character.
+static void test_cumulative_split_multibyte_character() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string full = "5" + kEuro;
+    std::vector<std::string> partials;
+    for (size_t n = 1; n <= full.size(); ++n) {
+        partials.push_back(full.substr(0, n));
+    }
+    const std::string view = client_view(tracker, partials, &emitted);
+
+    check_eq(view, full, "a byte-at-a-time cumulative report still assembles");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "cumulative fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check_eq(tracker.reconcile(full), "", "the final text adds nothing");
+}
+
+// An incremental family splitting a character across two fragments.
+static void test_incremental_split_multibyte_character() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view = client_view(
+        tracker, {"caf" + kEAcute.substr(0, 1), kEAcute.substr(1), " au lait"},
+        &emitted);
+
+    check_eq(view, "caf" + kEAcute + " au lait",
+             "an incremental split character still assembles");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "incremental fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check(emitted.size() == 3, "one fragment out per partial, none swallowed");
+    if (emitted.size() == 3) {
+        check_eq(emitted[0], "caf", "the lead byte of the character is held back");
+        check_eq(emitted[1], kEAcute,
+                 "the held byte is merged into the next fragment, not sent alone");
+        check_eq(emitted[2], " au lait", "the rest follows unchanged");
+    }
+}
+
+// A 4 byte character split three ways, so the held-back buffer has to survive
+// more than one round.
+static void test_four_byte_character_split_three_ways() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string view =
+        client_view(tracker,
+                    {"listen " + kEmoji.substr(0, 1), kEmoji.substr(1, 2),
+                     kEmoji.substr(3), " now"},
+                    &emitted);
+    check_eq(view, "listen " + kEmoji + " now",
+             "a 4 byte character survives three splits");
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "4 byte fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+}
+
+// The held-back bytes must reach the client. reconcile can always flush them,
+// because the final text is complete by construction.
+static void test_reconcile_flushes_a_held_back_sequence() {
+    TranscriptDeltaTracker tracker;
+    const std::string first = tracker.observe("done" + kEuro.substr(0, 2));
+    check_eq(first, "done", "the incomplete trailing sequence is held back");
+    check(is_valid_utf8(first), "what was emitted is valid UTF-8");
+    const std::string tail = tracker.reconcile("done" + kEuro);
+    check_eq(tail, kEuro, "reconcile flushes the completed character");
+    check(is_valid_utf8(tail), "the flushed tail is valid UTF-8");
+    check_eq(tracker.assembled(), "done" + kEuro, "the client holds the whole text");
+}
+
+// Held-back bytes are not lost track of: the next cumulative report emits the
+// whole character rather than only the bytes that just arrived.
+static void test_held_bytes_join_the_next_fragment() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("a" + kEuro.substr(0, 1)), "a",
+             "only the complete prefix goes out");
+    check_eq(tracker.observe("a" + kEuro), kEuro,
+             "the next cumulative report emits the whole character at once");
+    check_eq(tracker.assembled(), "a" + kEuro, "assembly is correct");
+}
+
+// A whole multi-byte character arriving at once must NOT be held back: holding
+// a complete sequence would stall every stream by one character.
+static void test_a_complete_character_is_not_held() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe("x" + kEuro), "x" + kEuro,
+             "a fragment ending on a boundary is emitted immediately");
+    check_eq(tracker.observe("x" + kEuro + kEmoji), kEmoji,
+             "and so is the next one");
+}
+
+// Bytes that can never complete must not be held forever: a stray continuation
+// byte or an invalid lead is passed through rather than stalling the stream.
+// Repairing a family's malformed output is not something a delta tracker can do
+// without altering the transcript.
+static void test_undecodable_bytes_are_not_held_forever() {
+    TranscriptDeltaTracker tracker;
+    check_eq(tracker.observe(std::string("ok\x80")), std::string("ok\x80"),
+             "a stray continuation byte is passed through, not held");
+    check_eq(tracker.observe(std::string("ok\x80") + "next"), "next",
+             "the stream continues");
+
+    // A lead byte the encoding does not define (0xF8 and above). Nothing can
+    // ever complete it, so holding it would stall the stream for good.
+    TranscriptDeltaTracker invalid_lead;
+    check_eq(invalid_lead.observe(std::string("ok\xFE")), std::string("ok\xFE"),
+             "an undefined lead byte is passed through, not held");
+    check_eq(invalid_lead.observe(std::string("ok\xFE") + "more"), "more",
+             "the stream continues past an undefined lead byte");
+
+    // The same byte followed by continuation bytes, which is the shape that
+    // looks most like a real sequence waiting to be completed.
+    TranscriptDeltaTracker invalid_run;
+    check_eq(invalid_run.observe(std::string("\xFE\x80\x80")), std::string("\xFE\x80\x80"),
+             "an undefined lead with continuations is passed through");
+
+    // Five continuation bytes with no lead in sight. They are DROPPED, not
+    // held: nothing can ever precede them, so holding would stall the stream
+    // for good, and emitting them would put invalid UTF-8 on the wire. See
+    // test_a_fragment_never_begins_mid_character.
+    TranscriptDeltaTracker orphans;
+    check_eq(orphans.observe(std::string("\x80\x80\x80\x80\x80")), "",
+             "a run of orphan continuation bytes is dropped, not emitted");
+    check_eq(orphans.observe("after"), "after",
+             "and the stream continues past them");
+}
+
+// THE SECOND HALF OF THE SAME BUG, and the one that survived fix round 1.
+//
+// Rule 2 discards a fragment the known text already starts with. When that
+// fragment is the LEAD BYTE of a NEW character it looks exactly like a repeat of
+// an earlier character beginning with the same byte, so it was discarded and
+// never held. Its continuation bytes then arrived on their own and began the
+// next delta, which is invalid UTF-8 at the FRONT, and utf8_complete_prefix_length
+// only ever inspected the TRAILING sequence.
+//
+// Reachable from shipping families, not synthetic: nemotron_asr's
+// decoder.cpp:550 cuts at a BYTE offset (current_text.substr(emitted_text.size()))
+// and vibevoice_asr's common_prefix_size (session.cpp:80-86) compares BYTES, so
+// both split characters mid-sequence. The trace below is exactly how they split
+// "ssee" spelled with the German sharp s, an e-acute, a euro sign and an o-grave,
+// three of which begin with the same 0xC3 lead byte.
+static void test_a_repeated_lead_byte_is_not_swallowed() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string sharp_s = "\xC3\x9F";  // U+00DF
+    const std::string e_acute = "\xC3\xA9";  // U+00E9
+    const std::string euro = "\xE2\x82\xAC"; // U+20AC
+    const std::string o_grave = "\xC3\xB2";  // U+00F2
+    const std::string full = sharp_s + e_acute + euro + o_grave;
+
+    const std::string view = client_view(tracker,
+                                         {sharp_s.substr(0, 1), sharp_s.substr(1),
+                                          e_acute.substr(0, 1),
+                                          e_acute.substr(1) + euro,
+                                          o_grave.substr(0, 1), o_grave.substr(1)},
+                                         &emitted);
+
+    for (size_t i = 0; i < emitted.size(); ++i) {
+        check(is_valid_utf8(emitted[i]),
+              "repeated-lead fragment " + std::to_string(i) + " is valid UTF-8");
+    }
+    check_eq(view, full, "no character is lost to a repeated lead byte");
+    check_eq(tracker.reconcile(full), "", "the final text adds nothing");
+}
+
+// The same shape one layer down, as a backstop: a fragment that BEGINS with
+// orphan continuation bytes must never go on the wire, whatever produced it.
+// Dropping bytes keeps the stream alive; emitting them ends it, and takes the
+// final_result that was still to come with it.
+static void test_a_fragment_never_begins_mid_character() {
+    TranscriptDeltaTracker tracker;
+    const std::string first = tracker.observe(std::string("\xA9") + "rest");
+    check(is_valid_utf8(first), "a leading orphan continuation byte is not emitted");
+    check_eq(first, "rest", "the rest of the fragment still goes out");
+
+    TranscriptDeltaTracker all_orphans;
+    check_eq(all_orphans.observe(std::string("\x82\xAC")), "",
+             "a fragment that is nothing but orphans emits nothing");
+    check_eq(all_orphans.assembled(), "",
+             "and nothing is recorded as delivered");
+    check_eq(all_orphans.observe("after"), "after", "the stream continues");
+}
+
+// PURE ASCII, no multi-byte character anywhere, and the transcript still comes
+// out wrong: a short incremental fragment that happens to be a byte prefix of
+// everything known was read as an already-delivered repeat and discarded.
+//
+// This is the shape both incremental families produce. nemotron_asr emits
+// current_text.substr(emitted_text.size()) per non-blank token (decoder.cpp:550)
+// and vibevoice_asr emits text.substr(common_prefix_size(...)) (session.cpp:89-105),
+// so a one-character fragment is ordinary output, and any of the transcript's
+// own leading characters will eventually arrive as one.
+//
+// Measured over 5,000 randomized traces per transcript before this was fixed:
+// 9.50% of pure-ASCII traces and 29.12% of French ones ended with the client
+// holding something other than final_result.text, with a 200 and no diagnostic.
+static void test_a_short_fragment_is_not_mistaken_for_a_repeat() {
+    TranscriptDeltaTracker tracker;
+    std::vector<std::string> emitted;
+    const std::string full = "pure ascii transcript";
+    const std::string view = client_view(
+        tracker, {"pure ", "ascii ", "trans", "c", "ri", "p", "t"}, &emitted);
+
+    check_eq(view, full, "the whole transcript reaches the client");
+    check_eq(tracker.assembled(), full, "and the assembly agrees with it");
+    check_eq(tracker.reconcile(full), "",
+             "the final text adds nothing, because nothing was lost");
+    check(emitted.size() == 7, "every fragment produced exactly one delta");
+}
+
+int main() {
+    test_incremental_family();
+    test_cumulative_family_with_duplicate_delivery();
+    test_an_exact_repeat_is_never_resent();
+    test_a_shrinking_hypothesis_is_read_as_incremental();
+    test_a_short_fragment_is_not_mistaken_for_a_repeat();
+    test_empty_partials_are_ignored();
+    test_offline_fallback_is_one_delta();
+    test_reconcile_emits_the_tail();
+    test_divergent_final_text_is_not_appended();
+    test_empty_final_text();
+    test_whitespace_fragments_survive();
+    test_prefix_extension_is_read_as_cumulative();
+    test_cumulative_split_multibyte_character();
+    test_incremental_split_multibyte_character();
+    test_four_byte_character_split_three_ways();
+    test_reconcile_flushes_a_held_back_sequence();
+    test_held_bytes_join_the_next_fragment();
+    test_a_complete_character_is_not_held();
+    test_undecodable_bytes_are_not_held_forever();
+    test_a_repeated_lead_byte_is_not_swallowed();
+    test_a_fragment_never_begins_mid_character();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all stream_delta checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/streaming_driver_ctest.cpp

@@ -0,0 +1,976 @@
+// Tests for the streaming drivers in loaded_model: begin_stream,
+// run_streaming_pull, run_streaming_audio and run_streaming_live, plus
+// resolve_model_path, which lives in the same engine-linked unit.
+//
+// Engine-linked, so this runs through ctest rather than
+// backend/cpp/run-unit-tests.sh. It builds no model and loads no file: a
+// LoadedModel::Session is a plain struct holding a pointer to an engine
+// interface, so a fake session exercises the drivers directly, which is the
+// only way to assert the STATE OBLIGATION (prepare, then start_stream, on every
+// stream) without a GPU and a gigabyte of weights.
+
+#include "inference_lane.h"
+#include "loaded_model.h"
+
+#include "engine/framework/runtime/session.h"
+
+#include <cstddef>
+#include <cstdio>
+#include <functional>
+#include <memory>
+#include <optional>
+#include <stdexcept>
+#include <string>
+#include <vector>
+
+namespace rt = engine::runtime;
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+static void check_eq(const std::string &got, const std::string &want,
+                     const std::string &name) {
+    check(got == want, name + " (got \"" + got + "\" want \"" + want + "\")");
+}
+
+static std::string join(const std::vector<std::string> &parts) {
+    std::string out;
+    for (const auto &part : parts) {
+        if (!out.empty()) {
+            out += "|";
+        }
+        out += part;
+    }
+    return out;
+}
+
+// --------------------------------------------------------------------------
+// Fakes
+// --------------------------------------------------------------------------
+
+// Consumes audio chunks, like every streaming ASR family.
+//
+// It deliberately does NOT override start_stream, so every test that drives it
+// also pins the claim the drivers rely on: IStreamingVoiceTaskSession's BASE
+// start_stream is a call to reset(). If upstream ever changes that base, the
+// replay test below fails rather than the backend silently continuing the
+// previous stream.
+class FakeAudioSession : public rt::IStreamingVoiceTaskSession {
+public:
+    std::vector<std::string> calls;
+    std::vector<rt::AudioChunk> chunks;
+    rt::StreamingPolicy policy;
+    // Set to make process_audio_chunk throw on the nth call (1-based).
+    int throw_on_chunk = 0;
+    // Cumulative partial text, which is voxtral_realtime's convention.
+    bool report_partials = true;
+
+    std::string family() const override { return "fake_audio"; }
+    rt::VoiceTaskKind task_kind() const override { return rt::VoiceTaskKind::Asr; }
+    rt::RunMode run_mode() const override { return rt::RunMode::Streaming; }
+
+    void prepare(const rt::SessionPreparationRequest &request) override {
+        calls.push_back("prepare");
+        prepared_ = true;
+        prepared_rate_ = request.audio.has_value() ? request.audio->sample_rate : 0;
+    }
+
+    rt::StreamingPolicy streaming_policy() const override { return policy; }
+
+    void set_stream_event_sink(rt::StreamEventCallback sink) override {
+        calls.push_back(sink ? "sink+" : "sink-");
+        sink_ = std::move(sink);
+    }
+
+    void reset() override {
+        if (!prepared_) {
+            // Exactly what silero_vad does, and the reason prepare() has to come
+            // first rather than being folded into session_for.
+            throw std::runtime_error("fake: prepare() must be called before reset()");
+        }
+        calls.push_back("reset");
+        seen_frames_ = 0;
+        seen_chunks_ = 0;
+        text_.clear();
+    }
+
+    rt::StreamEvent process_audio_chunk(const rt::AudioChunk &chunk) override {
+        calls.push_back("chunk");
+        chunks.push_back(chunk);
+        ++seen_chunks_;
+        if (throw_on_chunk == seen_chunks_) {
+            throw std::runtime_error("fake: chunk failure");
+        }
+        const int channels = chunk.channels > 0 ? chunk.channels : 1;
+        seen_frames_ += static_cast<std::int64_t>(chunk.samples.size()) / channels;
+        text_ += "w" + std::to_string(seen_chunks_);
+        rt::StreamEvent event;
+        if (report_partials) {
+            event.partial_text = rt::Transcript{text_, "en"};
+        }
+        return event;
+    }
+
+    rt::TaskResult finalize() override {
+        calls.push_back("finalize");
+        rt::TaskResult result;
+        result.text_output = rt::Transcript{
+            text_ + "/frames=" + std::to_string(seen_frames_), "en"};
+        return result;
+    }
+
+    // Emits through the SINK the way nemotron_asr does, from inside the final
+    // step rather than from process_audio_chunk.
+    void emit_through_sink(const std::string &fragment) {
+        if (!sink_) {
+            return;
+        }
+        rt::StreamEvent event;
+        event.partial_text = rt::Transcript{fragment, "en"};
+        sink_(event);
+    }
+
+    bool sink_installed() const { return static_cast<bool>(sink_); }
+    int prepared_rate() const { return prepared_rate_; }
+
+private:
+    rt::StreamEventCallback sink_;
+    bool prepared_ = false;
+    int prepared_rate_ = 0;
+    std::int64_t seen_frames_ = 0;
+    int seen_chunks_ = 0;
+    std::string text_;
+};
+
+// nemotron_asr's shape: partials arrive only through the sink, and only from
+// inside the finalize step.
+class SinkOnlyAudioSession : public FakeAudioSession {
+public:
+    SinkOnlyAudioSession() { report_partials = false; }
+
+    rt::TaskResult finalize() override {
+        emit_through_sink("late ");
+        emit_through_sink("partial");
+        return FakeAudioSession::finalize();
+    }
+};
+
+// Pulls events, like every streaming TTS family. Overrides start_stream the way
+// the seven real families do, calling reset() first.
+class FakePullSession : public rt::IStreamingVoiceTaskSession {
+public:
+    std::vector<std::string> calls;
+    std::size_t event_count = 3;
+    bool final_on_second = false;
+
+    std::string family() const override { return "fake_pull"; }
+    rt::VoiceTaskKind task_kind() const override { return rt::VoiceTaskKind::Tts; }
+    rt::RunMode run_mode() const override { return rt::RunMode::Streaming; }
+
+    void prepare(const rt::SessionPreparationRequest &) override {
+        calls.push_back("prepare");
+        prepared_ = true;
+    }
+
+    rt::StreamingPolicy streaming_policy() const override {
+        rt::StreamingPolicy policy;
+        policy.input = rt::StreamingInputKind::None;
+        policy.output = rt::StreamingOutputKind::PullEvents;
+        return policy;
+    }
+
+    void start_stream(const rt::TaskRequest &request) override {
+        calls.push_back("start_stream");
+        (void)request;
+        reset();
+    }
+
+    void set_stream_event_sink(rt::StreamEventCallback sink) override {
+        calls.push_back(sink ? "sink+" : "sink-");
+        sink_ = std::move(sink);
+    }
+
+    void reset() override {
+        if (!prepared_) {
+            throw std::runtime_error("fake: prepare() must be called before reset()");
+        }
+        calls.push_back("reset");
+        emitted_ = 0;
+    }
+
+    std::optional<rt::StreamEvent> next_stream_event() override {
+        if (emitted_ >= event_count) {
+            return std::nullopt;
+        }
+        rt::StreamEvent event;
+        rt::AudioBuffer audio;
+        audio.sample_rate = 24000;
+        audio.channels = 1;
+        audio.samples.assign(4, 0.25F);
+        // named_audio_outputs, NOT audio_output: this is where supertonic,
+        // omnivoice and voxcpm2 all put their streamed chunks.
+        event.named_audio_outputs.push_back(
+            {"chunk_" + std::to_string(emitted_), std::move(audio), {}});
+        ++emitted_;
+        if (final_on_second && emitted_ == 2) {
+            event.is_final = true;
+        }
+        calls.push_back("pull");
+        return event;
+    }
+
+    rt::StreamEvent process_audio_chunk(const rt::AudioChunk &) override {
+        throw std::runtime_error("fake_pull consumes no audio");
+    }
+
+    rt::TaskResult finalize() override {
+        calls.push_back("finalize");
+        rt::TaskResult result;
+        rt::AudioBuffer merged;
+        merged.sample_rate = 24000;
+        merged.channels = 1;
+        merged.samples.assign(4 * emitted_, 0.25F);
+        result.audio_output = std::move(merged);
+        return result;
+    }
+
+    bool sink_installed() const { return static_cast<bool>(sink_); }
+
+private:
+    rt::StreamEventCallback sink_;
+    bool prepared_ = false;
+    std::size_t emitted_ = 0;
+};
+
+// --------------------------------------------------------------------------
+// Helpers
+// --------------------------------------------------------------------------
+
+static audiocpp_backend::LoadedModel::Session
+streaming_session(rt::IStreamingVoiceTaskSession &fake,
+                  audiocpp_backend::Task task) {
+    audiocpp_backend::LoadedModel::Session session;
+    session.task = task;
+    session.mode = audiocpp_backend::Mode::Streaming;
+    session.streaming = &fake;
+    return session;
+}
+
+static rt::TaskRequest audio_request(int sample_rate, int channels,
+                                     std::int64_t frames) {
+    rt::TaskRequest request;
+    rt::AudioBuffer audio;
+    audio.sample_rate = sample_rate;
+    audio.channels = channels;
+    audio.samples.assign(static_cast<std::size_t>(frames * channels), 0.5F);
+    request.audio_input = std::move(audio);
+    return request;
+}
+
+// --------------------------------------------------------------------------
+// Tests
+// --------------------------------------------------------------------------
+
+static void test_begin_stream_prepares_then_starts() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakePullSession fake;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Tts);
+
+    rt::TaskRequest request;
+    audiocpp_backend::begin_stream(session, request, entry);
+
+    check_eq(join(fake.calls), "prepare|start_stream|reset",
+             "begin_stream prepares before it starts, and start_stream resets");
+}
+
+// The base implementation of start_stream IS a reset(). FakeAudioSession does
+// not override start_stream, so this is that guarantee, read out of the pinned
+// header rather than assumed.
+static void test_base_start_stream_resets() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    audiocpp_backend::begin_stream(session, audio_request(16000, 1, 10), entry);
+    check_eq(join(fake.calls), "prepare|reset",
+             "the interface's own start_stream resets the session");
+}
+
+static void test_begin_stream_refuses_a_non_streaming_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    audiocpp_backend::LoadedModel::Session session;
+    session.mode = audiocpp_backend::Mode::Offline;
+
+    bool threw_capability = false;
+    try {
+        rt::TaskRequest request;
+        audiocpp_backend::begin_stream(session, request, entry);
+    } catch (const audiocpp_backend::CapabilityError &) {
+        threw_capability = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_capability,
+          "begin_stream on an offline session throws CapabilityError, not a null deref");
+}
+
+// THE ONE THIS TASK IS ABOUT. A streaming session is cached, so the second
+// stream gets the object the first one left behind. Two identical runs against
+// the SAME session must produce identical output.
+static void test_a_refetched_session_replays_identically() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1536);
+
+    std::vector<std::string> first_fragments;
+    const auto first = audiocpp_backend::run_streaming_audio(
+        session, request, *request.audio_input,
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                first_fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    std::vector<std::string> second_fragments;
+    const auto second = audiocpp_backend::run_streaming_audio(
+        session, request, *request.audio_input,
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                second_fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    check_eq(join(second_fragments), join(first_fragments),
+             "a re-fetched streaming session replays the same partials");
+    check_eq(second.text_output.has_value() ? second.text_output->text : "",
+             first.text_output.has_value() ? first.text_output->text : "",
+             "a re-fetched streaming session replays the same final text");
+    check_eq(first.text_output.has_value() ? first.text_output->text : "",
+             "w1w2w3/frames=1536",
+             "the first run saw exactly the audio it was given");
+    // Not a tautology: without the reset the second run reports six words and
+    // 3072 frames, and both checks above fail.
+    check_eq(join(first_fragments), "w1|w1w2|w1w2w3", "cumulative partials");
+}
+
+static void test_run_streaming_audio_installs_and_clears_the_sink() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    SinkOnlyAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 1024;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1024);
+
+    std::vector<std::string> fragments;
+    const auto result = audiocpp_backend::run_streaming_audio(
+        session, request, *request.audio_input,
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    check_eq(join(fragments), "late |partial",
+             "a family that reports only through the sink is not silent");
+    check(!fake.sink_installed(),
+          "the sink is cleared before returning, so the cached session holds no "
+          "reference to the caller's frame");
+    check_eq(join(fake.calls), "sink+|prepare|reset|chunk|finalize|sink-",
+             "the sink is installed before the stream begins and cleared after it ends");
+    check(result.text_output.has_value(), "the final result still comes back");
+}
+
+static void test_the_sink_is_cleared_when_the_stream_throws() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    fake.throw_on_chunk = 1;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1024);
+
+    bool threw = false;
+    try {
+        audiocpp_backend::run_streaming_audio(
+            session, request, *request.audio_input,
+            [](const rt::StreamEvent &) {}, entry);
+    } catch (const std::exception &) {
+        threw = true;
+    }
+    check(threw, "a failing chunk propagates");
+    check(!fake.sink_installed(),
+          "the sink is cleared on the exception path too");
+}
+
+static void test_chunking_honours_the_policy_sample_count() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 16000;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    // 2.5 chunks, so the last one is short.
+    const auto request = audio_request(16000, 1, 40000);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 3, "40000 frames at 16000 per chunk is three chunks");
+    if (fake.chunks.size() == 3) {
+        check(fake.chunks[0].samples.size() == 16000, "first chunk is full");
+        check(fake.chunks[1].samples.size() == 16000, "second chunk is full");
+        check(fake.chunks[2].samples.size() == 8000, "last chunk is the remainder");
+        check(fake.chunks[0].start_sample == 0, "first chunk starts at zero");
+        check(fake.chunks[1].start_sample == 16000, "second chunk start index");
+        check(fake.chunks[2].start_sample == 32000, "third chunk start index");
+        check(fake.chunks[0].sample_rate == 16000, "chunk carries the buffer's rate");
+        check(fake.chunks[0].channels == 1, "chunk carries the buffer's channel count");
+    }
+}
+
+// A buffer whose float count is not a whole number of frames is REFUSED rather
+// than truncated. The integer division would otherwise drop the tail floats
+// from the fed audio, and therefore from the transcript, with no diagnostic.
+static void test_a_partial_trailing_frame_is_refused() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 100;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    // 501 floats across 2 channels: 250 whole frames and one stray float.
+    rt::TaskRequest request;
+    rt::AudioBuffer audio;
+    audio.sample_rate = 48000;
+    audio.channels = 2;
+    audio.samples.assign(501, 0.5F);
+    request.audio_input = std::move(audio);
+
+    bool threw_config = false;
+    try {
+        audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                              [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::ConfigError &) {
+        threw_config = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config,
+          "a buffer that is not a whole number of frames is refused with ConfigError");
+    check(fake.calls.empty(),
+          "the refusal precedes every call into the session, so no half-started "
+          "stream is left on the cached one");
+}
+
+// higgs_audio_stt states its window in seconds and leaves the sample count at
+// zero, so this branch is a real family's path rather than a defensive one.
+static void test_chunking_falls_back_to_the_policy_seconds() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 0;
+    fake.policy.preferred_audio_chunk_seconds = 4.0;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 96000); // 6 s
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 2, "6 s at a 4 s window is two chunks");
+    if (fake.chunks.size() == 2) {
+        check(fake.chunks[0].samples.size() == 64000, "first window is 4 s");
+        check(fake.chunks[1].samples.size() == 32000, "second window is the 2 s remainder");
+    }
+}
+
+static void test_chunking_falls_back_to_the_interface_default() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 0;
+    fake.policy.preferred_audio_chunk_seconds = 0.0;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(16000, 1, 1024);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 2, "a policy naming no window uses the interface's 512");
+    if (!fake.chunks.empty()) {
+        check(fake.chunks[0].samples.size() == 512, "default window is 512 frames");
+    }
+}
+
+// A zero sample rate must not turn a seconds-only policy into a zero-length
+// chunk, which would loop forever.
+static void test_a_seconds_policy_with_no_rate_falls_through() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 0;
+    fake.policy.preferred_audio_chunk_seconds = 4.0;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(0, 1, 1024);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+    check(fake.chunks.size() == 2, "a rateless buffer still chunks at the default 512");
+}
+
+// FRAMES, not floats. vibevoice_asr refuses a chunk whose sample count is not
+// divisible by its channel count, and offsets every span it reports by the
+// chunk's start_sample.
+static void test_stereo_chunks_are_frame_aligned() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 300;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+    const auto request = audio_request(48000, 2, 750);
+
+    audiocpp_backend::run_streaming_audio(session, request, *request.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 3, "750 frames at 300 frames per chunk is three chunks");
+    for (const auto &chunk : fake.chunks) {
+        check(chunk.samples.size() % 2 == 0, "every stereo chunk is a whole number of frames");
+    }
+    if (fake.chunks.size() == 3) {
+        check(fake.chunks[0].samples.size() == 600, "300 stereo frames is 600 floats");
+        check(fake.chunks[1].start_sample == 300,
+              "start_sample counts frames, not floats");
+        check(fake.chunks[2].samples.size() == 300, "the remainder is 150 frames");
+    }
+}
+
+static void test_pull_drains_every_event_and_installs_no_sink() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakePullSession fake;
+    fake.event_count = 3;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Tts);
+
+    std::vector<std::string> ids;
+    rt::TaskRequest request;
+    const auto result = audiocpp_backend::run_streaming_pull(
+        session, request,
+        [&](const rt::StreamEvent &event) {
+            for (const auto &named : event.named_audio_outputs) {
+                ids.push_back(named.id);
+            }
+        },
+        entry);
+
+    check_eq(join(ids), "chunk_0|chunk_1|chunk_2", "every pulled event reaches the caller");
+    check(!fake.sink_installed(),
+          "no stream event sink is installed on the pull path, so voxcpm2 cannot "
+          "deliver every chunk twice");
+    check_eq(join(fake.calls), "prepare|start_stream|reset|pull|pull|pull|finalize",
+             "prepare, start, drain, finish");
+    check(result.audio_output.has_value(), "the merged result comes back");
+    check(result.audio_output.has_value() && result.audio_output->samples.size() == 12,
+          "the merged result is the whole synthesis, not a tail");
+}
+
+static void test_pull_stops_on_a_final_event() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakePullSession fake;
+    fake.event_count = 5;
+    fake.final_on_second = true;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Tts);
+
+    int events = 0;
+    rt::TaskRequest request;
+    audiocpp_backend::run_streaming_pull(
+        session, request, [&](const rt::StreamEvent &) { ++events; }, entry);
+
+    check(events == 2, "an event marked final ends the pull loop");
+}
+
+static void test_pull_refuses_a_non_streaming_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    audiocpp_backend::LoadedModel::Session session;
+
+    bool threw_capability = false;
+    try {
+        rt::TaskRequest request;
+        audiocpp_backend::run_streaming_pull(
+            session, request, [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::CapabilityError &) {
+        threw_capability = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_capability, "run_streaming_pull refuses a session with no streaming half");
+}
+
+// prepare() runs on EVERY stream, not once per session: the preparation request
+// is derived from the request (audio contract, text, voice), so a second stream
+// at a different rate would otherwise run against the first one's contract.
+static void test_prepare_tracks_the_request_not_the_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 4096;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    const auto first = audio_request(16000, 1, 4096);
+    audiocpp_backend::run_streaming_audio(session, first, *first.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+    check(fake.prepared_rate() == 16000, "the first stream prepares at its own rate");
+
+    const auto second = audio_request(44100, 1, 4096);
+    audiocpp_backend::run_streaming_audio(session, second, *second.audio_input,
+                                          [](const rt::StreamEvent &) {}, entry);
+    check(fake.prepared_rate() == 44100,
+          "the second stream prepares at ITS rate, not the first one's");
+}
+
+// --------------------------------------------------------------------------
+// run_streaming_live
+// --------------------------------------------------------------------------
+
+// Hands the driver a fixed list of wire frames, the way a client's audio
+// callback would, and then closes.
+static std::function<bool(std::vector<float> &)>
+frames_from(const std::vector<std::size_t> &sizes) {
+    auto index = std::make_shared<std::size_t>(0);
+    auto list = std::make_shared<std::vector<std::size_t>>(sizes);
+    return [index, list](std::vector<float> &out) {
+        if (*index >= list->size()) {
+            return false;
+        }
+        out.assign((*list)[*index], 0.5F);
+        ++*index;
+        return true;
+    };
+}
+
+static rt::TaskRequest live_request(int sample_rate, int channels) {
+    rt::TaskRequest request;
+    rt::AudioBuffer contract;
+    contract.sample_rate = sample_rate;
+    contract.channels = channels;
+    request.audio_input = std::move(contract); // no samples: none exist yet
+    return request;
+}
+
+// The wire's frame size is a property of the client's audio callback. The
+// family's window is a statement about what it can decode. The driver feeds the
+// second, not the first.
+static void test_live_buffers_wire_frames_into_policy_windows() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 1600;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    // Ten 512-sample frames: 5120 samples, i.e. three full 1600 windows and a
+    // 320 sample tail.
+    const auto result = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1),
+        frames_from(std::vector<std::size_t>(10, 512)),
+        [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 4,
+          "5120 wire samples at a 1600 frame window is three windows and a tail");
+    if (fake.chunks.size() == 4) {
+        check(fake.chunks[0].samples.size() == 1600, "first window is full");
+        check(fake.chunks[2].samples.size() == 1600, "third window is full");
+        check(fake.chunks[3].samples.size() == 320, "the tail is what was left");
+        check(fake.chunks[0].start_sample == 0, "the first window starts at zero");
+        check(fake.chunks[1].start_sample == 1600, "start_sample counts frames");
+        check(fake.chunks[3].start_sample == 4800, "the tail is offset by all of it");
+        check(fake.chunks[0].sample_rate == 16000, "the chunk carries the session rate");
+    }
+    check(result.text_output.has_value() &&
+              result.text_output->text == "w1w2w3w4/frames=5120",
+          "every wire sample reaches the family exactly once");
+}
+
+// nemotron_asr's shape: no partials from process_audio_chunk, every one of them
+// through the sink from inside finalize.
+static void test_live_installs_and_clears_the_sink() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    SinkOnlyAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    std::vector<std::string> fragments;
+    audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({512}),
+        [&](const rt::StreamEvent &event) {
+            if (event.partial_text.has_value()) {
+                fragments.push_back(event.partial_text->text);
+            }
+        },
+        entry);
+
+    check_eq(join(fragments), "late |partial",
+             "a family that reports only through the sink is not silent live either");
+    check(!fake.sink_installed(),
+          "the sink is cleared before returning, so the cached session holds no "
+          "reference to this call's frame");
+    check_eq(join(fake.calls), "sink+|prepare|reset|chunk|finalize|sink-",
+             "sink installed before the stream begins, cleared after it ends");
+}
+
+// A client that opens a session and closes it without speaking. finalize is NOT
+// called: nemotron_asr throws "finalize requires streamed audio", and an empty
+// transcript is the truthful answer to transcribing nothing.
+static void test_live_with_no_audio_never_finalizes() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    const auto result = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({}),
+        [](const rt::StreamEvent &) {}, entry);
+
+    check_eq(join(fake.calls), "sink+|prepare|reset|sink-",
+             "an empty live stream begins and ends without a chunk or a finalize");
+    check(!result.text_output.has_value(),
+          "an empty live stream reports no transcript rather than an error");
+}
+
+// A tail shorter than a window is still fed. Without this the last fragment of
+// speech never reaches the model, and nothing says so.
+static void test_live_feeds_a_short_tail() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 16000;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    audiocpp_backend::run_streaming_live(session, live_request(16000, 1),
+                                         frames_from({100, 200}),
+                                         [](const rt::StreamEvent &) {}, entry);
+
+    check(fake.chunks.size() == 1,
+          "300 samples against a 16000 frame window is one short chunk, not none");
+    if (!fake.chunks.empty()) {
+        check(fake.chunks[0].samples.size() == 300, "the tail carries everything fed");
+    }
+}
+
+// A live request carries no samples, so the CONTRACT is the only thing that says
+// what rate the frames are in, and prepare() needs it: nemotron_asr's streaming
+// prepare throws without one.
+static void test_live_prepares_at_the_contract_rate() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    audiocpp_backend::run_streaming_live(session, live_request(16000, 1),
+                                         frames_from({512}),
+                                         [](const rt::StreamEvent &) {}, entry);
+    check(fake.prepared_rate() == 16000,
+          "the empty contract buffer still carries the rate into prepare()");
+
+    bool threw_config = false;
+    try {
+        rt::TaskRequest bare; // no audio_input at all
+        audiocpp_backend::run_streaming_live(session, bare, frames_from({512}),
+                                             [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::ConfigError &) {
+        threw_config = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_config, "a live request with no audio contract is refused");
+}
+
+// The pull function is the gRPC read, and a request the handler has to refuse
+// mid-stream unwinds through the driver. The sink must not survive it.
+static void test_live_clears_the_sink_when_the_puller_throws() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    bool threw = false;
+    try {
+        audiocpp_backend::run_streaming_live(
+            session, live_request(16000, 1),
+            [](std::vector<float> &) -> bool {
+                throw std::runtime_error("fake: the client vanished");
+            },
+            [](const rt::StreamEvent &) {}, entry);
+    } catch (const std::exception &) {
+        threw = true;
+    }
+    check(threw, "a failing pull propagates");
+    check(!fake.sink_installed(), "the sink is cleared on the pull's exception path");
+}
+
+// Two live streams over the SAME cached session must not run into each other.
+static void test_live_replays_identically_on_a_refetched_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    FakeAudioSession fake;
+    fake.policy.preferred_audio_chunk_samples = 512;
+    const auto session = streaming_session(fake, audiocpp_backend::Task::Asr);
+
+    const auto first = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({512, 512}),
+        [](const rt::StreamEvent &) {}, entry);
+    const auto second = audiocpp_backend::run_streaming_live(
+        session, live_request(16000, 1), frames_from({512, 512}),
+        [](const rt::StreamEvent &) {}, entry);
+
+    check_eq(second.text_output.has_value() ? second.text_output->text : "",
+             first.text_output.has_value() ? first.text_output->text : "",
+             "a re-fetched live session replays the same transcript");
+    // Not a tautology: without the reset the second run reports w1..w4 and 2048
+    // frames.
+    check_eq(first.text_output.has_value() ? first.text_output->text : "",
+             "w1w2/frames=1024", "the first live run saw exactly what was fed");
+}
+
+static void test_live_refuses_a_non_streaming_session() {
+    audiocpp_backend::InferenceLane lane("test");
+    audiocpp_backend::LaneEntry entry(lane, 0);
+    audiocpp_backend::LoadedModel::Session session;
+    session.mode = audiocpp_backend::Mode::Offline;
+
+    bool threw_capability = false;
+    try {
+        audiocpp_backend::run_streaming_live(session, live_request(16000, 1),
+                                             frames_from({512}),
+                                             [](const rt::StreamEvent &) {}, entry);
+    } catch (const audiocpp_backend::CapabilityError &) {
+        threw_capability = true;
+    } catch (const std::exception &) {
+    }
+    check(threw_capability,
+          "run_streaming_live refuses a session with no streaming half");
+}
+
+// --------------------------------------------------------------------------
+// resolve_model_path
+// --------------------------------------------------------------------------
+//
+// It lives in loaded_model.cpp and is a pure (dir, file, name) -> string, so it
+// is tested here rather than in a standalone unit: that file cannot compile
+// without the engine headers.
+//
+// It is tested at all because it shipped a bug no test could have caught. THE
+// SHAPES BELOW ARE THE PRODUCTION SHAPES, not convenient ones, and that
+// distinction is the entire point. Task 15 verified the bundled: form with a
+// hand-written LoadModel that left ModelFile empty, which is the one shape the
+// server never produces: pkg/model/loader.go's LoadModelWithFile always fills
+// ModelFile with filepath.Join(ModelPath, model), and core/backend/options.go
+// only overrides it for a managed artifact. The first case below is therefore
+// the regression test; the other three are what it must not have broken.
+
+// std::string::ends_with is C++20 and this target is C++17.
+static bool ends_with(const std::string &value, const std::string &suffix) {
+    return value.size() >= suffix.size() &&
+           value.compare(value.size() - suffix.size(), suffix.size(), suffix) == 0;
+}
+
+// THE REGRESSION CASE. What a model YAML saying `model: bundled:silero_vad`
+// actually arrives as: Model intact, ModelFile joined onto the models directory.
+static void test_bundled_in_model_survives_a_joined_model_file() {
+    const std::string resolved = audiocpp_backend::resolve_model_path(
+        "/models", "/models/bundled:silero_vad", "bundled:silero_vad");
+
+    check(ends_with(resolved, "/assets/silero_vad"),
+          "bundled: in Model resolves under the package assets dir (got \"" +
+              resolved + "\")");
+    // Checked separately from the suffix because this is the failure that
+    // shipped: the joined ModelFile came back verbatim and the load died on
+    // "model path does not exist: /models/bundled:silero_vad".
+    check(resolved.find("/models/") == std::string::npos,
+          "bundled: in Model is not resolved against the models directory (got \"" +
+              resolved + "\")");
+}
+
+// Task 15's shape: the form in ModelFile with Model empty. It worked before the
+// fix and must keep working.
+static void test_bundled_in_model_file_still_resolves() {
+    const std::string resolved =
+        audiocpp_backend::resolve_model_path("", "bundled:marblenet_vad", "");
+
+    check(ends_with(resolved, "/assets/marblenet_vad"),
+          "bundled: in ModelFile still resolves under the package assets dir (got \"" +
+              resolved + "\")");
+}
+
+// The ordinary case, and the one the bundled: lookup must not capture: a real
+// artifact path in ModelFile with a plain name in Model.
+static void test_a_plain_name_resolves_to_the_model_file() {
+    const std::string resolved = audiocpp_backend::resolve_model_path(
+        "/models", "/models/chatterbox-q8_0.gguf", "chatterbox-q8_0.gguf");
+
+    check_eq(resolved, "/models/chatterbox-q8_0.gguf",
+             "a plain name resolves to the absolute ModelFile");
+}
+
+// A relative ModelFile is still joined onto ModelPath. The fix does not touch
+// this branch, which is why it is pinned: the bundled: lookup now runs before it
+// and has to fall through for every non-bundled input.
+static void test_a_relative_model_file_joins_the_model_path() {
+    const std::string resolved = audiocpp_backend::resolve_model_path(
+        "/models", "sub/nemotron-asr-q8_0.gguf", "nemotron-asr");
+
+    check_eq(resolved, "/models/sub/nemotron-asr-q8_0.gguf",
+             "a relative ModelFile joins the models directory");
+}
+
+int main() {
+    test_bundled_in_model_survives_a_joined_model_file();
+    test_bundled_in_model_file_still_resolves();
+    test_a_plain_name_resolves_to_the_model_file();
+    test_a_relative_model_file_joins_the_model_path();
+    test_begin_stream_prepares_then_starts();
+    test_base_start_stream_resets();
+    test_begin_stream_refuses_a_non_streaming_session();
+    test_a_refetched_session_replays_identically();
+    test_run_streaming_audio_installs_and_clears_the_sink();
+    test_the_sink_is_cleared_when_the_stream_throws();
+    test_chunking_honours_the_policy_sample_count();
+    test_a_partial_trailing_frame_is_refused();
+    test_chunking_falls_back_to_the_policy_seconds();
+    test_chunking_falls_back_to_the_interface_default();
+    test_a_seconds_policy_with_no_rate_falls_through();
+    test_stereo_chunks_are_frame_aligned();
+    test_pull_drains_every_event_and_installs_no_sink();
+    test_pull_stops_on_a_final_event();
+    test_pull_refuses_a_non_streaming_session();
+    test_prepare_tracks_the_request_not_the_session();
+    test_live_buffers_wire_frames_into_policy_windows();
+    test_live_installs_and_clears_the_sink();
+    test_live_with_no_audio_never_finalizes();
+    test_live_feeds_a_short_tail();
+    test_live_prepares_at_the_contract_rate();
+    test_live_clears_the_sink_when_the_puller_throws();
+    test_live_replays_identically_on_a_refetched_session();
+    test_live_refuses_a_non_streaming_session();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all streaming driver checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/transcript_assembly.cpp

@@ -0,0 +1,215 @@
+#include "transcript_assembly.h"
+
+#include "audio_units.h"
+
+#include <algorithm>
+#include <cstdlib>
+#include <limits>
+
+namespace audiocpp_backend {
+namespace {
+
+std::int64_t midpoint(const Span &span) {
+    return span.start_sample + (span.end_sample - span.start_sample) / 2;
+}
+
+bool contains(const Span &span, std::int64_t sample) {
+    return sample >= span.start_sample && sample < span.end_sample;
+}
+
+std::int64_t overlap(const Span &a, const Span &b) {
+    const std::int64_t begin = std::max(a.start_sample, b.start_sample);
+    const std::int64_t end = std::min(a.end_sample, b.end_sample);
+    return end > begin ? end - begin : 0;
+}
+
+// Joins a segment's words into that segment's text.
+//
+// THE SEPARATOR IS NOT ALWAYS A SPACE, and getting it wrong is visible to every
+// caller rather than cosmetic: core/http/endpoints/openai/transcription.go
+// routes response_format text, srt, vtt and lrc through
+// schema.TranscriptionResponse, which builds the entire body out of
+// Segments[].Text and never reads the top-level text. For those four formats
+// the segment text IS the response.
+//
+// Two producer conventions have to be told apart:
+//
+//   whole words    "Some", "call", "me"   -> join with a space
+//   subword pieces "So", "me", " call"    -> concatenate
+//
+// The second is SentencePiece, where a word boundary is carried as a LEADING
+// SPACE on the piece; nemotron_asr emits one entry per token in exactly that
+// form. Space-joining those produced "So me  call   me  na ture ,", which is
+// what response_format=text returned while the correct sentence sat unread in
+// the top-level field. Concatenating them reproduces text_output exactly.
+//
+// The convention is read off the words themselves, because nothing else in the
+// result declares it. One leading space anywhere is enough to decide: a
+// whole-word producer has no reason to emit one, and a subword producer emits
+// one at every word boundary, so the two populations do not overlap. A producer
+// that mixed both conventions inside one segment could not be served correctly
+// by any single separator; this picks concatenation for it.
+//
+// This does NOT touch the top-level text, which stays text_output verbatim. The
+// rule that forbids deriving the transcript from the segments is about the
+// direction segments -> text. Segment text has no source other than its words
+// and is necessarily derived.
+std::string join_words(const std::vector<OutWord> &words) {
+    const bool subword_pieces =
+        std::any_of(words.begin(), words.end(), [](const OutWord &word) {
+            return !word.text.empty() && word.text.front() == ' ';
+        });
+    std::string out;
+    for (const auto &word : words) {
+        if (word.text.empty()) {
+            continue;
+        }
+        if (!subword_pieces && !out.empty()) {
+            out += " ";
+        }
+        out += word.text;
+    }
+    return out;
+}
+
+std::string speaker_for(const Span &segment,
+                        const std::vector<SpeakerSpan> &turns) {
+    std::string best;
+    std::int64_t best_overlap = 0;
+    for (const auto &turn : turns) {
+        const std::int64_t shared = overlap(segment, turn.span);
+        if (shared > best_overlap) {
+            best_overlap = shared;
+            best = turn.speaker;
+        }
+    }
+    return best;
+}
+
+// The chosen segmentation. labels is empty unless the spans were sourced from
+// the speaker turns themselves, in which case it is parallel to spans and holds
+// the label each span arrived with.
+struct SegmentSource {
+    std::vector<Span> spans;
+    std::vector<std::string> labels;
+};
+
+// Chooses the segment spans, per the documented precedence.
+SegmentSource choose_segment_spans(const std::string &text_output,
+                                   const std::vector<Span> &speech_segments,
+                                   const std::vector<SpeakerSpan> &speaker_turns,
+                                   const std::vector<WordSpan> &words) {
+    if (!speech_segments.empty()) {
+        return {speech_segments, {}};
+    }
+    if (!speaker_turns.empty()) {
+        // The labels are carried out rather than re-derived by overlap later. A
+        // turn wholly contained in another speaker's turn overlaps its own span
+        // completely, which is the largest overlap possible, so it can only tie
+        // with the containing turn and would then lose that tie on order.
+        // sortformer_diar binarizes each speaker's track independently and
+        // sorts the result by start sample, so the container always comes
+        // first, and the interjecting speaker would be silently relabelled to
+        // the speaker it interrupted.
+        SegmentSource source;
+        source.spans.reserve(speaker_turns.size());
+        source.labels.reserve(speaker_turns.size());
+        for (const auto &turn : speaker_turns) {
+            source.spans.push_back(turn.span);
+            source.labels.push_back(turn.speaker);
+        }
+        return source;
+    }
+    if (!words.empty()) {
+        Span covering = words.front().span;
+        for (const auto &word : words) {
+            covering.start_sample =
+                std::min(covering.start_sample, word.span.start_sample);
+            covering.end_sample = std::max(covering.end_sample, word.span.end_sample);
+        }
+        return {{covering}, {}};
+    }
+    if (!text_output.empty()) {
+        // A zero span rather than a fabricated duration: the model reported no
+        // timing, and inventing one would be a lie the caller cannot detect.
+        return {{Span{0, 0}}, {}};
+    }
+    return {};
+}
+
+// Returns the index of the segment a word belongs to, or the nearest segment
+// when the word falls outside all of them.
+size_t segment_index_for_word(const std::vector<Span> &spans, const Span &word) {
+    const std::int64_t centre = midpoint(word);
+    for (size_t i = 0; i < spans.size(); ++i) {
+        if (contains(spans[i], centre)) {
+            return i;
+        }
+    }
+    size_t nearest = 0;
+    std::int64_t best_distance = std::numeric_limits<std::int64_t>::max();
+    for (size_t i = 0; i < spans.size(); ++i) {
+        const std::int64_t distance = std::llabs(midpoint(spans[i]) - centre);
+        if (distance < best_distance) {
+            best_distance = distance;
+            nearest = i;
+        }
+    }
+    return nearest;
+}
+
+} // namespace
+
+AssembledTranscript assemble_transcript(const std::string &text_output,
+                                        const std::vector<Span> &speech_segments,
+                                        const std::vector<SpeakerSpan> &speaker_turns,
+                                        const std::vector<WordSpan> &words,
+                                        int sample_rate) {
+    AssembledTranscript assembled;
+    // THE RULE. Never derived from spans.
+    assembled.text = text_output;
+
+    const SegmentSource source =
+        choose_segment_spans(text_output, speech_segments, speaker_turns, words);
+    const std::vector<Span> &spans = source.spans;
+    if (spans.empty()) {
+        return assembled;
+    }
+
+    assembled.segments.resize(spans.size());
+    for (size_t i = 0; i < spans.size(); ++i) {
+        OutSegment &segment = assembled.segments[i];
+        segment.id = static_cast<int>(i);
+        segment.start_ns = samples_to_nanoseconds(spans[i].start_sample, sample_rate);
+        segment.end_ns = samples_to_nanoseconds(spans[i].end_sample, sample_rate);
+        // A segment that came from a speaker turn already knows its speaker.
+        // Only the other three sources have to look one up by overlap.
+        segment.speaker = source.labels.empty()
+                              ? speaker_for(spans[i], speaker_turns)
+                              : source.labels[i];
+    }
+
+    for (const auto &word : words) {
+        const size_t index = segment_index_for_word(spans, word.span);
+        OutWord out;
+        out.start_ns = samples_to_nanoseconds(word.span.start_sample, sample_rate);
+        out.end_ns = samples_to_nanoseconds(word.span.end_sample, sample_rate);
+        out.text = word.word;
+        assembled.segments[index].words.push_back(out);
+    }
+
+    for (auto &segment : assembled.segments) {
+        segment.text = join_words(segment.words);
+    }
+
+    // A single segment with no word timing carries the whole transcript. With
+    // several segments there is no defensible way to split the text, so their
+    // per-segment text stays empty and only the top-level text is authoritative.
+    if (assembled.segments.size() == 1 && assembled.segments[0].words.empty()) {
+        assembled.segments[0].text = text_output;
+    }
+
+    return assembled;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/transcript_assembly.h

@@ -0,0 +1,87 @@
+#pragma once
+
+// Builds LocalAI's TranscriptResult shape from audio.cpp's TaskResult spans.
+// Standard library only; result_map.cpp converts the engine types into these
+// PODs at the boundary.
+//
+// THE RULE: the top-level transcript text is text_output verbatim, always.
+// audio.cpp carries transcript text in exactly one place, TaskResult.text_output.
+// speech_segments, speaker_turns and word_timestamps carry spans and labels but
+// no text. Deriving the top-level text by concatenating per-segment text
+// therefore yields an empty transcript for every producer that reports segments
+// without word timestamps, which includes VibeVoice diarized ASR.
+
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace audiocpp_backend {
+
+// Sample-index span, mirroring engine::runtime::TimeSpan.
+struct Span {
+    std::int64_t start_sample = 0;
+    std::int64_t end_sample = 0;
+};
+
+struct WordSpan {
+    Span span;
+    std::string word;
+};
+
+struct SpeakerSpan {
+    Span span;
+    std::string speaker;
+};
+
+struct OutWord {
+    std::int64_t start_ns = 0;
+    std::int64_t end_ns = 0;
+    std::string text;
+};
+
+struct OutSegment {
+    int id = 0;
+    std::int64_t start_ns = 0;
+    std::int64_t end_ns = 0;
+    std::string text;
+    std::string speaker;
+    std::vector<OutWord> words;
+};
+
+struct AssembledTranscript {
+    std::string text;
+    std::vector<OutSegment> segments;
+};
+
+// Segment source, first non-empty wins:
+//   1. speech_segments
+//   2. speaker_turns
+//   3. one segment spanning all words, when words are present
+//   4. one zero-span segment carrying the full text, when text is present
+//   5. no segments
+//
+// Words attach to the segment whose range contains their midpoint; a word
+// outside every segment attaches to the nearest one by midpoint distance so it
+// is never silently dropped. A lone segment with no words carries the full text.
+//
+// A segment's text is its words joined, and the separator depends on the
+// producer's convention: whole words ("Some", "call") are joined with a space,
+// while SentencePiece-style subword pieces, which carry the word boundary as a
+// LEADING SPACE (" call"), are concatenated. One leading space anywhere in the
+// segment selects concatenation. This matters beyond tidiness: response_format
+// text, srt, vtt and lrc build their entire body out of the segment text and
+// never read the top-level text.
+//
+// A segment's speaker is the speaker turn with the greatest overlap, except
+// when the segments came from the speaker turns themselves (source 2), where
+// each segment keeps its own turn's label. Re-deriving it there loses a turn
+// nested inside another speaker's turn: the nested turn overlaps its own span
+// completely, so it can only tie with the containing turn, which is listed
+// first and wins the tie.
+AssembledTranscript assemble_transcript(const std::string &text_output,
+                                        const std::vector<Span> &speech_segments,
+                                        const std::vector<SpeakerSpan> &speaker_turns,
+                                        const std::vector<WordSpan> &words,
+                                        int sample_rate);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/transcript_assembly_test.cpp

@@ -0,0 +1,532 @@
+// Unit tests for transcript_assembly. Standard library only. The harness
+// compiles this as a single translation unit, so both implementations are
+// included directly rather than linked.
+//
+// Every fixture below either mirrors a producer shape actually observed from
+// audio.cpp families, and names the families it was checked against, or says in
+// its own comment that it is defensive. Do not replace an observed shape with an
+// invented one and do not quietly promote a defensive fixture to an observed
+// one: an invented shape is what let the earlier attempt ship an empty
+// transcript.
+
+#include "audio_units.cpp"
+#include "transcript_assembly.cpp"
+
+#include <cstddef>
+#include <cstdio>
+#include <string>
+#include <vector>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using namespace audiocpp_backend;
+
+// Indexed access that reports a named failure instead of running off the end.
+// std::vector::operator[] past the end is undefined behaviour, so a regression
+// that drops a segment would crash the process here and take every later check
+// with it. Returning a default element keeps the rest of the suite reporting.
+static const OutSegment &segment_at(const AssembledTranscript &out, size_t index,
+                                    const std::string &name) {
+    static const OutSegment missing;
+    if (index >= out.segments.size()) {
+        failures++;
+        fprintf(stderr, "FAIL: %s (segment %zu is missing)\n", name.c_str(), index);
+        return missing;
+    }
+    return out.segments[index];
+}
+
+static const OutWord &word_at(const OutSegment &segment, size_t index,
+                              const std::string &name) {
+    static const OutWord missing;
+    if (index >= segment.words.size()) {
+        failures++;
+        fprintf(stderr, "FAIL: %s (word %zu is missing)\n", name.c_str(), index);
+        return missing;
+    }
+    return segment.words[index];
+}
+
+static const int kRate = 16000;
+
+// Shape A: word timestamps only. Emitted by nemotron_asr, qwen3_asr and
+// qwen3_forced_aligner, all of which set text_output plus word_timestamps and
+// leave speech_segments empty.
+static void test_words_only() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "hello"},
+        {{8000, 16000}, "world"},
+    };
+    const auto out = assemble_transcript("hello world", {}, {}, words, kRate);
+
+    check(out.text == "hello world", "text is text_output verbatim");
+    check(out.segments.size() == 1, "words with no segments yield one segment");
+    const OutSegment &first = segment_at(out, 0, "words only segment");
+    check(first.start_ns == 0, "segment starts at the first word");
+    check(first.end_ns == 1000000000LL, "segment ends at the last word");
+    check(first.words.size() == 2, "both words attached");
+    check(word_at(first, 0, "first word").text == "hello", "first word text");
+    check(word_at(first, 1, "second word").start_ns == 500000000LL,
+          "second word start in ns");
+    check(first.text == "hello world", "segment text joins its words");
+    check(first.id == 0, "ids are zero based");
+}
+
+// Shape A-whole: the whole-word convention, stated explicitly rather than left
+// implicit in the shape A tests. qwen3_forced_aligner emits one entry per WORD
+// (processor.cpp parses per-word timestamp tokens), so its pieces carry no
+// leading space and must be joined with one.
+static void test_whole_words_are_space_joined() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "Some"},
+        {{8000, 16000}, "call"},
+        {{16000, 24000}, "me"},
+    };
+    const auto out = assemble_transcript("Some call me", {}, {}, words, kRate);
+
+    check(segment_at(out, 0, "whole words").text == "Some call me",
+          "whole words are joined with a single space");
+}
+
+// Shape A-subword: the SentencePiece convention, where the word boundary is a
+// LEADING SPACE on the piece. These are the first eleven word_timestamps
+// nemotron_asr actually returned for audio.cpp/assets/resources/sample_16k.wav
+// with the q8_0 GGUF, copied verbatim rather than invented, including the lone
+// " " piece at index 3.
+//
+// Space-joining these produced "So me  call   me  na ture ,  other s  call",
+// which is not a cosmetic problem: response_format text, srt, vtt and lrc build
+// their entire body from the segment text and never read the top-level text, so
+// that string WAS the transcription response for those formats.
+static void test_subword_pieces_are_concatenated() {
+    const std::vector<WordSpan> words = {
+        {{15360, 16640}, "So"},   {{15360, 16640}, "me"},
+        {{23040, 24320}, " call"}, {{28160, 29440}, " "},
+        {{28160, 29440}, "me"},   {{30720, 32000}, " na"},
+        {{33280, 34560}, "ture"}, {{35840, 37120}, ","},
+        {{38400, 39680}, " other"}, {{40960, 42240}, "s"},
+        {{43520, 44800}, " call"},
+    };
+    const auto out = assemble_transcript(
+        "Some call me nature, others call me mother nature.", {}, {}, words, kRate);
+
+    check(out.text == "Some call me nature, others call me mother nature.",
+          "the top-level text is still text_output verbatim");
+    check(segment_at(out, 0, "subword pieces").text ==
+              "Some call me nature, others call",
+          "subword pieces are concatenated, reproducing text_output");
+}
+
+// One leading space anywhere decides for the whole segment. A subword producer
+// emits a boundary space at every word start, so its first piece, which is
+// sentence-initial, does not have one; keying off the first piece alone would
+// therefore pick the wrong convention on every segment.
+static void test_a_single_leading_space_selects_concatenation() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "al"},
+        {{8000, 16000}, "pha"},
+        {{16000, 24000}, " beta"},
+    };
+    const auto out = assemble_transcript("alpha beta", {}, {}, words, kRate);
+
+    check(segment_at(out, 0, "mixed").text == "alpha beta",
+          "a leading space on a later piece selects concatenation");
+}
+
+// Shape A': the same producer, but text_output is punctuated and cased while
+// the word timestamps are not. qwen3_asr rebuilds text_output from its word
+// list only when timestamps are requested, so the two genuinely differ; this
+// pins the sole segment's text to its words rather than to the top-level text.
+static void test_words_only_with_punctuated_text_output() {
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "hello"},
+        {{8000, 16000}, "world"},
+    };
+    const auto out = assemble_transcript("Hello, world!", {}, {}, words, kRate);
+
+    check(out.text == "Hello, world!", "punctuated text_output is untouched");
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "punctuated segment").text == "hello world",
+          "a segment with words takes its text from the words, not text_output");
+}
+
+// Shape A'': a merged word list whose last word is not the one that ends
+// latest. audio.cpp concatenates per-chunk word lists in chunk order
+// (append_chunk_word_timestamps in framework/audio/chunking.cpp). It drops a
+// word whose global start falls before the chunk's keep span, but it never
+// clips a word's end to that boundary, so the last word kept from one chunk can
+// outlast the first word kept from the next. The covering span must therefore
+// be the extent of every word, not the span from the first to the last.
+static void test_covering_span_spans_every_word() {
+    const std::vector<WordSpan> words = {
+        {{0, 4000}, "a"},
+        // Kept from the earlier chunk, ending past the chunk boundary.
+        {{4000, 10000}, "b"},
+        // First word of the next chunk, shorter, so it ends earlier.
+        {{8000, 9000}, "c"},
+    };
+    const auto out = assemble_transcript("a b c", {}, {}, words, kRate);
+
+    check(out.segments.size() == 1, "one covering segment");
+    check(segment_at(out, 0, "covering segment").end_ns == 625000000LL,
+          "the covering span reaches the latest word end, not the last word's");
+}
+
+// Defensive, not observed: no pinned family emits a word with no text.
+// nemotron_asr's build_token_timestamps (models/nemotron_asr/decoder.cpp:97)
+// skips a token that decodes to an empty chunk before it ever becomes a
+// WordTimestamp. join_words guards against one anyway, and an unexercised guard
+// is a guard the next reader deletes as dead weight.
+static void test_empty_word_contributes_no_separator() {
+    const std::vector<WordSpan> words = {
+        {{0, 4000}, "alpha"},
+        {{4000, 8000}, ""},
+        {{8000, 12000}, "beta"},
+    };
+    const auto out = assemble_transcript("alpha beta", {}, {}, words, kRate);
+
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "sole segment").text == "alpha beta",
+          "an empty word adds no separator to the segment text");
+    check(segment_at(out, 0, "sole segment").words.size() == 3,
+          "the empty word still reports its span");
+}
+
+// Shape B: speech segments, no words. Emitted by ASR families that report
+// utterance boundaries without word-level timing.
+static void test_segments_without_words() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const auto out = assemble_transcript("one two three", segments, {}, {}, kRate);
+
+    // The regression: this must NOT be empty.
+    check(out.text == "one two three", "multi-segment text is not empty");
+    check(out.segments.size() == 2, "both segments survive");
+    check(segment_at(out, 0, "first segment").end_ns == 1000000000LL,
+          "first segment ends at 1s");
+    check(segment_at(out, 1, "second segment").start_ns == 1000000000LL,
+          "second segment starts at 1s");
+    check(segment_at(out, 0, "first segment").text.empty(),
+          "per-segment text stays empty when there are no words to split by");
+    check(segment_at(out, 1, "second segment").id == 1, "ids increment");
+}
+
+// Shape C: speech segments plus speaker turns, no words. This is the real
+// VibeVoice diarized ASR shape that broke the earlier attempt.
+static void test_segments_with_speaker_turns_no_words() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 16000}, "SPEAKER_00"},
+        {{16000, 32000}, "SPEAKER_01"},
+    };
+    const auto out = assemble_transcript("hi there", segments, turns, {}, kRate);
+
+    check(out.text == "hi there", "diarized multi-segment text is not empty");
+    check(out.segments.size() == 2, "two segments");
+    check(segment_at(out, 0, "first diarized segment").speaker == "SPEAKER_00",
+          "first speaker assigned");
+    check(segment_at(out, 1, "second diarized segment").speaker == "SPEAKER_01",
+          "second speaker assigned");
+}
+
+// Defensive, not observed: speech segments and speaker turns that disagree.
+// vibevoice_asr builds each SpeakerTurn with turn.span = speech_segment.span in
+// one loop (models/vibevoice_asr/session.cpp:965) and shifts and clips both
+// lists identically when merging chunks, so in practice the two lists are 1:1
+// with identical spans. That is exactly why the shape C fixture above cannot
+// show which list is the segment source: swapping the precedence there produces
+// byte-identical output. This fixture pins the precedence, and it is the shape
+// any future family that segments and diarizes separately would produce.
+static void test_speech_segments_outrank_speaker_turns() {
+    const std::vector<Span> segments = {{0, 32000}};
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 16000}, "SPEAKER_00"},
+        {{16000, 32000}, "SPEAKER_01"},
+    };
+    const auto out = assemble_transcript("hi there", segments, turns, {}, kRate);
+
+    check(out.segments.size() == 1,
+          "speech segments decide the segmentation, not speaker turns");
+    check(segment_at(out, 0, "single utterance").end_ns == 2000000000LL,
+          "the utterance keeps its own span");
+}
+
+// Defensive, not observed: no pinned family emits speaker turns and word
+// timestamps together. It pins rule 2 against rule 3, which nothing else does:
+// a diarized result is segmented by who spoke, and words only fill the turns in.
+static void test_speaker_turns_outrank_words() {
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 16000}, "SPEAKER_00"},
+        {{16000, 32000}, "SPEAKER_01"},
+    };
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "hi"},
+        {{16000, 24000}, "there"},
+    };
+    const auto out = assemble_transcript("hi there", {}, turns, words, kRate);
+
+    check(out.segments.size() == 2, "the two turns segment the result");
+    check(segment_at(out, 0, "turn 0").text == "hi", "first turn takes its word");
+    check(segment_at(out, 1, "turn 1").text == "there", "second turn takes its word");
+}
+
+// Shape D: text only. Emitted by ASR families that report no timing at all,
+// such as hviske_asr and citrinet_asr.
+static void test_text_only() {
+    const auto out = assemble_transcript("just text", {}, {}, {}, kRate);
+
+    check(out.text == "just text", "text survives");
+    check(out.segments.size() == 1, "a single synthetic segment is emitted");
+    const OutSegment &only = segment_at(out, 0, "synthetic segment");
+    check(only.start_ns == 0 && only.end_ns == 0,
+          "synthetic segment has zero span, not a fabricated duration");
+    check(only.text == "just text", "the sole segment carries the full text");
+}
+
+// Shape E: speaker turns only, no speech segments and no text. This is
+// sortformer_diar, reached through the Diarize RPC.
+static void test_speaker_turns_only() {
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 24000}, "0"},
+        {{24000, 48000}, "1"},
+    };
+    const auto out = assemble_transcript("", {}, turns, {}, kRate);
+
+    check(out.text.empty(), "no text is reported when the model produced none");
+    check(out.segments.size() == 2, "turns become segments");
+    check(segment_at(out, 0, "turn 0").speaker == "0",
+          "speaker label preserved verbatim");
+    check(segment_at(out, 1, "turn 1").start_ns == 1500000000LL,
+          "second turn starts at 1.5s");
+}
+
+// Shape E', the same producer with one speaker talking over another.
+// decode_sortformer_speaker_turns (models/sortformer_diar/postprocess.cpp)
+// binarizes each speaker's probability track independently, which is the whole
+// point of sortformer, then sorts the turns by start sample. So a turn can be
+// wholly contained in another speaker's turn, and the containing turn always
+// comes first. A segment sourced from a speaker turn must keep that turn's own
+// label: re-deriving it by overlap can only ever tie with the containing turn,
+// which then wins on order and silently erases the interjecting speaker.
+static void test_nested_speaker_turn_keeps_its_own_label() {
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 100000}, "speaker_0"},
+        {{10000, 20000}, "speaker_1"},
+    };
+    const auto out = assemble_transcript("", {}, turns, {}, kRate);
+
+    check(out.segments.size() == 2, "both turns become segments");
+    check(segment_at(out, 0, "containing turn").speaker == "speaker_0",
+          "the containing turn keeps its label");
+    check(segment_at(out, 1, "nested turn").speaker == "speaker_1",
+          "a turn nested inside another is not relabelled to the container");
+}
+
+// Shape F: nothing at all. A model that ran but produced no output must not
+// crash or fabricate a segment.
+static void test_empty() {
+    const auto out = assemble_transcript("", {}, {}, {}, kRate);
+    check(out.text.empty(), "empty stays empty");
+    check(out.segments.empty(), "no segments are invented");
+}
+
+// Shape H: speech segments with no text and no words at all. This is the VAD
+// path, silero_vad and marblenet_vad, which fill speech_segments and never
+// touch text_output. It reaches the lone-segment rule with nothing to carry.
+static void test_vad_segments_without_text() {
+    const std::vector<Span> segments = {{0, 16000}, {24000, 32000}};
+    const auto out = assemble_transcript("", segments, {}, {}, kRate);
+
+    check(out.text.empty(), "VAD reports no text");
+    check(out.segments.size() == 2, "both speech regions survive");
+    check(segment_at(out, 1, "second speech region").start_ns == 1500000000LL,
+          "second region starts at 1.5s");
+    check(segment_at(out, 0, "first speech region").text.empty(),
+          "a VAD segment carries no text");
+
+    const std::vector<Span> one = {{0, 16000}};
+    const auto single = assemble_transcript("", one, {}, {}, kRate);
+    check(single.segments.size() == 1, "a single speech region survives");
+    check(segment_at(single, 0, "lone speech region").text.empty(),
+          "a lone VAD segment does not fabricate text");
+}
+
+// Shape G: segments and words together. Words are assigned by midpoint so a
+// word straddling a boundary lands in exactly one segment.
+static void test_words_distributed_into_segments() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const std::vector<WordSpan> words = {
+        {{0, 4000}, "alpha"},
+        {{4000, 8000}, "beta"},
+        // Straddles the boundary; midpoint 16000 falls in the second segment.
+        {{12000, 20000}, "gamma"},
+        {{20000, 28000}, "delta"},
+    };
+    const auto out = assemble_transcript("alpha beta gamma delta", segments, {},
+                                         words, kRate);
+
+    check(out.text == "alpha beta gamma delta", "top level text unchanged");
+    check(out.segments.size() == 2, "two segments");
+    check(segment_at(out, 0, "first segment").words.size() == 2,
+          "first segment takes two words");
+    check(segment_at(out, 1, "second segment").words.size() == 2,
+          "second segment takes two words");
+    check(segment_at(out, 0, "first segment").text == "alpha beta",
+          "first segment text");
+    check(segment_at(out, 1, "second segment").text == "gamma delta",
+          "boundary-straddling word lands by midpoint");
+}
+
+// The midpoint rule is not the same as either endpoint rule. "early" starts in
+// the first segment but ends in the second, and "late" the other way round;
+// each must land where its midpoint says, which no start-only or end-only rule
+// reproduces.
+static void test_words_assigned_by_midpoint_not_endpoint() {
+    const std::vector<Span> segments = {{0, 16000}, {16000, 32000}};
+    const std::vector<WordSpan> words = {
+        // Midpoint 12000 -> first segment, although it ends in the second.
+        {{4000, 20000}, "early"},
+        // Midpoint 20000 -> second segment, although it starts in the first.
+        {{12000, 28000}, "late"},
+    };
+    const auto out = assemble_transcript("early late", segments, {}, words, kRate);
+
+    check(segment_at(out, 0, "first segment").text == "early",
+          "a word ending past the boundary stays where its midpoint is");
+    check(segment_at(out, 1, "second segment").text == "late",
+          "a word starting before the boundary follows its midpoint");
+}
+
+// A word outside every segment must still be reachable rather than dropped
+// silently, so it attaches to the nearest segment by midpoint distance.
+static void test_word_outside_all_segments() {
+    const std::vector<Span> segments = {{0, 16000}};
+    const std::vector<WordSpan> words = {
+        {{0, 8000}, "inside"},
+        {{40000, 48000}, "outside"},
+    };
+    const auto out = assemble_transcript("inside outside", segments, {}, words,
+                                         kRate);
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "sole segment").words.size() == 2,
+          "the stray word is not dropped");
+}
+
+// The fallback picks the nearest segment, which is not the same as picking the
+// first. With one segment the two are indistinguishable, so this uses three and
+// puts the stray word past the last one.
+static void test_stray_word_goes_to_the_nearest_segment() {
+    const std::vector<Span> segments = {{0, 8000}, {8000, 16000}, {16000, 24000}};
+    const std::vector<WordSpan> words = {
+        // Midpoint 44000, nearest the third segment.
+        {{40000, 48000}, "trailing"},
+    };
+    const auto out = assemble_transcript("trailing", segments, {}, words, kRate);
+
+    check(segment_at(out, 0, "first segment").words.empty(),
+          "the stray word does not fall back to the first segment");
+    check(segment_at(out, 2, "third segment").text == "trailing",
+          "the stray word attaches to the nearest segment");
+}
+
+// "Nearest" is measured from the segment's midpoint, and it is neither "the
+// first segment" nor "the last". A leading stray word is the case a
+// trailing-only fixture cannot reach: forced-aligner words scored against VAD
+// segments produce one, and with only trailing coverage it would land at the
+// end of the transcript with the suite green. Here the leading word's nearest
+// midpoint is the first segment while its nearest start is the second, and the
+// trailing word's nearest midpoint is the third while its nearest end is the
+// second, so no endpoint rule reproduces this assignment either.
+static void test_stray_word_distance_is_measured_from_the_midpoint() {
+    const std::vector<Span> segments = {{0, 2000}, {8000, 200000}, {300000, 302000}};
+    const std::vector<WordSpan> words = {
+        {{4000, 6000}, "lead"},
+        {{249000, 251000}, "trail"},
+    };
+    const auto out = assemble_transcript("lead trail", segments, {}, words, kRate);
+
+    check(out.segments.size() == 3, "three segments");
+    check(segment_at(out, 0, "first segment").text == "lead",
+          "the leading stray word goes to the nearest segment by midpoint");
+    check(segment_at(out, 2, "third segment").text == "trail",
+          "the trailing stray word goes to the nearest segment by midpoint");
+    check(segment_at(out, 1, "middle segment").words.empty(),
+          "the long middle segment claims neither stray word");
+}
+
+// Speaker assignment uses greatest overlap, not first match, so a turn that
+// barely touches a segment does not win over one that covers it.
+static void test_speaker_assigned_by_greatest_overlap() {
+    const std::vector<Span> segments = {{8000, 24000}};
+    const std::vector<SpeakerSpan> turns = {
+        {{0, 9000}, "brief"},   // overlaps 1000 samples
+        {{9000, 24000}, "main"} // overlaps 15000 samples
+    };
+    const auto out = assemble_transcript("x", segments, turns, {}, kRate);
+    check(out.segments.size() == 1, "one segment");
+    check(segment_at(out, 0, "sole segment").speaker == "main",
+          "greatest overlap wins");
+}
+
+// A segment no turn touches gets no speaker rather than the label of whichever
+// turn happened to be listed first.
+static void test_segment_without_any_overlapping_turn_has_no_speaker() {
+    const std::vector<Span> segments = {{0, 8000}, {40000, 48000}};
+    const std::vector<SpeakerSpan> turns = {{{0, 8000}, "SPEAKER_00"}};
+    const auto out = assemble_transcript("x", segments, turns, {}, kRate);
+
+    check(segment_at(out, 0, "overlapped segment").speaker == "SPEAKER_00",
+          "the overlapped segment is labelled");
+    check(segment_at(out, 1, "unlabelled segment").speaker.empty(),
+          "a segment no turn overlaps is left unlabelled");
+}
+
+static void test_zero_sample_rate_is_safe() {
+    const std::vector<Span> segments = {{0, 16000}};
+    const auto out = assemble_transcript("x", segments, {}, {}, 0);
+    check(out.segments.size() == 1, "a zero sample rate still yields the segment");
+    const OutSegment &only = segment_at(out, 0, "sole segment");
+    check(only.start_ns == 0 && only.end_ns == 0,
+          "unknown sample rate yields zero timings rather than garbage");
+}
+
+int main() {
+    test_words_only();
+    test_whole_words_are_space_joined();
+    test_subword_pieces_are_concatenated();
+    test_a_single_leading_space_selects_concatenation();
+    test_words_only_with_punctuated_text_output();
+    test_covering_span_spans_every_word();
+    test_empty_word_contributes_no_separator();
+    test_segments_without_words();
+    test_segments_with_speaker_turns_no_words();
+    test_speech_segments_outrank_speaker_turns();
+    test_speaker_turns_outrank_words();
+    test_text_only();
+    test_speaker_turns_only();
+    test_nested_speaker_turn_keeps_its_own_label();
+    test_empty();
+    test_vad_segments_without_text();
+    test_words_distributed_into_segments();
+    test_words_assigned_by_midpoint_not_endpoint();
+    test_word_outside_all_segments();
+    test_stray_word_goes_to_the_nearest_segment();
+    test_stray_word_distance_is_measured_from_the_midpoint();
+    test_speaker_assigned_by_greatest_overlap();
+    test_segment_without_any_overlapping_turn_has_no_speaker();
+    test_zero_sample_rate_is_safe();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all transcript_assembly checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/upstream_absence_ctest.cpp

@@ -0,0 +1,283 @@
+// Asserts the ABSENCES that capability_routing.cpp's five refusal reasons rest
+// on, against the engine itself rather than against somebody's reading of it.
+//
+// Those reasons are prose making checkable claims about a pinned third-party
+// checkout: "VoiceTaskKind has no codec entry", "no family advertises spk",
+// "miocodec advertises only vc and s2s". Prose rots silently across an
+// AUDIO_CPP_VERSION bump, and it rots in the worst possible place, since a
+// refusal that states a false fact is worse than a bare UNIMPLEMENTED: it will
+// be believed. One of the four claims this backend was planned against was
+// already false when it was written ("streaming exists for tts and asr only" is
+// contradicted by silero_vad). This test is what turns the next such change
+// from a silent lie on the wire into a build failure.
+//
+// Pinned at audio.cpp e800d435d130dc776baf6f3e6129bb62b1495c89. What follows
+// was true of that commit; a bump is exactly when it needs to be re-run.
+//
+// It links engine_runtime and queries make_default_registry(), touching only
+// include/engine/framework/**, like every other unit in this backend. It loads
+// no model and reads no file: advertise_loaders() is the path-free catalog
+// upstream publishes for --list-loaders.
+//
+// FOUR CAVEATS, so nobody reads more into a green run than it earns:
+//
+//   1. It cannot assert the AudioToAudioStream reason. That one contrasts
+//      LocalAI's OpenAI-Realtime contract (conversation, system prompt, tool
+//      loop) with what audio.cpp's s2s families actually do, and semantics are
+//      not a queryable property. What IS asserted is the enumerable half: that
+//      s2s is advertised by exactly miocodec and vevo2, which is the clause the
+//      message names by hand.
+//   2. It queries the LOADER catalog, while grpc-server.cpp reads the LOADED
+//      model's own capabilities(). The two agree today, cross-checked on the
+//      wire: this test asserts miocodec advertises {vc/offline, s2s/offline},
+//      and a live LoadModel of miocodec-q8_0.gguf reports exactly
+//      "vc/offline, s2s/offline". A family whose loaded capabilities diverged
+//      from its advertisement would slip past, but nothing loads without a
+//      model file and a ctest cannot depend on one.
+//   3. Capabilities need not come from a loader at all.
+//      src/framework/model_spec/metadata.cpp's advertised_capabilities() builds
+//      a CapabilitySet from a spec's "capabilities"/"tasks"/"modes" keys, which
+//      is a second route by which a bump could falsify claim 3. Today no
+//      shipped model_specs/*.json carries a top-level "tasks" key and no loader
+//      calls that function, so the route is dead. It is covered anyway to the
+//      extent that a loader adopting it would surface through advertise_loaders
+//      like any other capability, which is why every assertion below queries
+//      advertised capabilities rather than loader source.
+//   4. An absence test passes trivially when the query is broken, so
+//      test_catalog_is_populated below is a POSITIVE control and is not
+//      optional. It proves the registry is non-empty and that the query does
+//      find a capability it should, before any absence is believed.
+
+#include "engine/framework/runtime/registry.h"
+#include "engine/framework/runtime/session.h"
+
+#include <algorithm>
+#include <cstdio>
+#include <set>
+#include <stdexcept>
+#include <string>
+#include <vector>
+
+namespace {
+
+int failures = 0;
+
+void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using engine::runtime::LoaderAdvertisement;
+using engine::runtime::RunMode;
+using engine::runtime::VoiceTaskKind;
+
+// Built once. make_default_registry constructs every loader, which is the
+// expensive part, and none of the assertions mutates it.
+const std::vector<LoaderAdvertisement> &catalog() {
+    static const std::vector<LoaderAdvertisement> kCatalog =
+        engine::runtime::make_default_registry().advertise_loaders();
+    return kCatalog;
+}
+
+bool advertises(const LoaderAdvertisement &loader, VoiceTaskKind task,
+                RunMode mode) {
+    for (const auto &capability : loader.capabilities.supported_tasks) {
+        if (capability.task != task) {
+            continue;
+        }
+        return std::find(capability.modes.begin(), capability.modes.end(), mode) !=
+               capability.modes.end();
+    }
+    return false;
+}
+
+bool advertises_any_mode(const LoaderAdvertisement &loader, VoiceTaskKind task) {
+    return advertises(loader, task, RunMode::Offline) ||
+           advertises(loader, task, RunMode::Streaming);
+}
+
+std::string describe(const LoaderAdvertisement &loader) {
+    std::string out;
+    for (const auto &capability : loader.capabilities.supported_tasks) {
+        for (const RunMode mode : capability.modes) {
+            if (!out.empty()) {
+                out += ", ";
+            }
+            out += engine::runtime::to_string(capability.task);
+            out += "/";
+            out += engine::runtime::to_string(mode);
+        }
+    }
+    return out.empty() ? "nothing" : out;
+}
+
+// THE POSITIVE CONTROL. Every other test here asserts an absence, and an
+// absence is what a broken query returns for everything. If make_default_registry
+// ever returns an empty registry, or advertise_loaders stops populating modes,
+// this is the test that fails instead of the suite going quietly green while
+// asserting nothing.
+void test_catalog_is_populated() {
+    check(catalog().size() >= 20,
+          "the default registry advertises a plausible number of families (" +
+              std::to_string(catalog().size()) + ")");
+
+    bool found_streaming_asr = false;
+    for (const auto &loader : catalog()) {
+        if (advertises(loader, VoiceTaskKind::Asr, RunMode::Streaming)) {
+            found_streaming_asr = true;
+            break;
+        }
+    }
+    check(found_streaming_asr,
+          "the query finds a capability that IS advertised (asr/streaming)");
+}
+
+// The AudioEncode and AudioDecode premise. Not "no family does codec" but the
+// stronger "the task kind does not exist", which is what makes those two RPCs
+// unroutable rather than merely unserved.
+//
+// Note what is NOT asserted: model_spec/schema.cpp's task whitelist DOES accept
+// the string "codec", so a spec declaring it validates and then fails here. That
+// is a hole in upstream's own validation, and it is deliberately kept off the
+// wire; the refusal rests on this parser, which is the thing routing would have
+// to go through.
+void test_no_codec_task_kind() {
+    bool threw = false;
+    try {
+        (void)engine::runtime::parse_voice_task_kind("codec");
+    } catch (const std::exception &) {
+        threw = true;
+    }
+    check(threw, "no codec task kind: parse_voice_task_kind(\"codec\") throws");
+
+    // The control for the line above: a real name must NOT throw, or the test
+    // would pass against a parser that rejected everything.
+    bool tts_threw = false;
+    try {
+        (void)engine::runtime::parse_voice_task_kind("tts");
+    } catch (const std::exception &) {
+        tts_threw = true;
+    }
+    check(!tts_threw, "parse_voice_task_kind accepts a real name (\"tts\")");
+}
+
+// The VoiceEmbed premise. SpeakerRecognition IS in the enum, and TitaNet and
+// ECAPA-TDNN exist as internal conditioning encoders; what is missing is any
+// registered family advertising the task, which is what the message says.
+void test_no_family_advertises_speaker_recognition() {
+    std::string offenders;
+    for (const auto &loader : catalog()) {
+        if (advertises_any_mode(loader, VoiceTaskKind::SpeakerRecognition)) {
+            if (!offenders.empty()) {
+                offenders += ", ";
+            }
+            offenders += loader.family;
+        }
+    }
+    check(offenders.empty(),
+          "no family advertises spk" +
+              (offenders.empty() ? std::string() : " (found: " + offenders + ")"));
+}
+
+// The AudioTransformStream premise, and the one that has to be scoped exactly.
+// The broad claim "upstream streams tts and asr only" is FALSE: silero_vad
+// advertises vad/streaming. The claim that holds is that none of the four tasks
+// AudioTransform routes to is advertised streaming by anybody.
+//
+// NEGATIVE CONTROL: add VoiceTaskKind::Tts to kTransformTasks and this test must
+// fail, naming the tts families. A run where that edit stays green means the
+// query is broken and every absence above is worthless.
+void test_no_streaming_for_the_transform_tasks() {
+    const VoiceTaskKind kTransformTasks[] = {
+        VoiceTaskKind::SourceSeparation,
+        VoiceTaskKind::VoiceConversion,
+        VoiceTaskKind::Svc,
+        VoiceTaskKind::SpeechToSpeech,
+    };
+
+    std::string offenders;
+    for (const VoiceTaskKind task : kTransformTasks) {
+        for (const auto &loader : catalog()) {
+            if (advertises(loader, task, RunMode::Streaming)) {
+                if (!offenders.empty()) {
+                    offenders += ", ";
+                }
+                offenders += loader.family;
+                offenders += "/";
+                offenders += engine::runtime::to_string(task);
+            }
+        }
+    }
+    check(offenders.empty(),
+          "no streaming for sep/vc/svc/s2s" +
+              (offenders.empty() ? std::string() : " (found: " + offenders + ")"));
+}
+
+// The clause the AudioEncode message names by hand: miocodec carries a Codec tag
+// in upstream's README, and its loader advertises only vc and s2s. Asserted
+// exactly, not as a subset, so a bump that ADDS a codec capability to miocodec
+// fails here rather than leaving the message stale.
+void test_miocodec_advertises_exactly_vc_and_s2s() {
+    const LoaderAdvertisement *miocodec = nullptr;
+    for (const auto &loader : catalog()) {
+        if (loader.family == "miocodec") {
+            miocodec = &loader;
+            break;
+        }
+    }
+    if (miocodec == nullptr) {
+        check(false, "miocodec is a registered family");
+        return;
+    }
+    check(describe(*miocodec) == "vc/offline, s2s/offline",
+          "miocodec advertises exactly vc/offline, s2s/offline (got: " +
+              describe(*miocodec) + ")");
+}
+
+// The "declared only by" clause in the AudioToAudioStream message. Exact set,
+// for the same reason as above: a third s2s family would make the message stale
+// without making it obviously wrong.
+void test_speech_to_speech_is_exactly_miocodec_and_vevo2() {
+    std::set<std::string> families;
+    for (const auto &loader : catalog()) {
+        if (advertises_any_mode(loader, VoiceTaskKind::SpeechToSpeech)) {
+            families.insert(loader.family);
+        }
+    }
+    const std::set<std::string> expected = {"miocodec", "vevo2"};
+    std::string found;
+    for (const auto &family : families) {
+        if (!found.empty()) {
+            found += ", ";
+        }
+        found += family;
+    }
+    check(families == expected,
+          "s2s is advertised by exactly miocodec and vevo2 (got: " +
+              (found.empty() ? "nothing" : found) + ")");
+}
+
+} // namespace
+
+int main() {
+    test_catalog_is_populated();
+    test_no_codec_task_kind();
+    test_no_family_advertises_speaker_recognition();
+    test_no_streaming_for_the_transform_tasks();
+    test_miocodec_advertises_exactly_vc_and_s2s();
+    test_speech_to_speech_is_exactly_miocodec_and_vevo2();
+    if (failures) {
+        fprintf(stderr,
+                "%d upstream absence check(s) failed. A refusal message in "
+                "capability_routing.cpp now states something that is not true "
+                "of the pinned audio.cpp; fix the message, not this test.\n",
+                failures);
+        return 1;
+    }
+    fprintf(stderr, "all upstream absence checks passed\n");
+    return 0;
+}

backend/cpp/audio-cpp/wav_header.cpp

@@ -0,0 +1,63 @@
+#include "wav_header.h"
+
+#include <cstdint>
+#include <limits>
+
+namespace audiocpp_backend {
+namespace {
+
+void append_u32(std::string &out, std::uint32_t value) {
+    out.push_back(static_cast<char>(value & 0xFF));
+    out.push_back(static_cast<char>((value >> 8) & 0xFF));
+    out.push_back(static_cast<char>((value >> 16) & 0xFF));
+    out.push_back(static_cast<char>((value >> 24) & 0xFF));
+}
+
+void append_u16(std::string &out, std::uint16_t value) {
+    out.push_back(static_cast<char>(value & 0xFF));
+    out.push_back(static_cast<char>((value >> 8) & 0xFF));
+}
+
+// The unknown-length sentinel, in both the RIFF and the data chunk size.
+constexpr std::uint32_t kStreamingSize = 0xFFFFFFFFu;
+constexpr std::uint16_t kBitsPerSample = 16;
+constexpr std::uint32_t kPcmFmtChunkSize = 16;
+constexpr std::uint16_t kFormatTagPcm = 1;
+constexpr int kMaxChannels = 65535;
+
+} // namespace
+
+std::string streaming_wav_header(int sample_rate, int channels) {
+    int clamped_channels = channels > 0 ? channels : 1;
+    if (clamped_channels > kMaxChannels) {
+        clamped_channels = kMaxChannels;
+    }
+    const auto channel_count = static_cast<std::uint16_t>(clamped_channels);
+    const auto rate = static_cast<std::uint32_t>(sample_rate > 0 ? sample_rate : 0);
+    const auto block_align =
+        static_cast<std::uint16_t>(channel_count * (kBitsPerSample / 8));
+    // uint32 arithmetic on purpose: 384 kHz by 8 channels is 6.1 MB/s, which
+    // does not fit the uint16 block align it is derived from.
+    const std::uint32_t byte_rate = rate * static_cast<std::uint32_t>(block_align);
+    static_assert(std::numeric_limits<std::uint32_t>::max() >= 0xFFFFFFFFu,
+                  "the streaming sentinel must be representable");
+
+    std::string header;
+    header.reserve(44);
+    header += "RIFF";
+    append_u32(header, kStreamingSize); // unknown total length
+    header += "WAVE";
+    header += "fmt ";
+    append_u32(header, kPcmFmtChunkSize);
+    append_u16(header, kFormatTagPcm);
+    append_u16(header, channel_count);
+    append_u32(header, rate);
+    append_u32(header, byte_rate);
+    append_u16(header, block_align);
+    append_u16(header, kBitsPerSample);
+    header += "data";
+    append_u32(header, kStreamingSize); // unknown payload length
+    return header;
+}
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/wav_header.h

@@ -0,0 +1,39 @@
+#pragma once
+
+// Builds the 44 byte canonical WAV header that precedes a streamed PCM body.
+// Standard library only.
+//
+// TTSStream chunks travel in Reply.audio and the FIRST chunk must be this
+// header, or an HTTP client has no format to decode the PCM with. Because the
+// total length is unknown while the model is still generating, both size fields
+// carry 0xFFFFFFFF; that is the convention backend/go/vibevoice-cpp established
+// (govibevoicecpp.go, TTSStream) and the one core/backend/tts.go writes when it
+// synthesises a header itself.
+//
+// WHY THIS BACKEND SENDS THE HEADER RATHER THAN LETTING GO DO IT.
+// core/backend/tts.go's ModelTTSStream will emit a header of its own, but only
+// when the FIRST Reply carries a non-empty `message` field holding a JSON blob
+// with a sample_rate. This backend sends audio and never sets `message`, so
+// that branch never fires and there is exactly one header on the wire: this
+// one. Do not start setting Reply.message on this RPC without deleting the
+// header below, or every stream gains a second header 44 bytes into the PCM.
+//
+// The header this produces is byte-identical to the one pkg/audio.WAVHeader
+// serialises, which is what core/http/endpoints/openai/realtime_model.go
+// assumes when it reads the sample rate out of byte offset 24 of the first
+// callback.
+
+#include <string>
+
+namespace audiocpp_backend {
+
+// 16-bit PCM, little endian, interleaved.
+//
+// `channels` is clamped to at least 1 and at most 65535, so a garbage channel
+// count can never write a zero block align, which is what a reader divides the
+// data size by. `sample_rate` is clamped to at least 0 rather than wrapped:
+// a zero rate is visibly wrong to whoever reads the header, whereas the
+// 4294967295 an unsigned conversion of -1 would write looks like a real field.
+std::string streaming_wav_header(int sample_rate, int channels);
+
+} // namespace audiocpp_backend

backend/cpp/audio-cpp/wav_header_test.cpp

@@ -0,0 +1,163 @@
+// Unit tests for wav_header. Standard library only. The harness compiles this
+// as a single translation unit, so the implementation is included directly.
+
+#include "wav_header.cpp"
+
+#include <cstdint>
+#include <cstdio>
+#include <string>
+
+static int failures = 0;
+
+static void check(bool ok, const std::string &name) {
+    if (!ok) {
+        failures++;
+        fprintf(stderr, "FAIL: %s\n", name.c_str());
+    } else {
+        fprintf(stderr, "ok:   %s\n", name.c_str());
+    }
+}
+
+using audiocpp_backend::streaming_wav_header;
+
+static std::uint32_t read_u32(const std::string &data, size_t offset) {
+    return static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset])) |
+           (static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset + 1])) << 8) |
+           (static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset + 2])) << 16) |
+           (static_cast<std::uint32_t>(static_cast<unsigned char>(data[offset + 3])) << 24);
+}
+
+static std::uint16_t read_u16(const std::string &data, size_t offset) {
+    return static_cast<std::uint16_t>(
+        static_cast<std::uint16_t>(static_cast<unsigned char>(data[offset])) |
+        (static_cast<std::uint16_t>(static_cast<unsigned char>(data[offset + 1])) << 8));
+}
+
+static std::string to_hex(const std::string &data) {
+    static const char *digits = "0123456789abcdef";
+    std::string out;
+    out.reserve(data.size() * 2);
+    for (const char byte : data) {
+        const auto value = static_cast<unsigned char>(byte);
+        out.push_back(digits[value >> 4]);
+        out.push_back(digits[value & 0x0F]);
+    }
+    return out;
+}
+
+static void test_header_layout() {
+    const std::string header = streaming_wav_header(24000, 1);
+
+    check(header.size() == 44, "canonical 44 byte header");
+    check(header.compare(0, 4, "RIFF") == 0, "RIFF magic");
+    check(header.compare(8, 4, "WAVE") == 0, "WAVE magic");
+    check(header.compare(12, 4, "fmt ") == 0, "fmt chunk id");
+    check(header.compare(36, 4, "data") == 0, "data chunk id");
+
+    check(read_u32(header, 16) == 16, "PCM fmt chunk is 16 bytes");
+    check(read_u16(header, 20) == 1, "format tag 1 is PCM");
+    check(read_u16(header, 22) == 1, "mono channel count");
+    check(read_u32(header, 24) == 24000, "sample rate");
+    // byte rate = rate * channels * bytes per sample
+    check(read_u32(header, 28) == 24000 * 1 * 2, "byte rate");
+    check(read_u16(header, 32) == 2, "block align for mono 16 bit");
+    check(read_u16(header, 34) == 16, "16 bits per sample");
+}
+
+// The field-wise checks above can all pass while the fields sit in the wrong
+// ORDER, since several of them hold the same value. This pins the whole 44 byte
+// string against a literal transcribed from the layout in pkg/audio/audio.go,
+// which is the struct binary.Write serializes for every Go LocalAI backend.
+// Independent of the implementation: it was written out by hand rather than
+// captured from a run.
+static void test_exact_bytes_match_the_go_layout() {
+    const std::string expected =
+        "52494646"          // "RIFF"
+        "ffffffff"          // chunk size: streaming sentinel
+        "57415645"          // "WAVE"
+        "666d7420"          // "fmt "
+        "10000000"          // subchunk1 size 16
+        "0100"              // audio format 1 (PCM)
+        "0100"              // channels 1
+        "c05d0000"          // sample rate 24000
+        "80bb0000"          // byte rate 48000
+        "0200"              // block align 2
+        "1000"              // bits per sample 16
+        "64617461"          // "data"
+        "ffffffff";         // subchunk2 size: streaming sentinel
+    check(to_hex(streaming_wav_header(24000, 1)) == expected,
+          "byte for byte match with the canonical mono 24 kHz header");
+}
+
+// The whole point: a streaming header cannot know the final length, so both
+// size fields are the sentinel. A client that sees a real size stops early.
+static void test_streaming_sentinels() {
+    const std::string header = streaming_wav_header(16000, 1);
+    check(read_u32(header, 4) == 0xFFFFFFFFu, "RIFF chunk size is the sentinel");
+    check(read_u32(header, 40) == 0xFFFFFFFFu, "data chunk size is the sentinel");
+}
+
+static void test_stereo() {
+    const std::string header = streaming_wav_header(44100, 2);
+    check(read_u16(header, 22) == 2, "stereo channel count");
+    check(read_u32(header, 28) == 44100 * 2 * 2, "stereo byte rate");
+    check(read_u16(header, 32) == 4, "block align for stereo 16 bit");
+    check(header.size() == 44, "stereo header is still 44 bytes");
+}
+
+static void test_degenerate_inputs() {
+    // A zero or negative channel count must not produce a header that divides
+    // by zero downstream; clamp to mono.
+    const std::string zero_channels = streaming_wav_header(16000, 0);
+    check(read_u16(zero_channels, 22) == 1, "zero channels clamps to mono");
+    check(read_u16(zero_channels, 32) == 2, "zero channels still block aligns as mono");
+    check(read_u32(zero_channels, 28) == 32000, "zero channels byte rate is the mono one");
+
+    const std::string negative_channels = streaming_wav_header(16000, -3);
+    check(read_u16(negative_channels, 22) == 1, "negative channels clamps to mono");
+
+    // A channel count past the field's range must saturate rather than wrap:
+    // 65536 truncated to uint16 is 0, and a zero channel count writes a zero
+    // block align, which is what a reader divides the data size by.
+    const std::string too_many = streaming_wav_header(16000, 65536);
+    check(read_u16(too_many, 22) == 65535, "an out of range channel count saturates");
+    check(read_u16(too_many, 32) != 0, "an out of range channel count never writes a zero block align");
+
+    // A non-positive rate is written as zero rather than wrapping through the
+    // unsigned conversion: 0 is visibly wrong to whoever reads the header,
+    // 4294967295 looks like a plausible field nobody checks.
+    const std::string zero_rate = streaming_wav_header(0, 1);
+    check(read_u32(zero_rate, 24) == 0, "zero sample rate stays zero");
+    check(read_u32(zero_rate, 28) == 0, "zero sample rate yields a zero byte rate");
+    const std::string negative_rate = streaming_wav_header(-48000, 1);
+    check(read_u32(negative_rate, 24) == 0, "negative sample rate is clamped to zero");
+    check(read_u32(negative_rate, 28) == 0, "negative sample rate yields a zero byte rate");
+
+    // The sentinels are unconditional. A degenerate rate must not turn the
+    // stream into one a client thinks it can measure.
+    check(read_u32(zero_rate, 4) == 0xFFFFFFFFu, "degenerate input keeps the RIFF sentinel");
+    check(read_u32(zero_rate, 40) == 0xFFFFFFFFu, "degenerate input keeps the data sentinel");
+}
+
+// Large but legal: 384 kHz 8 channel would overflow a 16 bit byte rate and
+// must not overflow the 32 bit one either.
+static void test_large_but_legal() {
+    const std::string header = streaming_wav_header(384000, 8);
+    check(read_u32(header, 28) == 384000u * 8u * 2u, "high rate multichannel byte rate");
+    check(read_u16(header, 32) == 16, "high channel count block align");
+}
+
+int main() {
+    test_header_layout();
+    test_exact_bytes_match_the_go_layout();
+    test_streaming_sentinels();
+    test_stereo();
+    test_degenerate_inputs();
+    test_large_but_legal();
+    if (failures) {
+        fprintf(stderr, "%d check(s) failed\n", failures);
+        return 1;
+    }
+    fprintf(stderr, "all wav_header checks passed\n");
+    return 0;
+}

backend/cpp/run-unit-tests.sh

@@ -37,9 +37,19 @@ if [ -z "$JSON_INC" ]; then
 fi
 
 # Active source dirs only - exclude per-variant build copies, dev snapshots and
-# the vendored upstream llama.cpp tree.
+# the vendored upstream checkouts.
+#
+# Every upstream checkout needs its own -not -path, and audio.cpp is safe TODAY
+# only by luck: its 44 tests all put "test" at the FRONT (17 test-*.cpp, 27
+# test_*.cpp, zero *_test.cpp), so the glob below misses every one of them.
+# Nothing enforces that. This gate runs on every PR for every backend, and it
+# compiles each match as a STANDALONE translation unit with nothing but
+# nlohmann/json on the include path, so the day upstream adds or renames one
+# test to *_test.cpp the whole gate goes red repo-wide, on an Apache-2.0 file
+# nobody here wrote. Exclude it now rather than diagnose that later.
 mapfile -t tests < <(find "$ROOT" -name '*_test.cpp' \
     -not -path '*/llama.cpp/*' \
+    -not -path '*/audio.cpp/*' \
     -not -path '*-build/*' \
     -not -path '*-dev/*' \
     -not -path '*fallback*' | sort)