backend(audio-cpp): keep streaming deltas on UTF-8 boundaries, refuse dtypes that abort

TranscriptStreamResponse.delta is a proto3 string, whose wire format requires
valid UTF-8. voxtral_realtime reports its hypothesis as a concatenation of raw
token BYTES (tokenizer_text.cpp:171-183), so the cumulative difference between
two consecutive reports is eventually a lone continuation byte, and the C++
runtime serializes that with only a logged warning while the Go runtime refuses
to unmarshal it: the client loses the remaining deltas AND the final_result.
Measured on a trace of a non-ASCII sentence, 11 of 31 messages failed to
unmarshal and every accented character was lost. TranscriptDeltaTracker now
holds back an incomplete trailing sequence and merges it into the next fragment;
reconcile flushes it, which it always can because the final text is complete.
The same trace now unmarshals in full with zero failures.

A streaming buffer whose float count is not a whole number of frames is refused
rather than truncated. The integer division dropped the tail floats from the fed
audio and therefore from the transcript, with no diagnostic; vibevoice_asr
refuses the same thing from the other side of the call.

A supertonic GGUF whose weights are not f32 is refused at load. It reaches
ggml_concat with mismatched operand types and ggml_abort takes the whole backend
process down on the first request, so nothing downstream can report it: the
model loads, then every request kills the process. Attributed rather than
assumed, the unary TTS path aborts identically, and upstream records that
package as untested. The refusal names the orig package and says what to run
before deleting the guard.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
2026-07-26 23:12:16 +00:00
committed by localai-org-maint-bot
parent a996d7dd23
commit 356a41e10d
6 changed files with 514 additions and 25 deletions

View File

@@ -1365,6 +1365,15 @@ public:
// postprocessor runs over the merged buffer at the end. Audio
// already on the wire cannot be post-processed, so any streaming
// implementation has to accept this.
//
// This guard also depends on the pull loop having DRAINED the
// session. It does today: no family sets is_final on a pulled
// event, so the loop runs to nullopt. If one ever does, the loop
// breaks early, and omnivoice's finish_stream drains and merges the
// chunks that were never emitted (session.cpp:576) into the result
// this branch then discards, silently truncating the audio. Whoever
// teaches a family to end a pull stream early has to revisit the
// pair together.
if (!header_sent) {
if (result.audio_output.has_value()) {
write_audio(*result.audio_output);
@@ -1402,8 +1411,10 @@ public:
// that concatenates every delta must end up with the final text and must
// never see a prefix repeated. The families do not agree on what they report
// (nemotron_asr and vibevoice_asr send fragments, voxtral_realtime sends the
// whole hypothesis every time, and sends it twice), so the reconciliation
// lives in TranscriptDeltaTracker rather than here. See stream_delta.h.
// whole hypothesis and repeats the last event of each batch), and a delta
// must additionally be valid UTF-8 or the Go client cannot unmarshal it at
// all, so the reconciliation lives in TranscriptDeltaTracker rather than
// here. See stream_delta.h.
//
// THE OFFLINE FALLBACK. mode_candidates lets this RPC fall back to an
// offline route, so a family with no streaming ASR still answers rather than