Rule 2 discarded any partial the known text merely started with. For a cumulative
family that is a duplicate; for an incremental family it is an ordinary short
fragment that happens to coincide with the start of the transcript, and it was
dropped, silently corrupting the text. Pure ASCII, no multi-byte character
anywhere: the fragments "pure ", "ascii ", "trans", "c", "ri", "p", "t" left the
client holding "pure ascii transcrit". Over 5,000 randomized traces per
transcript, 9.50% of pure-ASCII and 29.12% of French traces ended with the client
holding something other than final_result.text, with a 200 and no diagnostic.
Both incremental families emit fragments that small routinely, since nemotron_asr
cuts at a byte offset and vibevoice_asr at a common prefix.
Narrowing rule 2 to an exact repeat drives that to zero on all six transcripts
and changes no cumulative stream at all: 30,000 randomized cumulative traces are
byte-identical to the previous commit.
What rule 2 guarded was established from upstream rather than from its own
comment. The only duplicate any pinned family produces is voxtral_realtime's,
where process_available_stream_chunks feeds each event to the sink from inside
its loop and returns the last of the batch, so that event arrives twice with
byte-equal text. A duplicate is an exact repeat, so equality still covers it. The
case given up is a cumulative report that SHRINKS, which no pinned family can
produce: voxtral decodes a token vector that is only push_back'ed and cleared by
reset(), so within a stream it can only grow.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The first UTF-8 fix closed the cumulative half only. Rule 2 discards a fragment
the known text already starts with, and when that fragment is the LEAD BYTE of a
new character it looks exactly like a repeat of an older character beginning
with the same byte. It was discarded rather than held, its continuation bytes
then arrived alone and began the next delta, and utf8_complete_prefix_length
only ever inspected the trailing sequence, so a delta invalid at the FRONT went
out whole. Through a real Go proto.Unmarshal the review's four-character repro
gave 3 deltas, 2 unmarshal failures and a lost transcript.
Reachable from the incremental families, not only from voxtral: nemotron_asr's
decoder cuts at a byte offset and vibevoice_asr's common_prefix_size compares
bytes, so both split characters. Measured over 30,000 randomized incremental
traces, 53.28% of Japanese traces and 9.52% of French ones carried at least one
delta the Go runtime refuses.
Two changes. Rule 2 no longer judges a fragment that ends mid-character, so the
lead byte is held instead of swallowed and the character survives intact; the
cost is a few duplicated bytes in a shrinking cumulative report, which no pinned
family produces. release() additionally drops leading orphan continuation bytes,
so no delta can begin mid-character whatever the rules above it decide. Losing a
byte keeps the stream alive; emitting one ends the RPC and takes the
final_result with it.
Post-fix all 60,000 traces produce zero unmarshal failures, and the cumulative
streams plus both pure-ASCII incremental streams are byte-identical to the
previous commit, so nothing changed for the families already working.
The weight-dtype allow list moves to family_gate, where it is stdlib-only and
pinned by a test rather than only by a comment. Two comment citations corrected.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
TranscriptStreamResponse.delta is a proto3 string, whose wire format requires
valid UTF-8. voxtral_realtime reports its hypothesis as a concatenation of raw
token BYTES (tokenizer_text.cpp:171-183), so the cumulative difference between
two consecutive reports is eventually a lone continuation byte, and the C++
runtime serializes that with only a logged warning while the Go runtime refuses
to unmarshal it: the client loses the remaining deltas AND the final_result.
Measured on a trace of a non-ASCII sentence, 11 of 31 messages failed to
unmarshal and every accented character was lost. TranscriptDeltaTracker now
holds back an incomplete trailing sequence and merges it into the next fragment;
reconcile flushes it, which it always can because the final text is complete.
The same trace now unmarshals in full with zero failures.
A streaming buffer whose float count is not a whole number of frames is refused
rather than truncated. The integer division dropped the tail floats from the fed
audio and therefore from the transcript, with no diagnostic; vibevoice_asr
refuses the same thing from the other side of the call.
A supertonic GGUF whose weights are not f32 is refused at load. It reaches
ggml_concat with mismatched operand types and ggml_abort takes the whole backend
process down on the first request, so nothing downstream can report it: the
model loads, then every request kills the process. Attributed rather than
assumed, the unary TTS path aborts identically, and upstream records that
package as untested. The refusal names the orig package and says what to run
before deleting the guard.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
TTSStream leads with a streaming WAV header carrying 0xFFFFFFFF sizes, matching
the convention backend/go/vibevoice-cpp established, so an HTTP client can start
playback before the full PCM exists. Its chunks are read from
StreamEvent::named_audio_outputs and not audio_output: supertonic, omnivoice and
voxcpm2 all put their streamed audio there and leave audio_output empty until
the very end, so reading the obvious field yields a stream with no audio in it.
The finish_stream result is the family's own merged whole rather than a tail, so
it is emitted only when nothing was streamed.
Streaming transcription sends incremental deltas and degrades to a single delta
plus the final result on families that offer no streaming ASR, which is the same
message sequence with fewer deltas. The four streaming ASR families disagree on
what partial_text means: nemotron_asr, vibevoice_asr and higgs_audio_stt report
incremental fragments while voxtral_realtime reports the whole hypothesis and
reports it twice, so the reconciliation lives in one tested unit rather than in
the handler. nemotron_asr reports only through the stream event sink, and only
from inside finalize, so the audio driver installs one and clears it again
before returning: the session is cached and a sink left holding the caller's
frame is a use after free waiting for the next stream.
begin_stream is now the only implementation of the streaming state obligation,
prepare then start_stream. Streaming sessions are cached, and what clears the
previous stream is start_stream's reset; a family override that dropped it would
break every call site with no compile error, so there is one call site.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
