mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 18:09:05 -04:00
7ce4b36837
Segment text is not decoration. core/http/endpoints/openai/transcription.go
routes response_format text, srt, vtt and lrc through
schema.TranscriptionResponse, which builds the entire body out of
Segments[].Text and never reads the top-level text. So for those four
formats the segment text IS the response.
nemotron_asr emits one word_timestamp per SentencePiece token, and the
word boundary is carried as a LEADING SPACE on the piece ("So", "me",
" call"). join_words inserted a space unconditionally, so
response_format=text returned "So me call me na ture ," while the
correct sentence sat unread in the top-level field. The separator is now
chosen from the words themselves: whole words are space-joined, subword
pieces are concatenated, and one leading space anywhere selects the
latter. Concatenating the real nemotron pieces reproduces text_output
exactly, verified end to end.
This does not touch the top-level text, which is still text_output
verbatim. The rule that forbids deriving the transcript from the segments
is about the direction segments -> text; segment text has no source other
than its words.
Two smaller corrections in the same area:
timestamp_granularities ["word"] set only "word_timestamps", a key no
family in the pinned upstream reads. It now sets "return_timestamps",
which qwen3_asr does read and which both runs its forced aligner and
shortens its chunk window, so asking for word granularity no longer
silently returns nothing.
The request-option comment claimed more than it delivered. prompt,
translate and temperature are read by no ASR family, and are forwarded
only so a family adopting them works unchanged; the comment now says so
per key, and gives TranscriptRequest.diarize the same explicit treatment
threads already had.
Also: the shipping target now carries -Wall -Wextra -Wpedantic, which it
never did, so "the build is clean" starts meaning something; and
fill_transcript_result no longer swallows a null response pointer, since
answering OK with an empty transcript is the one failure mode this unit
exists to prevent.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
86 lines
3.6 KiB
C++
86 lines
3.6 KiB
C++
#include "result_map.h"
|
|
|
|
#include "transcript_assembly.h"
|
|
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
namespace audiocpp_backend {
|
|
|
|
void fill_transcript_result(const engine::runtime::TaskResult &result,
|
|
int sample_rate, float duration_seconds,
|
|
backend::TranscriptResult *out) {
|
|
// No null guard on `out`, deliberately. gRPC always hands a handler a
|
|
// response message, so a null here would be a programming error in a
|
|
// caller, and a guard that returned quietly would answer the client with an
|
|
// untouched, empty transcript and an OK status. That is the same
|
|
// indistinguishable-from-silence failure the rest of this unit exists to
|
|
// prevent; crashing on the developer's machine is the cheaper outcome.
|
|
std::vector<Span> speech_segments;
|
|
speech_segments.reserve(result.speech_segments.size());
|
|
for (const auto &segment : result.speech_segments) {
|
|
speech_segments.push_back(
|
|
Span{segment.span.start_sample, segment.span.end_sample});
|
|
}
|
|
|
|
std::vector<SpeakerSpan> speaker_turns;
|
|
speaker_turns.reserve(result.speaker_turns.size());
|
|
for (const auto &turn : result.speaker_turns) {
|
|
speaker_turns.push_back(
|
|
SpeakerSpan{Span{turn.span.start_sample, turn.span.end_sample},
|
|
turn.speaker_id});
|
|
}
|
|
|
|
std::vector<WordSpan> words;
|
|
words.reserve(result.word_timestamps.size());
|
|
for (const auto &word : result.word_timestamps) {
|
|
words.push_back(
|
|
WordSpan{Span{word.span.start_sample, word.span.end_sample},
|
|
word.word});
|
|
}
|
|
|
|
// The ONLY read of transcript text in this function, and the only one there
|
|
// may ever be. See THE RULE in the header.
|
|
const std::string text =
|
|
result.text_output.has_value() ? result.text_output->text : std::string();
|
|
|
|
const AssembledTranscript assembled = assemble_transcript(
|
|
text, speech_segments, speaker_turns, words, sample_rate);
|
|
|
|
out->set_text(assembled.text);
|
|
// language has no source inside transcript_assembly, which is span-shaped
|
|
// only, so it is read straight off the engine result here. Left untouched
|
|
// when the family reported no text output at all: an empty string would be
|
|
// indistinguishable from a family that genuinely detected no language, and
|
|
// the field is documented as optional.
|
|
if (result.text_output.has_value()) {
|
|
out->set_language(result.text_output->language);
|
|
}
|
|
out->set_duration(duration_seconds);
|
|
|
|
// Cleared rather than appended to. A caller that fills the same message
|
|
// twice (a stream's final_result being rebuilt, say) would otherwise emit
|
|
// every segment twice, and the second call's ids would restart at 0 and
|
|
// collide with the first call's.
|
|
out->clear_segments();
|
|
for (const auto &segment : assembled.segments) {
|
|
auto *out_segment = out->add_segments();
|
|
out_segment->set_id(segment.id);
|
|
// NANOSECONDS. TranscriptSegment and TranscriptWord are the only
|
|
// messages in backend.proto that use them; VADSegment and DiarizeSegment
|
|
// are float seconds. assemble_transcript has already converted.
|
|
out_segment->set_start(segment.start_ns);
|
|
out_segment->set_end(segment.end_ns);
|
|
out_segment->set_text(segment.text);
|
|
out_segment->set_speaker(segment.speaker);
|
|
for (const auto &word : segment.words) {
|
|
auto *out_word = out_segment->add_words();
|
|
out_word->set_start(word.start_ns);
|
|
out_word->set_end(word.end_ns);
|
|
out_word->set_text(word.text);
|
|
}
|
|
}
|
|
}
|
|
|
|
} // namespace audiocpp_backend
|