mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-31 18:38:23 -04:00
bc6fd03a4b
Segment text is not decoration. core/http/endpoints/openai/transcription.go
routes response_format text, srt, vtt and lrc through
schema.TranscriptionResponse, which builds the entire body out of
Segments[].Text and never reads the top-level text. So for those four
formats the segment text IS the response.
nemotron_asr emits one word_timestamp per SentencePiece token, and the
word boundary is carried as a LEADING SPACE on the piece ("So", "me",
" call"). join_words inserted a space unconditionally, so
response_format=text returned "So me call me na ture ," while the
correct sentence sat unread in the top-level field. The separator is now
chosen from the words themselves: whole words are space-joined, subword
pieces are concatenated, and one leading space anywhere selects the
latter. Concatenating the real nemotron pieces reproduces text_output
exactly, verified end to end.
This does not touch the top-level text, which is still text_output
verbatim. The rule that forbids deriving the transcript from the segments
is about the direction segments -> text; segment text has no source other
than its words.
Two smaller corrections in the same area:
timestamp_granularities ["word"] set only "word_timestamps", a key no
family in the pinned upstream reads. It now sets "return_timestamps",
which qwen3_asr does read and which both runs its forced aligner and
shortens its chunk window, so asking for word granularity no longer
silently returns nothing.
The request-option comment claimed more than it delivered. prompt,
translate and temperature are read by no ASR family, and are forwarded
only so a family adopting them works unchanged; the comment now says so
per key, and gives TranscriptRequest.diarize the same explicit treatment
threads already had.
Also: the shipping target now carries -Wall -Wextra -Wpedantic, which it
never did, so "the build is clean" starts meaning something; and
fill_transcript_result no longer swallows a null response pointer, since
answering OK with an empty transcript is the one failure mode this unit
exists to prevent.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
38 lines
1.8 KiB
C++
38 lines
1.8 KiB
C++
#pragma once
|
|
|
|
// Converts engine::runtime results into LocalAI proto messages. All of the
|
|
// non-trivial shaping lives in transcript_assembly, which is stdlib-only and
|
|
// unit tested; this unit is the thin engine-typed boundary around it.
|
|
|
|
#include "backend.pb.h"
|
|
|
|
#include "engine/framework/runtime/session.h"
|
|
|
|
namespace audiocpp_backend {
|
|
|
|
// Fills text, language, duration, segments and per-segment words.
|
|
//
|
|
// THE RULE: the top-level text is TaskResult.text_output verbatim. It is never
|
|
// derived from segments or words. audio.cpp carries transcript text in
|
|
// text_output and nowhere else: speech_segments, speaker_turns and
|
|
// word_timestamps carry spans and labels and no text at all. Deriving the
|
|
// transcript from them therefore returns an EMPTY text for every producer that
|
|
// reports segments without word timing, which real VibeVoice diarized ASR does.
|
|
// An earlier attempt at this backend shipped exactly that bug. assemble_transcript
|
|
// enforces the rule and is heavily tested; this unit's job is not to re-derive
|
|
// it but to not undo it at the proto boundary.
|
|
//
|
|
// `sample_rate` is the rate the result's spans are expressed in, which is the
|
|
// rate of the AudioBuffer that was handed to the session, NOT the rate of the
|
|
// file the caller uploaded. Those differ whenever read_audio_file resampled,
|
|
// which is why the handler passes the buffer's rate rather than the file's.
|
|
//
|
|
// Segments are replaced, not appended to, so a message filled twice does not
|
|
// accumulate. `out` must be non-null and is not checked; see the note at the
|
|
// top of the implementation for why that is not an oversight.
|
|
void fill_transcript_result(const engine::runtime::TaskResult &result,
|
|
int sample_rate, float duration_seconds,
|
|
backend::TranscriptResult *out);
|
|
|
|
} // namespace audiocpp_backend
|