mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-31 10:28:43 -04:00
6ee95ab01b
Adds result_map, the engine-to-proto boundary, and wires the offline transcription RPC. The handler branches on the ROUTED task: for Asr the request's prompt is whisper-style decoding context and becomes a request option, for Alignment the same field IS the transcript to align and becomes the text input. Routing has already decided which. The result text is TaskResult.text_output verbatim and is never derived from the segments. audio.cpp carries transcript text in text_output and nowhere else, so deriving it returns an empty transcript for every producer that reports segments without word timing. transcript_assembly already enforces that; this commit's job is not to undo it at the proto boundary, and result_map_ctest pins it there. read_audio_file now takes the sample rate the caller needs. Both file-fed speech handlers ask for 16 kHz mono, for two reasons: silero_vad and sortformer_diar refuse anything else outright, which turned an ordinary 44.1 kHz upload into INTERNAL, and nemotron_asr emits word timestamps in its own 16 kHz feature domain whatever the input was, so only a 16 kHz buffer makes the emitted nanoseconds right. Zero keeps the file's native rate and channels, which is what source separation will need. LoadedModel::check_can_serve answers a capability refusal before the lane is taken and before the input file is read. Routing is a pure read of the immutable capabilities, so a model that cannot serve an RPC no longer waits out somebody else's run to say so. VAD and Diarize use it too. Assisted-by: Claude:claude-opus-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
37 lines
1.7 KiB
C++
37 lines
1.7 KiB
C++
#pragma once
|
|
|
|
// Converts engine::runtime results into LocalAI proto messages. All of the
|
|
// non-trivial shaping lives in transcript_assembly, which is stdlib-only and
|
|
// unit tested; this unit is the thin engine-typed boundary around it.
|
|
|
|
#include "backend.pb.h"
|
|
|
|
#include "engine/framework/runtime/session.h"
|
|
|
|
namespace audiocpp_backend {
|
|
|
|
// Fills text, language, duration, segments and per-segment words.
|
|
//
|
|
// THE RULE: the top-level text is TaskResult.text_output verbatim. It is never
|
|
// derived from segments or words. audio.cpp carries transcript text in
|
|
// text_output and nowhere else: speech_segments, speaker_turns and
|
|
// word_timestamps carry spans and labels and no text at all. Deriving the
|
|
// transcript from them therefore returns an EMPTY text for every producer that
|
|
// reports segments without word timing, which real VibeVoice diarized ASR does.
|
|
// An earlier attempt at this backend shipped exactly that bug. assemble_transcript
|
|
// enforces the rule and is heavily tested; this unit's job is not to re-derive
|
|
// it but to not undo it at the proto boundary.
|
|
//
|
|
// `sample_rate` is the rate the result's spans are expressed in, which is the
|
|
// rate of the AudioBuffer that was handed to the session, NOT the rate of the
|
|
// file the caller uploaded. Those differ whenever read_audio_file resampled,
|
|
// which is why the handler passes the buffer's rate rather than the file's.
|
|
//
|
|
// Segments are replaced, not appended to, so a message filled twice does not
|
|
// accumulate.
|
|
void fill_transcript_result(const engine::runtime::TaskResult &result,
|
|
int sample_rate, float duration_seconds,
|
|
backend::TranscriptResult *out);
|
|
|
|
} // namespace audiocpp_backend
|