mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-31 02:18:50 -04:00
8d6cd6149a
Adds result_map, the engine-to-proto boundary, and wires the offline transcription RPC. The handler branches on the ROUTED task: for Asr the request's prompt is whisper-style decoding context and becomes a request option, for Alignment the same field IS the transcript to align and becomes the text input. Routing has already decided which. The result text is TaskResult.text_output verbatim and is never derived from the segments. audio.cpp carries transcript text in text_output and nowhere else, so deriving it returns an empty transcript for every producer that reports segments without word timing. transcript_assembly already enforces that; this commit's job is not to undo it at the proto boundary, and result_map_ctest pins it there. read_audio_file now takes the sample rate the caller needs. Both file-fed speech handlers ask for 16 kHz mono, for two reasons: silero_vad and sortformer_diar refuse anything else outright, which turned an ordinary 44.1 kHz upload into INTERNAL, and nemotron_asr emits word timestamps in its own 16 kHz feature domain whatever the input was, so only a 16 kHz buffer makes the emitted nanoseconds right. Zero keeps the file's native rate and channels, which is what source separation will need. LoadedModel::check_can_serve answers a capability refusal before the lane is taken and before the input file is read. Routing is a pure read of the immutable capabilities, so a model that cannot serve an RPC no longer waits out somebody else's run to say so. VAD and Diarize use it too. Assisted-by: Claude:claude-opus-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
56 lines
2.6 KiB
C++
56 lines
2.6 KiB
C++
#pragma once
|
|
|
|
// Thin wrappers over the framework's public audio IO. Engine-linked, so this
|
|
// unit is built and tested through the CMake target rather than by
|
|
// backend/cpp/run-unit-tests.sh. The pure part of the arithmetic these
|
|
// wrappers feed lives in audio_units, which is stdlib-only and does have a
|
|
// standalone test.
|
|
|
|
#include "engine/framework/runtime/session.h"
|
|
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
namespace audiocpp_backend {
|
|
|
|
// Reads a WAV file. Throws ConfigError when the file is missing, is not
|
|
// readable as WAV, or declares a non-positive sample rate: all three are
|
|
// user-fixable input problems rather than backend faults.
|
|
//
|
|
// A declared sample rate of zero is refused rather than passed on, because
|
|
// every downstream conversion in audio_units answers 0 for a non-positive rate.
|
|
// Accepting it would turn a corrupt header into a response full of zero
|
|
// timestamps, which reads as a real answer.
|
|
//
|
|
// `target_sample_rate` is the rate the CALLER needs, in Hz:
|
|
//
|
|
// 0 (or negative) keep the file's own rate and channel count.
|
|
// positive downmix to mono and resample to that rate. Resampling is
|
|
// skipped when the file already declares it, so passing the
|
|
// rate a route needs costs nothing on the common input.
|
|
//
|
|
// It is a parameter, and not a constant inside this function, because the
|
|
// routes that read audio do not agree on an answer. Speech routes want 16 kHz
|
|
// mono; source separation does not, and folding a 44.1 kHz stereo input to
|
|
// 16 kHz mono for demucs or roformer would destroy the very thing they separate
|
|
// (both refuse a rate other than their own outright). Making the caller name
|
|
// the rate keeps that decision where the route is known.
|
|
//
|
|
// Downmixing along with the resample is not an extra liberty: every family a
|
|
// positive rate is used for (silero_vad, sortformer_diar and every ASR family)
|
|
// begins by calling the same mixdown_interleaved_to_mono_average on whatever it
|
|
// is given. Doing it once here produces the identical samples and halves the
|
|
// buffer that is then moved through the request.
|
|
engine::runtime::AudioBuffer read_audio_file(const std::string &path,
|
|
int target_sample_rate);
|
|
|
|
// Writes 16-bit PCM WAV, creating parent directories. Throws ConfigError when
|
|
// the destination cannot be written.
|
|
void write_audio_file(const std::string &path,
|
|
const engine::runtime::AudioBuffer &audio);
|
|
|
|
engine::runtime::AudioBuffer buffer_from_mono(std::vector<float> samples,
|
|
int sample_rate);
|
|
|
|
} // namespace audiocpp_backend
|