feat(llama-cpp): serve Qwen3-TTS through the llama.cpp backend (#11392)

* fix(config): do not read a TTS speaker-encoder mmproj as vision support Qwen3-TTS on llama-cpp ships an mmproj holding the speaker encoder and code predictor. VisionSupported() treated any non-empty MMProj as proof of image input, so every such model would be advertised as vision-capable. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): add TTS request option parsing helper Validates text and speaker reference presence and strictly parses the top_k / top_p per-request params, in a header with no llama.cpp or gRPC dependencies so the standalone C++ unit test gate picks it up. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): range-check the TTS top_k and top_p request params Format validation alone let NaN, infinity and out-of-range values through. The consumer copies both values into the audio generation input unconditionally and only guards its separate sampler assignment with "> 0", a test NaN also fails, so a NaN reached llama.cpp with the guard never firing. top_k must now be >= 0 and top_p must fall within 0.0 to 1.0 inclusive, with the bound written as a negated in-range test so NaN is rejected rather than silently accepted. Also cover the two checks the suite could not previously kill: the whole-string check in the float parser and the int32 range check. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump pin to f9e832c10 and carry the TTS server task Picks up ggml-org/llama.cpp#26254 (Qwen3-TTS via mtmd) and #26536 (the short-input audio chunk fix). Adds 0002-add-server-task-type-tts.patch, the server-side half of the still-draft #26603, so TTS runs through the slot scheduler instead of racing it. Remove that patch when #26603 merges. The patch is rebased on top of the score patch: its tokenize-switch hunk collided with the SERVER_TASK_TYPE_SCORE case, and its lone SRV_WRN call passes no variadic argument, which the macro cannot expand. The score patch itself needed no refresh. Also fixes fallout from the bump in grpc-server.cpp: upstream dropped the per-slot n_ctx argument from server_schema::eval_llama_cmpl_schema. Only the schema branch loses it, since forks predating the server-schema split still expect the old argument list. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): implement the TTS and TTSStream RPCs Both were declared in backend.proto but unimplemented. They now submit a SERVER_TASK_TYPE_TTS task and drain the response reader, the same shape PredictStream uses. The streaming path emits a leading sample_rate message and then raw PCM, because ModelTTSStream builds the WAV header itself; the non-streaming path emits a complete WAV to the requested dst. The streamed samples are converted from the pipeline's float32 to signed 16-bit first. MTMD_HELPER_GEN_AUDIO_OUTTYPE_PCM hands back floats, while the header ModelTTSStream writes announces 16-bit samples, so shipping the floats verbatim would decode as noise. prepare.sh and CMakeLists.txt now stage tts_request_options.h alongside the other grpc-server helpers, and register its standalone test with ctest the way passthrough_options_test is registered. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): mask non-codec tokens for Qwen3-TTS generation The Qwen3-TTS gen-audio pipeline maps a sampled backbone token to a codebook row with an unchecked subtraction, in mtmd-helper-gen.cpp: inp.code0 = sampled - codec_0; For ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF the vocab is 155008 tokens, <|codec_0|> is 151936 and the codec codes end at 153983. The model's own tokenizer.ggml.suppress_tokens holds 1023 ids covering 153984..155007, every special above the codec range except <|codec_eos_token|> (154086) which stays reachable as the stop token. Nothing masks the text range 0..151935, so the backbone can sample a text token at any step, the subtraction goes negative, and ggml_compute_forward_get_rows aborts the whole backend process on GGML_ASSERT(i01 >= 0 && i01 < ne01). Complete the mask upstream started: bias every token below <|codec_0|> to -INFINITY for TTS tasks so only codec codes and the codec EOS remain reachable. The biases are appended to task.params.sampling.logit_bias, which common_sampler_init already merges with the model's suppress tokens into one llama_sampler_init_logit_bias, so no sampler is added to the chain. Measured cost is 0.082 ms per sampled token and 1.16 MB, set against a forward pass in the multi-millisecond range. It lands in launch_slot_with_task rather than in a route handler so that llama.cpp's own POST /tts and LocalAI's TTS/TTSStream RPCs are both covered, and <|codec_0|> is resolved from the vocab rather than hardcoded so a model without it is left alone. This is reproducible with upstream's own llama-tts and no LocalAI code loaded, aborting at frame 55 on Q4_K_M and frame 71 on Q8_0, so it is neither a quantization artifact nor an artifact of the gRPC adapter. Two further defects in the same draft pipeline still prevent end-to-end audio; they are independent of this one and are recorded in the task report for an upstream bug report. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump pin to 9de0fcf2b and drop the TTS codec mask Upstream fixed the Qwen3-TTS abort in ggml-org/llama.cpp c8e03ce81 ("mtmd/ggml: add ggml_build_forward_order", #26649), landed one hour after the previous pin. ggml_build_forward_expand marks a tensor and all its ancestors for compute, so using it as a pure ordering hint defeated ggml_build_forward_select and made GEN_WAV calls execute the GEN_CODE branch against a stale inp_code0, hitting the get_rows bound assert in ggml_compute_forward_get_rows. That single defect accounts for every abort seen on this model, so 0003-mask-non-codec-tokens-for-tts.patch is removed rather than rebased. The mask changed the observed behavior, but it was perturbing a graph ordering bug rather than fixing a sampling one: at the new pin the whole path works without it. Keeping it would have meant carrying a 152k-entry logit bias, and rebasing it on every pin bump, for no benefit. Verified at 9de0fcf2b with only 0001 and 0002 applied, which both apply clean with no fuzz and needed no rebase: non-streaming HTTP 200, 410924 bytes, 8.56 s RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 24000 Hz streaming HTTP 200, 560684 bytes, 11.68 s, exactly one RIFF at byte 0, same format, which also exercises the float32-to-s16 conversion at runtime for the first time Pristine unpatched llama-tts at the same pin now also completes, 130 frames to a valid WAV, where it aborted at frame 55 before. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): clear the TTS slot sequence between requests Only the first TTS request in a backend process succeeded. Every later one failed instantly, in about 0.13 s, with "TTS prompt processing failed" from step_prompt, regardless of streaming or non-streaming and regardless of the text. With LOCALAI_SINGLE_ACTIVE_BACKEND=true the process is kept alive between requests, so a deployment would have served exactly one utterance per backend start. The cause is missing KV hygiene, not anything in the gRPC adapter. TTS slots never enter the shared batch: pre_decode() returns early for them and process_tts_slots() drives them instead, so they skip the prompt-cache bookkeeping that clears a slot's sequence between requests. Nothing in the gen-audio path makes up for it: mtmd_helper_gen_audio_reset only clears host-side buffers, and the pipeline always decodes from position 0 into the sequence identified by slot.id. So the second task on a slot writes positions 0..N over the first task's tokens and llama_decode fails. Fix is one call to slot.prompt_clear(), the same helper the normal path uses, in the SERVER_TASK_TYPE_TTS branch of launch_slot_with_task before set_input. It goes into 0002 rather than a new patch file because it is a defect in the code that patch introduces, and the header now records it as ours so we know whether it still needs carrying if #26603 merges without it. Verified in one backend process, different text on every request: three consecutive non-streaming requests, three consecutive streaming requests, and an interleaved non-streaming, streaming, non-streaming, streaming run. All ten returned HTTP 200 with RIFF ... WAVE audio, Microsoft PCM, 16 bit, mono 24000 Hz, the streamed ones carrying exactly one RIFF header at byte 0, and every output measured as real speech rather than silence or a truncated fragment. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): expose max_frames for TTS requests The Qwen3-TTS backbone does not always emit <|codec_eos_token|>, and when it does not, generation runs to upstream's 512-frame n_predict default. At the model's 12.5 Hz frame rate that is 40.96 s of audio, which a short input can trigger: one request in this session produced 40.96 s for a ten-word sentence. prepareTTSTask hardcoded n_predict to -1, so callers had no way to bound it. Add a max_frames key alongside top_k and top_p, parsed with the same strict whole-string parsing so a typo is an error rather than a silently truncated value, and rejected with a field-naming message when negative. 0 keeps the existing sentinel convention and means unset, so a request that omits it behaves exactly as before. Named max_frames rather than n_predict because frames are what the parameter means at a TTS endpoint: one frame is 0.08 s of audio. The 512-frame default is deliberately unchanged. Lowering it would truncate legitimately long inputs, which is a worse failure than an occasionally overlong one. Verified end to end on one text of thirty words: max_frames=25 HTTP 200, 96044 bytes, 2.00 s, exactly 25 frames max_frames=50 HTTP 200, 192044 bytes, 4.00 s, exactly 50 frames no max_frames HTTP 200, 572204 bytes, 11.92 s, stopped at its own codec EOS after 149 frames, unchanged behavior max_frames=-1 InvalidArgument "max_frames must be >= 0, got \"-1\"" max_frames=many InvalidArgument "max_frames must be an integer, got \"many\"" Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): send the TTS sample rate up front, and tidy three review items Four items from the Task 4 review. Streaming first-byte latency. TTSStream sent the sample-rate reply only once the first audio result arrived, and a chunk needs a whole 72-frame window, roughly 5.8 s of audio and far longer in wall time on CPU. The Go side blocks on that reply before it can emit the WAV header, so a streaming client sat at zero bytes for the whole stretch. The rate is a property of the loaded model and is available synchronously from mtmd_gen_audio_get_info, so it now goes out immediately after post_task and the rate_sent bookkeeping is gone. Measured on a warm model, first byte drops from 30.48 s to 0.014 s, and the output is still a valid WAV with exactly one RIFF header at byte 0. Unchecked close. The non-streaming path ignored ofstream::close(), so a failure that only surfaces on flush was reported as success while leaving a truncated file at dst. It now returns INTERNAL like the other write failures. Wrong comment on set_lang. gen_audio::inp::get() already maps a stored blank to nullptr, so our guard is behavior-preserving, not behavior-fixing. The comment claimed otherwise; the code was right. Repetition penalty. penalty_last_n = -1 is inert at this pin, because llama_sampler_init_penalties clamps it with std::max(penalty_last_n, 0) and then builds a disabled sampler, so the 1.05 penalty never applies. Upstream's README attributes looping to a missing repeat_penalty, so it was worth testing as a root-cause fix for the model running to the frame cap. Dropping the line lets the sampling default of 64 apply, which was confirmed in the sampler chain trace as penalty_last_n = 64 with repeat_penalty = 1.050. Over 15 uncapped short requests each way it did not help: 0 of 15 ran to the cap with the penalty inert, 1 of 15 with it active. Both lines are therefore kept for parity with upstream's draft, and a comment now records that the pair is inert and why, so the next reader does not believe a penalty is applied. max_frames remains the way to bound output. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * build(llama-cpp): let unpatched forks opt out of the TTS task turboquant and bonsai copy grpc-server.cpp into llama.cpp forks that do not carry our patches. disable-tts-task.sh injects the same kind of preprocessor switch disable-score-task.sh already uses, so those builds answer UNIMPLEMENTED rather than failing to compile. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(config): keep a TTS speaker-encoder projector out of vision detection Task 1 exempted a declared-TTS model's mmproj from VisionSupported, but the first real gallery entry with an mmproj still came back vision-capable through two paths the earlier fix did not close. GuessUsecases has no FLAG_VISION branch, so it falls through to true for any chat-ish model. That is not just a wrong answer at the call site: syncKnownUsecasesFromString rewrites KnownUsecaseStrings from HasUsecases, and the loader calls it more than once per config file, so the guessed FLAG_VISION is written out and parsed back into KnownUsecases as if the operator had declared it. Give GuessUsecases a FLAG_VISION branch that defers to the same explicit signals VisionSupported uses. Second, llama.cpp builds an mtmd context for the speaker-encoder projector and reports its media marker on the first chat probe, which resurrected vision after the model had been used once. Apply the same declared-TTS exemption to MediaMarker that the mmproj check already had. Verified against the qwen3-tts-llamacpp-q4 gallery entry: no vision capability and no image input modality, before load, after a TTS request, and after a chat probe. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add Qwen3-TTS entries for the llama-cpp backend Two entries over upstream's own GGUF conversion, Q8_0 and Q4_K_M, each pairing a backbone with the Q8_0 projector. Named to sit alongside the existing qwen3-tts-cpp entries rather than replace them. Also tags the llama-cpp backend text-to-speech / TTS so the backend browser surfaces the capability. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: cover Qwen3-TTS on the llama-cpp backend Adds the gallery variants, the two-file mmproj configuration, the required voice reference, and the language and sampling knobs. Also corrects the streaming-support list, which named only voxcpm. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(config): register llama-cpp as a TTS and voice-cloning backend The branch taught the llama-cpp backend to serve Qwen3-TTS and shipped two gallery entries for it, but never told the capability table. llama-cpp still declared only the text RPCs and usecases, so: - VoiceCloningForModel returned nil at the capability check, before it ever reached the model's own tts.voice_cloning override, and /tts answered 400 "selected model does not support reference-audio voice cloning" for any localai://voice-profiles/... voice. No model YAML could opt back in. - GET /api/backends/usecases did not list tts for llama-cpp, so the gallery greyed out the TTS filter for the entries this branch adds. - The React TTS page saw voice_cloning: null and kept both models out of the Voice Library. Add the TTS RPCs and usecase, and the reference-audio contract. The contract needs narrowing, because the per-backend switch in VoiceCloningForModel ends in a permissive default: an unnarrowed entry would have advertised reference-audio cloning on every GGUF chat model in the gallery. Narrow on the declared TTS usecase rather than the model name. The TTS checkpoints are the only llama-cpp models carrying known_usecases: [tts]; name matching would have to guess at third-party repacks, and "base", the substring the neighbouring Qwen and vLLM cases key on, is a routine word in text-model names. The check reads the declared bit directly instead of going through HasUsecases, which falls through to GuessUsecases and would hand the decision to a heuristic that never had a llama.cpp TTS model in mind. DefaultUsecases stays [chat]: a bare GGUF served by llama.cpp is a chat model, and both the gallery filter and the importer read that field. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): declare what nemotron-3-nano-omni actually accepts The entry is backend: vllm-omni with known_usecases: [chat, completion], no mmproj and no media marker, so it used to report vision only through the blanket GuessUsecases fallthrough that the vision branch in this branch removed. Nemotron 3 Nano Omni is a multimodal understanding model: image, video and audio in, text out. Declaring that is what the sibling vllm-omni-qwen3-omni-30b already does. known_usecases gains vision only. FLAG_VIDEO is video GENERATION, an output modality, and this model generates none; video and audio input belong in known_input_modalities, which is where AudioInputSupported and VideoInputSupported read them from. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(importers): import a Qwen3-TTS GGUF repo as TTS, not chat The llama-cpp importer hardcodes known_usecases: [chat] and assigns any mmproj-matching file as a vision projector, so ggml-org/Qwen3-TTS-12Hz-1.7B- Base-GGUF imported as a chat model with vision. Both fields were wrong, and the model was unreachable from /tts and from the Voice Library. Filenames cannot fix this. A Qwen3-TTS repo has the exact shape of a vision repo, one backbone GGUF plus one mmproj-*.gguf, so the projector's own header is the only honest signal: mtmd writes clip.has_gen_audio_encoder for the projectors it can drive as a speech pipeline and refuses to build one without it. Probe the selected mmproj for that flag, reusing the range-fetch the MTP detection already does, and declare tts when it is set. The mmproj assignment then stops reading as vision on its own, since a declared-TTS model already exempts its projector from vision detection. The probe is best-effort like the MTP one: a network blip leaves the chat default in place rather than failing the import. Verified against the real artifacts on disk: the Qwen3-TTS projector reports gen-audio, its backbone does not. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): stop non-TTS models crashing on the new pin Two regressions, both hit every ordinary llama-cpp model and neither was caught locally because every test on this branch loaded a TTS model. The first is a null dereference. server_slot::tts_ctx::reset() called mtmd_helper_gen_audio_reset() unconditionally, but the gen-audio pipeline is only allocated for models carrying a gen-audio mmproj, and upstream's implementation reads ctx->pipeline before null-checking anything. Since server_slot::reset() runs during slot initialization for every model, any non-TTS model segfaulted the backend the moment it loaded. Guard the call on the is_supported() predicate already defined beside it, and keep the plain field resets unconditional. The second is unrelated to TTS and came in with the pin bump. PredictOptions.Penalty is a bare proto float, so a caller that names no repetition penalty sends 0 rather than omitting the field. Since 9de0fcf2b, common_sampler_init() rejects a non-positive penalty_repeat outright because it would divide logits by zero, turning every such request into "Failed to initialize samplers". Treat 0 as unset and leave llama.cpp's own neutral default in place. Verified with the same suite CI runs, which is what caught both: tests/e2e-backends passes 6 of 6 including the load and predict specs that were red. Qwen3-TTS still synthesises on both paths, 24 kHz mono 16-bit WAV with exactly one RIFF header on the streamed output. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-12 22:33:54 -04:00 · 2026-08-10 10:18:47 +02:00
1 parent 7a22586399
commit 7b9167eaad
@@ -55,6 +55,7 @@
 #include "llama_compat.h" // fork-skew switches, generated by prepare.sh
 #include "message_content.h"
 #include "passthrough_options.h"
+#include "tts_request_options.h"
 #include <getopt.h>
 #include <grpcpp/ext/proto_server_reflection_plugin.h>
 #include <grpcpp/grpcpp.h>
@@ -65,6 +66,7 @@
 #include <atomic>
 #include <cmath>
 #include <cstdlib>
+#include <cstring>
 #include <fstream>
 #include <iterator>
 #include <list>
@@ -233,7 +235,15 @@ json parse_options(bool streaming, const backend::PredictOptions* predict, const
    data["typical_p"] = predict->typicalp();
    data["temperature"] = predict->temperature();
    data["repeat_last_n"] = predict->repeat();
-    data["repeat_penalty"] = predict->penalty();
+    // PredictOptions.Penalty is a bare proto float, so a caller that names no
+    // repetition penalty sends 0 rather than omitting the field. Since
+    // llama.cpp 9de0fcf2b, common_sampler_init() rejects a non-positive
+    // penalty_repeat outright (it would divide logits by zero), which turned
+    // every such request into "Failed to initialize samplers". Treat 0 as
+    // "unset" and leave llama.cpp's own neutral default in place.
+    if (predict->penalty() > 0.0f) {
+        data["repeat_penalty"] = predict->penalty();
+    }
    data["frequency_penalty"] = predict->frequencypenalty();
    data["presence_penalty"] = predict->presencepenalty();
    data["mirostat"] = predict->mirostat();
@@ -1445,6 +1455,26 @@ static void params_parse(server_context& /*ctx_server*/, const backend::ModelOpt
 }


+#ifndef LOCALAI_LLAMA_CPP_NO_TTS_TASK
+// MTMD_HELPER_GEN_AUDIO_OUTTYPE_PCM hands back raw float32 samples, but the
+// WAV header core/backend/tts.go builds around the streamed chunks announces
+// 16-bit samples, so the wire has to carry s16 or the client decodes floats as
+// integers and hears noise. The scaling matches write_wav16() in
+// tools/mtmd/mtmd-helper-gen.cpp, which is what the non-streaming path writes.
+static std::string tts_pcm_f32_to_s16(const std::string & samples) {
+    const size_t n = samples.size() / sizeof(float);
+    std::string out;
+    out.resize(n * sizeof(int16_t));
+    for (size_t i = 0; i < n; i++) {
+        float v = 0.0f;
+        std::memcpy(&v, samples.data() + i * sizeof(float), sizeof(float));
+        const int16_t s = (int16_t) (std::max(-1.0f, std::min(1.0f, v)) * 32767.0f);
+        std::memcpy(&out[i * sizeof(int16_t)], &s, sizeof(int16_t));
+    }
+    return out;
+}
+#endif
+
 // GRPC Server start
 class BackendServiceImpl final : public backend::Backend::Service {
 private:
@@ -2089,15 +2119,23 @@ public:

                task.tokens    = std::move(inputs[i]);
 #ifdef LOCALAI_HAS_SERVER_SCHEMA
+                // The schema evaluator no longer takes the per-slot n_ctx: upstream
+                // dropped the parameter and server-schema stopped consulting n_ctx at
+                // all, leaving the context bound to the slot. Forks that predate the
+                // server-schema split still expect it, so only this branch loses it.
                task.params           = server_schema::eval_llama_cmpl_schema(
+                        ctx_server.impl->vocab,
+                        params_base,
+                        ctx_server.get_meta().logit_bias_eog,
+                        data);
 #else
                task.params           = server_task::params_from_json_cmpl(
-#endif
                        ctx_server.impl->vocab,
                        params_base,
                        ctx_server.get_meta().slot_n_ctx,
                        ctx_server.get_meta().logit_bias_eog,
                        data);
+#endif
                task.id_slot = json_value(data, "id_slot", -1);

                // OAI-compat: enable autoparser (PEG-based chat parsing) so that
@@ -2659,15 +2697,23 @@ public:

                task.tokens    = std::move(inputs[i]);
 #ifdef LOCALAI_HAS_SERVER_SCHEMA
+                // The schema evaluator no longer takes the per-slot n_ctx: upstream
+                // dropped the parameter and server-schema stopped consulting n_ctx at
+                // all, leaving the context bound to the slot. Forks that predate the
+                // server-schema split still expect it, so only this branch loses it.
                task.params           = server_schema::eval_llama_cmpl_schema(
+                        ctx_server.impl->vocab,
+                        params_base,
+                        ctx_server.get_meta().logit_bias_eog,
+                        data);
 #else
                task.params           = server_task::params_from_json_cmpl(
-#endif
                        ctx_server.impl->vocab,
                        params_base,
                        ctx_server.get_meta().slot_n_ctx,
                        ctx_server.get_meta().logit_bias_eog,
                        data);
+#endif
                task.id_slot = json_value(data, "id_slot", -1);

                // OAI-compat: enable autoparser (PEG-based chat parsing) so that
@@ -2994,6 +3040,229 @@ public:
        return grpc::Status::OK;
    }

+#ifndef LOCALAI_LLAMA_CPP_NO_TTS_TASK
+    // Builds the shared TTS task from a request. Returns a non-OK status and
+    // leaves `task` untouched when the request is malformed or the loaded model
+    // cannot synthesise audio.
+    grpc::Status prepareTTSTask(const backend::TTSRequest* request, bool stream, server_task & task) {
+        if (!ctx_server.get_meta().has_cap_tts) {
+            return grpc::Status(grpc::StatusCode::UNIMPLEMENTED,
+                "the loaded model does not support audio generation (no gen-audio mmproj)");
+        }
+
+        std::map<std::string, std::string> params(request->params().begin(), request->params().end());
+        const auto opts = llama_grpc::parse_tts_request_options(
+            request->text(),
+            request->voice(),
+            request->has_language() ? request->language() : std::string(),
+            params);
+        if (!opts.ok) {
+            return grpc::Status(grpc::StatusCode::INVALID_ARGUMENT, opts.error);
+        }
+
+        auto wrapper = mtmd_helper_bitmap_init_from_file(ctx_server.impl->mctx, opts.voice_path.c_str(), false);
+        if (!wrapper.bitmap) {
+            return grpc::Status(grpc::StatusCode::INVALID_ARGUMENT,
+                "failed to read speaker reference audio: " + opts.voice_path);
+        }
+
+        task.tts_inp.set_prompt(opts.text);
+        // core/backend/tts.go always sets TTSRequest.language, so has_language()
+        // is true even when the caller named no language and the string is empty.
+        // gen_audio::inp::get() already maps a stored blank to nullptr, so this
+        // guard is behavior-preserving rather than behavior-fixing. It is kept
+        // so the "unset" intent is visible at the call site instead of resting
+        // on a detail of the helper.
+        if (!opts.language.empty()) {
+            task.tts_inp.set_lang(opts.language);
+        }
+        task.tts_inp.set_speaker_ref(mtmd::bitmap_ptr(wrapper.bitmap));
+        task.tts_inp.data.top_k    = opts.top_k;
+        task.tts_inp.data.top_p    = opts.top_p;
+        task.tts_inp.data.stream   = stream;
+        task.tts_inp.data.out_type = stream
+            ? MTMD_HELPER_GEN_AUDIO_OUTTYPE_PCM   // Go prepends its own WAV header, see core/backend/tts.go
+            : MTMD_HELPER_GEN_AUDIO_OUTTYPE_WAV;
+
+        task.params.stream    = stream;
+        // -1 keeps upstream's 512-frame default. The model does not always emit
+        // its codec EOS, so a short input can otherwise generate the full cap.
+        task.params.n_predict = opts.max_frames > 0 ? opts.max_frames : -1;
+        task.params.sampling  = params_base.sampling;
+        // Both values mirror upstream's draft POST /tts handler. Note that the
+        // pair is INERT at this pin: llama_sampler_init_penalties() clamps
+        // penalty_last_n with std::max(penalty_last_n, 0), so -1 means "off",
+        // not "the whole generation", and the penalty sampler is then built
+        // disabled. No repetition penalty is actually applied.
+        //
+        // That is deliberate. Dropping the second line lets the sampling
+        // default of 64 apply and genuinely engages the 1.05 penalty, which was
+        // measured here against the model's habit of never emitting its codec
+        // EOS and running to the frame cap: 0 of 15 short requests ran away
+        // with the penalty inert, 1 of 15 with it active over the last 64
+        // tokens. It does not fix the runaway, so the line stays for parity
+        // with the draft. Use max_frames to bound the output instead.
+        task.params.sampling.penalty_repeat = 1.05f;
+        task.params.sampling.penalty_last_n = -1;
+        if (opts.top_k > 0) {
+            task.params.sampling.top_k = opts.top_k;
+        }
+        if (opts.top_p > 0) {
+            task.params.sampling.top_p = opts.top_p;
+        }
+
+        return grpc::Status::OK;
+    }
+
+    grpc::Status TTS(ServerContext* context, const backend::TTSRequest* request, backend::Result* result) override {
+        auto auth = checkAuth(context);
+        if (!auth.ok()) return auth;
+        auto identity = checkModelIdentity(request);
+        if (!identity.ok()) return identity;
+        if (params_base.model.path.empty()) {
+            return grpc::Status(grpc::StatusCode::FAILED_PRECONDITION, "Model not loaded");
+        }
+        if (request->dst().empty()) {
+            return grpc::Status(grpc::StatusCode::INVALID_ARGUMENT, "dst must name an output file path");
+        }
+
+        server_task task(SERVER_TASK_TYPE_TTS);
+        auto prepared = prepareTTSTask(request, /* stream= */ false, task);
+        if (!prepared.ok()) return prepared;
+
+        auto rd = ctx_server.get_response_reader();
+        task.id = rd.get_new_id();
+        rd.post_task(std::move(task));
+
+        auto should_stop = [context]() { return context->IsCancelled(); };
+
+        std::string audio;
+        while (true) {
+            auto res = rd.next(should_stop);
+            if (!res) {
+                return grpc::Status(grpc::StatusCode::CANCELLED, "TTS request cancelled");
+            }
+            if (res->is_error()) {
+                return grpc::Status(grpc::StatusCode::INTERNAL, res->to_json().dump());
+            }
+            auto * tts_res = dynamic_cast<server_task_result_tts *>(res.get());
+            if (tts_res == nullptr) {
+                return grpc::Status(grpc::StatusCode::INTERNAL, "unexpected result type for a TTS task");
+            }
+            audio.append(tts_res->audio);
+            if (tts_res->final) {
+                break;
+            }
+        }
+
+        std::ofstream out(request->dst(), std::ios::binary | std::ios::trunc);
+        if (!out) {
+            return grpc::Status(grpc::StatusCode::INTERNAL, "failed to open output file: " + request->dst());
+        }
+        out.write(audio.data(), (std::streamsize) audio.size());
+        if (!out) {
+            return grpc::Status(grpc::StatusCode::INTERNAL, "failed to write output file: " + request->dst());
+        }
+        // Buffered data is flushed here, so a full disk or a failing device can
+        // surface for the first time on close. Reporting success then would
+        // leave a truncated file behind under the name the caller will read.
+        out.close();
+        if (!out) {
+            return grpc::Status(grpc::StatusCode::INTERNAL, "failed to close output file: " + request->dst());
+        }
+
+        result->set_success(true);
+        result->set_message("TTS audio generated");
+        return grpc::Status::OK;
+    }
+
+    grpc::Status TTSStream(ServerContext* context, const backend::TTSRequest* request, grpc::ServerWriter<backend::Reply>* writer) override {
+        auto auth = checkAuth(context);
+        if (!auth.ok()) return auth;
+        auto identity = checkModelIdentity(request);
+        if (!identity.ok()) return identity;
+        if (params_base.model.path.empty()) {
+            return grpc::Status(grpc::StatusCode::FAILED_PRECONDITION, "Model not loaded");
+        }
+
+        server_task task(SERVER_TASK_TYPE_TTS);
+        auto prepared = prepareTTSTask(request, /* stream= */ true, task);
+        if (!prepared.ok()) return prepared;
+
+        auto rd = ctx_server.get_response_reader();
+        task.id = rd.get_new_id();
+        rd.post_task(std::move(task));
+
+        auto should_stop = [context]() { return context->IsCancelled(); };
+
+        // core/backend/tts.go:ModelTTSStream builds the WAV header itself from
+        // the sample rate in the first reply's Message, then concatenates every
+        // Reply.Audio verbatim. So the rate goes out once, up front, and the
+        // chunks stay raw PCM.
+        //
+        // Send it before draining rather than off the first audio result: a
+        // chunk needs a whole 72-frame window, about 5.8 s of audio and far
+        // longer in wall time on CPU, and the Go side cannot emit the WAV
+        // header until this reply lands. Waiting would hold the client at zero
+        // bytes for that entire stretch. The rate is a property of the loaded
+        // model, available synchronously, so there is nothing to wait for.
+        {
+            backend::Reply header;
+            const json info = { {"sample_rate", mtmd_gen_audio_get_info(ctx_server.impl->mctx).sample_rate} };
+            header.set_message(info.dump());
+            if (!writer->Write(header)) {
+                return grpc::Status(grpc::StatusCode::CANCELLED, "client closed the TTS stream");
+            }
+        }
+
+        while (true) {
+            auto res = rd.next(should_stop);
+            if (!res) {
+                return grpc::Status(grpc::StatusCode::CANCELLED, "TTS request cancelled");
+            }
+            if (res->is_error()) {
+                return grpc::Status(grpc::StatusCode::INTERNAL, res->to_json().dump());
+            }
+            auto * tts_res = dynamic_cast<server_task_result_tts *>(res.get());
+            if (tts_res == nullptr) {
+                return grpc::Status(grpc::StatusCode::INTERNAL, "unexpected result type for a TTS task");
+            }
+
+            if (!tts_res->audio.empty()) {
+                backend::Reply chunk;
+                chunk.set_audio(tts_pcm_f32_to_s16(tts_res->audio));
+                if (!writer->Write(chunk)) {
+                    return grpc::Status(grpc::StatusCode::CANCELLED, "client closed the TTS stream");
+                }
+            }
+
+            if (tts_res->final) {
+                break;
+            }
+        }
+
+        return grpc::Status::OK;
+    }
+#else
+    grpc::Status TTS(ServerContext* context, const backend::TTSRequest* request, backend::Result* result) override {
+        auto auth = checkAuth(context);
+        if (!auth.ok()) return auth;
+        (void) request;
+        (void) result;
+        return grpc::Status(grpc::StatusCode::UNIMPLEMENTED,
+            "TTS is unavailable in this llama.cpp fork backend");
+    }
+
+    grpc::Status TTSStream(ServerContext* context, const backend::TTSRequest* request, grpc::ServerWriter<backend::Reply>* writer) override {
+        auto auth = checkAuth(context);
+        if (!auth.ok()) return auth;
+        (void) request;
+        (void) writer;
+        return grpc::Status(grpc::StatusCode::UNIMPLEMENTED,
+            "TTSStream is unavailable in this llama.cpp fork backend");
+    }
+#endif
+
    // Score returns the model's joint log-probability of each candidate
    // continuation given a shared prompt.
    //