mirror of https://github.com/mudler/LocalAI.git synced 2026-07-30 09:57:57 -04:00

Files

mudler's LocalAI [bot] 2d889e61a6 feat(backend): add magpie-tts-cpp text-to-speech backend (#11115 )

* feat(backend): add magpie-tts-cpp text-to-speech backend

Add a Go + purego backend wrapping the magpie-tts.cpp ggml port of NVIDIA's
Magpie TTS Multilingual 357M (encoder + autoregressive decoder over NanoCodec
tokens), producing 22.05 kHz mono audio in 5 baked voices (Aria, Jason, John,
Leo, Sofia; case-insensitive names or indices 0-4) across 9+ languages from a
single self-contained GGUF. Mirrors qwen3-tts-cpp / moss-tts-cpp: dlopen the
static-ggml shared library, bind the flat magpie_tts_capi_* C-API via purego
(no local C shim needed, the upstream .so exports it directly), and serve the
gRPC TTS + TTSStream methods behind base.SingleThread (the C context is not
reentrant across synthesize calls).

The backend CMakeLists translates the Makefile's -DGGML_{CUDA,METAL,VULKAN,HIP}
flags into upstream's MAGPIE_GGML_* toggles (upstream FORCE-overwrites the ggml
cache entries from those), pinned to magpie-tts.cpp v0.1.1
(e3f3dd1ebe22b64e7405f93b519f2d1930712568), which statically links ggml into
libmagpie-tts.so (ldd shows only system libs).

Wires the full registration: backend-matrix.yml (CPU amd64/arm64, CUDA 12/13,
Intel SYCL f16/f32, Vulkan amd64/arm64, ROCm, NVIDIA L4T + L4T CUDA 13, and
Darwin metal), backend/index.yaml metas and image entries, the root Makefile
build targets, the changed-backends backend-filter path mapping, the bump_deps
auto-bump matrix, a test-extra per-backend smoke job, the /backends/known
pref-only importer entry, the backend capabilities map (TTS + TTSStream, no
voice cloning), and the README / compatibility-table docs rows.

Verified locally: unit + e2e Ginkgo suites pass against the real q8_0 GGUF
(22.05 kHz mono WAV, RMS > 0.01), a live gRPC LoadModel + TTS round-trip
returns valid non-silent audio, and the pre-commit gates (make lint,
make test-coverage-check) pass, run manually with LOCALAI_TEST_HTTP_PORT
overriding the locally-occupied 9090.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* gallery: add magpie-tts-cpp model entries (q8_0 + f16)

Add the Magpie TTS Multilingual 357M GGUFs from mudler/magpie-tts.cpp-gguf to
the model gallery: q8_0 (~624 MB, near-lossless, fastest decode, recommended)
with an f16 (~784 MB) variant, both served by the magpie-tts-cpp backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* magpie-tts-cpp: bump pin to rewritten upstream v0.1.1 SHA

Upstream history was rewritten to purge accidentally committed build
artifacts; v0.1.1 now resolves to 6f7696cf.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

2026-07-25 08:38:48 +02:00

2.2 KiB

Raw Permalink Blame History

Magpie TTS C++ backend

This backend runs NVIDIA's Magpie TTS Multilingual 357M GGUF through magpie-tts.cpp, a from-scratch C++/ggml port (model + NanoCodec + tokenizer + G2P dictionaries in one self-contained GGUF, no Python at inference time). It generates 22.05 kHz mono speech in 5 baked voices across 9+ languages.

The library is loaded via purego (cgo-less dlopen) exactly like qwen3-tts-cpp / moss-tts-cpp; the flat C-API (magpie_tts_capi_*) is exported directly by the upstream shared library, so there is no local C shim.

Model configuration

The model path points at the single GGUF:

name: magpie-tts-cpp
backend: magpie-tts-cpp
parameters:
  model: magpie-tts-multilingual-357m-q8_0.gguf
known_usecases:
  - tts
options:
  - "speaker:Aria"   # optional default voice (Aria, Jason, John, Leo, Sofia, or 0-4)
  - "language:en"    # optional default language

GGUFs live at mudler/magpie-tts.cpp-gguf (q8_0 recommended: near-lossless, ~624 MB, fastest decode).

Voices and languages

Magpie has 5 baked speakers - Aria, Jason, John, Leo, Sofia - and no voice cloning. The request voice accepts the names case-insensitively or the indices 0-4; empty selects Aria. Languages: en, es, de, fr, it, pt-BR, hi, vi, ko, ar-AE, ar-SA, ar-MSA (case-insensitive; default en).

API example

curl http://localhost:8080/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "magpie-tts-cpp",
    "input": "Hello world, this is a test of the text to speech system.",
    "voice": "sofia",
    "language": "en"
  }' \
  --output speech.wav

Native end-to-end test

The labeled test loads a real GGUF, synthesizes WAVs (verifying rate, layout and non-silence), and exercises the streaming path:

make -C backend/go/magpie-tts-cpp magpie-tts-cpp

MAGPIETTS_MODEL=/path/to/magpie-tts-multilingual-357m-q8_0.gguf \
MAGPIETTS_LIBRARY=backend/go/magpie-tts-cpp/libgomagpiettscpp-fallback.so \
  go test ./backend/go/magpie-tts-cpp -ginkgo.label-filter=e2e

bash test.sh does the same and auto-downloads the q8_0 GGUF when MAGPIETTS_MODEL is unset.

2.2 KiB Raw Permalink Blame History