mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-31 10:28:43 -04:00
2d889e61a6
* feat(backend): add magpie-tts-cpp text-to-speech backend
Add a Go + purego backend wrapping the magpie-tts.cpp ggml port of NVIDIA's
Magpie TTS Multilingual 357M (encoder + autoregressive decoder over NanoCodec
tokens), producing 22.05 kHz mono audio in 5 baked voices (Aria, Jason, John,
Leo, Sofia; case-insensitive names or indices 0-4) across 9+ languages from a
single self-contained GGUF. Mirrors qwen3-tts-cpp / moss-tts-cpp: dlopen the
static-ggml shared library, bind the flat magpie_tts_capi_* C-API via purego
(no local C shim needed, the upstream .so exports it directly), and serve the
gRPC TTS + TTSStream methods behind base.SingleThread (the C context is not
reentrant across synthesize calls).
The backend CMakeLists translates the Makefile's -DGGML_{CUDA,METAL,VULKAN,HIP}
flags into upstream's MAGPIE_GGML_* toggles (upstream FORCE-overwrites the ggml
cache entries from those), pinned to magpie-tts.cpp v0.1.1
(e3f3dd1ebe22b64e7405f93b519f2d1930712568), which statically links ggml into
libmagpie-tts.so (ldd shows only system libs).
Wires the full registration: backend-matrix.yml (CPU amd64/arm64, CUDA 12/13,
Intel SYCL f16/f32, Vulkan amd64/arm64, ROCm, NVIDIA L4T + L4T CUDA 13, and
Darwin metal), backend/index.yaml metas and image entries, the root Makefile
build targets, the changed-backends backend-filter path mapping, the bump_deps
auto-bump matrix, a test-extra per-backend smoke job, the /backends/known
pref-only importer entry, the backend capabilities map (TTS + TTSStream, no
voice cloning), and the README / compatibility-table docs rows.
Verified locally: unit + e2e Ginkgo suites pass against the real q8_0 GGUF
(22.05 kHz mono WAV, RMS > 0.01), a live gRPC LoadModel + TTS round-trip
returns valid non-silent audio, and the pre-commit gates (make lint,
make test-coverage-check) pass, run manually with LOCALAI_TEST_HTTP_PORT
overriding the locally-occupied 9090.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* gallery: add magpie-tts-cpp model entries (q8_0 + f16)
Add the Magpie TTS Multilingual 357M GGUFs from mudler/magpie-tts.cpp-gguf to
the model gallery: q8_0 (~624 MB, near-lossless, fastest decode, recommended)
with an f16 (~784 MB) variant, both served by the magpie-tts-cpp backend.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* magpie-tts-cpp: bump pin to rewritten upstream v0.1.1 SHA
Upstream history was rewritten to purge accidentally committed build
artifacts; v0.1.1 now resolves to 6f7696cf.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
70 lines
2.2 KiB
Markdown
70 lines
2.2 KiB
Markdown
# Magpie TTS C++ backend
|
|
|
|
This backend runs NVIDIA's **Magpie TTS Multilingual 357M** GGUF through
|
|
[magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp), a from-scratch
|
|
C++/ggml port (model + NanoCodec + tokenizer + G2P dictionaries in one
|
|
self-contained GGUF, no Python at inference time). It generates **22.05 kHz
|
|
mono** speech in 5 baked voices across 9+ languages.
|
|
|
|
The library is loaded via purego (cgo-less `dlopen`) exactly like
|
|
`qwen3-tts-cpp` / `moss-tts-cpp`; the flat C-API (`magpie_tts_capi_*`) is
|
|
exported directly by the upstream shared library, so there is no local C shim.
|
|
|
|
## Model configuration
|
|
|
|
The model path points at the single GGUF:
|
|
|
|
```yaml
|
|
name: magpie-tts-cpp
|
|
backend: magpie-tts-cpp
|
|
parameters:
|
|
model: magpie-tts-multilingual-357m-q8_0.gguf
|
|
known_usecases:
|
|
- tts
|
|
options:
|
|
- "speaker:Aria" # optional default voice (Aria, Jason, John, Leo, Sofia, or 0-4)
|
|
- "language:en" # optional default language
|
|
```
|
|
|
|
GGUFs live at
|
|
[mudler/magpie-tts.cpp-gguf](https://huggingface.co/mudler/magpie-tts.cpp-gguf)
|
|
(q8_0 recommended: near-lossless, ~624 MB, fastest decode).
|
|
|
|
## Voices and languages
|
|
|
|
Magpie has 5 baked speakers - `Aria`, `Jason`, `John`, `Leo`, `Sofia` - and no
|
|
voice cloning. The request `voice` accepts the names case-insensitively or the
|
|
indices `0`-`4`; empty selects Aria. Languages: `en`, `es`, `de`, `fr`, `it`,
|
|
`pt-BR`, `hi`, `vi`, `ko`, `ar-AE`, `ar-SA`, `ar-MSA` (case-insensitive;
|
|
default `en`).
|
|
|
|
## API example
|
|
|
|
```bash
|
|
curl http://localhost:8080/v1/audio/speech \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{
|
|
"model": "magpie-tts-cpp",
|
|
"input": "Hello world, this is a test of the text to speech system.",
|
|
"voice": "sofia",
|
|
"language": "en"
|
|
}' \
|
|
--output speech.wav
|
|
```
|
|
|
|
## Native end-to-end test
|
|
|
|
The labeled test loads a real GGUF, synthesizes WAVs (verifying rate, layout
|
|
and non-silence), and exercises the streaming path:
|
|
|
|
```bash
|
|
make -C backend/go/magpie-tts-cpp magpie-tts-cpp
|
|
|
|
MAGPIETTS_MODEL=/path/to/magpie-tts-multilingual-357m-q8_0.gguf \
|
|
MAGPIETTS_LIBRARY=backend/go/magpie-tts-cpp/libgomagpiettscpp-fallback.so \
|
|
go test ./backend/go/magpie-tts-cpp -ginkgo.label-filter=e2e
|
|
```
|
|
|
|
`bash test.sh` does the same and auto-downloads the q8_0 GGUF when
|
|
`MAGPIETTS_MODEL` is unset.
|